Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to interpret variables using cftime #117

Closed
TomNicholas opened this issue May 16, 2024 · 5 comments · Fixed by #122
Closed

Option to interpret variables using cftime #117

TomNicholas opened this issue May 16, 2024 · 5 comments · Fixed by #122

Comments

@TomNicholas
Copy link
Member

#68 (comment) explains how we don't currently have support for virtualizing encoded time variables.

To handle this it looks like we need to an another option to open_virtual_dataset, a cftime_vars kwarg, which specifies which variables should be interpreted using cftime. (I think cftime_vars should be a subset of loadable_vars.) This suggestion comes from how kerchunk deals with this problem:

https://fsspec.github.io/kerchunk/reference.html#kerchunk.combine.MultiZarrToZarr

”cf:{var}”, interpret the value of var using cftime, returning a datetime. 
These will be automatically re-encoded with cftime, unless you specify an “M8[*]” 
dtype for the coordinate, in which case a conversion will be attempted.

Originally posted by @dcherian in #73 (comment)

@jsignell
Copy link
Contributor

Just to say that I am taking a look at this.

@jbusecke
Copy link
Contributor

jbusecke commented Jun 3, 2024

Just wanted to add +1 here, since this is quite important for our work to bring virtual zarr access to ESGF (jbusecke/esgf-virtual-zarr-data-access#8). Please let me know if/where I could help test, and ill try to make some time.

@jsignell
Copy link
Contributor

jsignell commented Jun 4, 2024

This is still on my radar and I appreciate the nudge. The test is the one thing that I am confident in 😅 it's more the implementation that needs some thought.

@jsignell
Copy link
Contributor

jsignell commented Jun 4, 2024

I'm reading up on encodings now.

@TomNicholas
Copy link
Member Author

TomNicholas commented Jun 24, 2024

It's possible the proper way to implement this is actually to use xarray.decode_cf, which eventually sends variables through these very-relevant-looking CFDateTimeCoder decoding methods

https://github.com/pydata/xarray/blob/56209bd9a3192e4f1e82c21e5ffcf4c3bacaaae3/xarray/conventions.py#L288

see #157 for general idea of using xarray to handle CF conventions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants