wrong time encoding after padding

[xzenggit]

What happened:

If I open a netcdf with default settings (contain a daily time dimension) and then pad with hourly values, even the padded dataset shows hourly time values, the hourly values cannot be saved. I think this is due to the encoding, but I'm not sure how to fix it.

What you expected to happen:
I expected the final line of code give me

#array(['2000-01-01T00:00:00.000000000', '2000-01-01T01:00:00.000000000',
#      '2000-01-01T02:00:00.000000000', '2000-01-01T03:00:00.000000000',
#       '2000-01-01T04:00:00.000000000'], dtype='datetime64[ns]')

Instead, it outputs

#array(['2000-01-01T00:00:00.000000000', '2000-01-01T00:00:00.000000000',
#      '2000-01-01T00:00:00.000000000', '2000-01-01T00:00:00.000000000',
#      '2000-01-01T00:00:00.000000000'], dtype='datetime64[ns]')

Minimal Complete Verifiable Example:

import xarray as xr

time = pd.date_range("2000-01-01", freq="1D", periods=365 )
ds = xr.Dataset({"foo": ("time", np.arange(365)), "time": time})
ds.to_netcdf('test5.nc')

ds = xr.open_dataset('test5.nc')
ds.time.encoding

# padding
ds_hourly = ds.resample(time='1h').pad()
ds_hourly.time.values[0:5]
#array(['2000-01-01T00:00:00.000000000', '2000-01-01T01:00:00.000000000',
#      '2000-01-01T02:00:00.000000000', '2000-01-01T03:00:00.000000000',
#       '2000-01-01T04:00:00.000000000'], dtype='datetime64[ns]')

ds_hourly.to_netcdf('test6.nc')

# load the padded data file
ds_hourly_load = xr.open_dataset('test6.nc')
ds_hourly_load.time.values[0:5]
#array(['2000-01-01T00:00:00.000000000', '2000-01-01T00:00:00.000000000',
#      '2000-01-01T00:00:00.000000000', '2000-01-01T00:00:00.000000000',
#      '2000-01-01T00:00:00.000000000'], dtype='datetime64[ns]')

Anything else we need to know?:

Environment:
xarray version: '0.15.1'

Output of xr.show_versions()

[dcherian]

Hmm.. there is a similar issue with zarr on this repo. You probably need to del ds_hourly.time.encoding["units"]. We should issue a warning when the values cannot be roundtripped with the specified encoding.

[xzenggit]

Thanks a lot, @dcherian . del ds_hourly.time.encoding["units"] works!

We can also set ds_hourly.time.encoding = {} before saving the dataset to override previous encoding. Definitely agree with you. There should be a warning here. Thanks!

Hmm.. there is a similar issue with zarr on this repo. You probably need to del ds_hourly.time.encoding["units"]. We should issue a warning when the values cannot be roundtripped with the specified encoding.

[stale]

In order to maintain a list of currently relevant issues, we mark issues as stale after a period of inactivity

If this issue remains relevant, please comment here or remove the stale label; otherwise it will be marked as closed automatically

[kmuehlbauer]

This is fixed in current xarray and a meaningful warning is issued:

UserWarning: Times can't be serialized faithfully to int64 with requested units 'days since 
2000-01-01'. Serializing with units 'hours since 2000-01-01' instead. Set encoding['dtype'] to 
floating point dtype to serialize with units 'days since 2000-01-01'. Set encoding['units'] to 
'hours since 2000-01-01' to silence this warning .

array(['2000-01-01T00:00:00.000000000', '2000-01-01T01:00:00.000000000',
       '2000-01-01T02:00:00.000000000', '2000-01-01T03:00:00.000000000',
       '2000-01-01T04:00:00.000000000'], dtype='datetime64[ns]')