have cfgrib exit on skipping variable
raybellwaves opened this issue · comments
What happened?
Reading in ECMWF data on s3. Seeing
skipping variable: paramId==167 shortName='t2m'
Traceback (most recent call last):
File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 680, in build_dataset_components
dict_merge(variables, coord_vars)
File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 611, in dict_merge
raise DatasetBuildError(
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='heightAboveGround' value=Variable(dimensions=(), data=10.0) new_value=Variable(dimensions=(), data=2.0)
In my terminal but the code continues. I would like it to exit.
Not sure why it says Traceback is behaves more like a warning than as error
What are the steps to reproduce the bug?
import fsspec
import xarray as xr
file = fsspec.open_local("simplecache::s3://ecmwf-forecasts/20230727/00z/0p4-beta/oper/20230727000000-0h-oper-fc.grib2", s3={"anon": True}, filecache={"cache_storage": "/tmp/files"})
ds = xr.open_dataset(file, engine="cfgrib")
skipping variable: paramId==167 shortName='t2m'
Traceback (most recent call last):
File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 680, in build_dataset_components
dict_merge(variables, coord_vars)
File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 611, in dict_merge
raise DatasetBuildError(
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='heightAboveGround' value=Variable(dimensions=(), data=10.0) new_value=Variable(dimensions=(), data=2.0)
>>>
### Version
0.9.10.4
### Platform (OS and architecture)
mac
### Relevant log output
_No response_
### Accompanying data
_No response_
### Organisation
_No response_
I see the error is displayed via
https://github.com/ecmwf/cfgrib/blob/master/cfgrib/dataset.py#L656
Need to see how to pass that to xarray
Think i got it
import cfgrib
import sys
import fsspec
import xarray as xr
file = fsspec.open_local("simplecache::s3://ecmwf-forecasts/20230727/00z/0p4-beta/oper/20230727000000-0h-oper-fc.grib2", s3={"anon": True}, filecache={"cache_storage": "/tmp/files"})
ds = xr.open_dataset(file, engine="cfgrib")
try:
ds = xr.open_dataset(file, engine="cfgrib", backend_kwargs={"errors": "raise"})
except cfgrib.dataset.DatasetBuildError as e:
print(f"Error occurred: {e}")
sys.exit(1)
Thanks Ray, we're working on writing some proper documentation written for cfgrib - this is a good example of a hidden feature!