ecmwf / cfgrib

A Python interface to map GRIB files to the NetCDF Common Data Model following the CF Convention using ecCodes

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

have cfgrib exit on skipping variable

raybellwaves opened this issue · comments

What happened?

Reading in ECMWF data on s3. Seeing

skipping variable: paramId==167 shortName='t2m'
Traceback (most recent call last):
  File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 680, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 611, in dict_merge
    raise DatasetBuildError(
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='heightAboveGround' value=Variable(dimensions=(), data=10.0) new_value=Variable(dimensions=(), data=2.0)

In my terminal but the code continues. I would like it to exit.

Not sure why it says Traceback is behaves more like a warning than as error

What are the steps to reproduce the bug?

import fsspec
import xarray as xr

file = fsspec.open_local("simplecache::s3://ecmwf-forecasts/20230727/00z/0p4-beta/oper/20230727000000-0h-oper-fc.grib2", s3={"anon": True}, filecache={"cache_storage": "/tmp/files"})
ds = xr.open_dataset(file, engine="cfgrib")

skipping variable: paramId==167 shortName='t2m'
Traceback (most recent call last):
  File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 680, in build_dataset_components
    dict_merge(variables, coord_vars)
  File "/Users/ray/miniforge3/envs/wxtools/lib/python3.10/site-packages/cfgrib/dataset.py", line 611, in dict_merge
    raise DatasetBuildError(
cfgrib.dataset.DatasetBuildError: key present and new value is different: key='heightAboveGround' value=Variable(dimensions=(), data=10.0) new_value=Variable(dimensions=(), data=2.0)
>>> 


### Version

0.9.10.4

### Platform (OS and architecture)

mac

### Relevant log output

_No response_

### Accompanying data

_No response_

### Organisation

_No response_

I see the error is displayed via

https://github.com/ecmwf/cfgrib/blob/master/cfgrib/dataset.py#L656

Need to see how to pass that to xarray

Think i got it

import cfgrib
import sys
import fsspec
import xarray as xr

file = fsspec.open_local("simplecache::s3://ecmwf-forecasts/20230727/00z/0p4-beta/oper/20230727000000-0h-oper-fc.grib2", s3={"anon": True}, filecache={"cache_storage": "/tmp/files"})
ds = xr.open_dataset(file, engine="cfgrib")

try:
    ds = xr.open_dataset(file, engine="cfgrib", backend_kwargs={"errors": "raise"})
except cfgrib.dataset.DatasetBuildError as e:
    print(f"Error occurred: {e}")
    sys.exit(1)

Thanks Ray, we're working on writing some proper documentation written for cfgrib - this is a good example of a hidden feature!