google / weather-tools

Tools to make weather data accessible and useful.

Home Page:https://weather-tools.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Problems downloading MARS data

lakshmanok opened this issue · comments

I registered on ECMWF site (https://apps.ecmwf.int/registration/) and got an API key from https://api.ecmwf.int/v1/key/ and created ~/.ecmwfapirc as shown on the page.

When I run:

weather-dl mars_example_config.cfg --local-run

I get:

File "/opt/conda/lib/python3.7/site-packages/apache_beam/transforms/core.py", line 1635, in <lambda>
    wrapper = lambda x, *args, **kwargs: [fn(x, *args, **kwargs)]
  File "/opt/conda/lib/python3.7/site-packages/weather_dl/download_pipeline/pipeline.py", line 241, in fetch_data
    client.retrieve(dataset, selection, temp.name)
  File "/opt/conda/lib/python3.7/site-packages/weather_dl/download_pipeline/clients.py", line 172, in retrieve
    self.c.execute(req=selection, target=output)
  File "/opt/conda/lib/python3.7/site-packages/ecmwfapi/api.py", line 645, in execute
    quiet=self.quiet,
  File "/opt/conda/lib/python3.7/site-packages/ecmwfapi/api.py", line 418, in __init__
    info = self.connection.call("%s/%s/%s" % (self.url, self.service, "info")).get(
  File "/opt/conda/lib/python3.7/site-packages/ecmwfapi/api.py", line 148, in wrapped
    return func(self, *args, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/ecmwfapi/api.py", line 350, in call
    raise APIException("ecmwf.API error 1: %s" % (self.last["error"],))
ecmwfapi.api.APIException: "ecmwf.API error 1: User 'lak@...' has no access to services/mars [while running 'FetchData']"

I think the issue at hand is that this API key needs to be a paid research or commercial license with ECWMF. The default MARS config here specifies data in their paid catalog. (This documentation describes the payment rules for accessing the archive.)

I did a local test of running this config, and did not receive this error. Though, the license on my machine is associated with ECMWF's commercial offering.

If you'd like to try out weather-dl using a free license, I recommend you use a CDS config example, e.g. era5_example_config_local_run.cfg. This requires setting up CDS's RC file (https://cds.climate.copernicus.eu/api-how-to).

Thanks! Data more than 30 days is supposed to be open access, but I didn't realize this handling charge thing.