ecmwf / eccodes-python

Python interface to the ecCodes GRIB/BUFR decoder/encoder

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Writing index file twice causes `ecCodes assertion failed: `field->file'`

acowlikeobject opened this issue · comments

  • Ubuntu 18.04.3 LTS
  • eccodes 2.16.0
  • conda install -c conda-forge eccodes and pip install eccodes-python

If I try to write an index file off the same grib file twice:

import eccodes

grib_name = 'era5-levels-members.grib'

for i in range(2):
    index_name = grib_name + f'.idx{i}'
    with eccodes.GribIndex(grib_name, ['level']) as index:
        index.write(index_name)

    with eccodes.GribIndex(file_index=index_name) as index:
        msgs = index.select({'level': 500})  # Fails here on second run of loop.
        print(f'Using {index_name}: {len(msgs)} messages')

I get an ecCodes assertion error:

Using era5-levels-members.grib.idx0: 192 messages
ecCodes assertion failed: `field->file' in /home/conda/feedstock_root/build_artifacts/eccodes_1579074340882/work/src/grib_index.c:1470
Aborted (core dumped)

The two index files written are not identical:

cmp era5-levels-members.grib.idx0 era5-levels-members.grib.idx1
era5-levels-members.grib.idx0 era5-levels-members.grib.idx1 differ: byte 36, line 1

However, if I replace index.write with a command line call to grib_index_build, it works:

import subprocess
import eccodes

grib_name = 'era5-levels-members.grib'

for i in range(2):
    index_name = grib_name + f'.idx{i}'
    subprocess.call(f'grib_index_build -o {index_name} -k level {grib_name}'.split(' '))

    with eccodes.GribIndex(file_index=index_name) as index:
        msgs = index.select({'level': 500})  # Fails here on second run of loop.
        print(f'Using {index_name}: {len(msgs)} messages')

Output:

--- grib_index_build: processing era5-levels-members.grib
--- grib_index_build: keys included in the index file era5-levels-members.grib.idx0:
--- level
--- level = { 500, 850 }
--- 160 messages indexed
Using era5-levels-members.grib.idx0: 192 messages
--- grib_index_build: processing era5-levels-members.grib
--- grib_index_build: keys included in the index file era5-levels-members.grib.idx1:
--- level
--- level = { 500, 850 }
--- 160 messages indexed
Using era5-levels-members.grib.idx1: 192 messages

Seems like some state is being retained incorrectly after the first call?