Nest `CombineReferences` inside `WriteCombinedReferences`
cisaacstern opened this issue · comments
Working on #610 has me thinking about how to easily communicate common recipe styles. Over there, I'm centering on the idea that the basic version of all common pipeline styles should be expressible as: FilePattern | Opener | Writer
. StoreToZarr
fits this style, but our kerchunk recipes do not:
pangeo-forge-recipes/examples/feedstock/hrrr_kerchunk_concat_step.py
Lines 48 to 61 in f6af0e0
To align kerchunk recipes with this style, we can nest CombineReferences
inside WriteCombinedReferences
, making the basic kerchunk recipe expressible as:
recipe = (
beam.Create(pattern.items())
| OpenWithKerchunk(...)
| WriteCombinedReferences(...)
)
This seems like a good call!
Closed by #635