google / xarray-beam

Distributed Xarray with Apache Beam

https://xarray-beam.readthedocs.io

google/xarray-beam Issues

Rechunk memory usage seems over `max_mem` limit
Closed 4 months ago2
Dimension order produced by Rechunk is opaque and not controllable; mismatch can cause errors from subsequent ChunksToZarr.
Updated 6 months ago
Race condition in ChunksToZarr when template is not supplied explicitly as an xarray.Dataset
Updated a year ago
Multi-stage rechunking
Closed a year ago1
Error in ChunksToZarr appears on docs page
Closed a year ago
Register Beam coder(s) to avoid "Using fallback deterministic coder" warnings
Updated 2 years ago
Add CI to check that code has been formatted with pyink formatter.
Updated 2 years ago
Simultaneously read multiple Datasets into an Xarray-Beam pipeline
Updated 2 years ago2
Require using make_template() if providing a template to ChunksToZarr?
Closed 2 years ago2
Consider adding ZarrToChunks() and/or an open_zarr() helper function
Closed 2 years ago
Support missing chunks in ConsolidateChunks
Updated 2 years ago
Consider omitting unchunked dimensions from Key objects created with DatasetToChunks
Updated 2 years ago1
Help with opening netcdf4 files via HTTP
Closed 2 years ago19
Adding source distribution for xarray-beam on PyPI
Closed 2 years ago2
Document how to add `weather-dl` clients & manifests in the Contributing guide.
Closed 3 years ago1
Running pipelines on AWS?
Updated 3 years ago1
Add `split_vars` to FilePatternToChunks transform.
Updated 3 years ago
Support opening datasets with file-like objects in a Beam pipeline
Updated 3 years ago
Better documentation of behaviour with irregular source chunks
Updated 3 years ago5
Some notes on alignment with pangeo-forge-recipes around "keys"
Updated 3 years ago1
Support pangeo_forge_recipes.patterns.MergeDim in `FilePatternToChunks`
Updated 3 years ago
Indicate variables in xarray-beam keys
Closed 3 years ago
Figure out how to combine efforts with xpartition
Updated 3 years ago7
Support for striding / rolling windows
Updated 3 years ago1