ZARR metadata consolidation consolidates MAIN and subtable metadata into one file
sjperkins opened this issue · comments
- dask-ms version: master
- Python version: 3.8
- Operating System: Ubuntu 20.04
Description
While trying to reference a zarr dataset on S3 for the https://github.com/ratt-ru/quartical-aws-tutorial, Quartical was unable to open one the ANTENNA subtable because the metadata for the MAIN and all subtables had been consolidated into the root directory. This change was introduced in:
I propose moving metadata consolidation into either dask-ms convert and/or dask-ms consolidate so that MS-like datasets are consolidated holistically, rather than dealing with MAIN/subtables individually