Giters

splitgraph / seafowl

Analytical database for data-driven Web applications 🪶

Home Page:https://seafowl.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Seafowl 0.4 migration instructions

gruuya opened this issue · comments

commented

In Seafowl 0.3 we have replaced our custom storage layer with delta-rs, which all newly created tables now use. For tables created prior to this (with Seafowl 0.2 and 0.1), known as legacy tables, we have maintained a reading capability in order to maintain backward compatibility and facilitate migration to the new table format.

With the Seafowl 0.4 release we're now completely phasing out legacy tables, and consequently need to ensure a smooth migration without any unwanted data loss. In order to ensure this you have two options:

  • Option A: Manually migrate and drop legacy tables in Seafowl 0.3 before transitioning to Seafowl 0.4
    1. Spin up Seafowl v0.3
    2. (Optional) Migrate all desired legacy tables to new tables via CREATE TABLE new_table AS SELECT * FROM old_table
    3. DROP all remaining legacy tables
    4. Clean up any remaining partitions with VACUUM PARTITIONS
    5. Proceed with using Seafowl v0.4
  • Option B: Start Seafowl 0.4 with SEAFOWL_0_4_AUTODROP_LEGACY_TABLES=true env var for the first time, which will make Seafowl automatically delete all legacy tables and partition files, and then perform the metadata migrations.

You can also do a combination of the two, e.g. migrate only a subset of legacy tables in Seafowl v0.3, and then run Seafowl v0.4 with the above env var.