Need to migrate large amount of data from default

[Charles Liu]

Hi, I have an issue where I have a table that has been partitioned, however, auto partitioning was not setup and the latest partition was from 2023w09. This has resulted in all the data since then going into the default partition. I am now trying to setup auto partitioning again. I want to keep the data in default and also want to prevent the database from going offline if possible. I know that we could try running partition_data_proc, but the default partition has around 2.5 TB of data and over 1 billion rows. I'm concerned about the length of time partition_data_proc will take to move the data from default, especially if there are reads and writes to the table at the same time. I have also considered moving the data out of default, fixing the partitions and then moving the data back into the correct partitions, or just creating a new database entirely with partitioning setup correctly. Is there a way to run partition_data_proc or create_patition_time asynchronously that won't lock the table? I'm wondering what is the best solution here. Thanks!