Shopify / ghostferry

The swiss army knife of live data migrations

https://shopify.github.io/ghostferry

Shopify/ghostferry Issues

Resuming can caused missed replication events
Closed a month ago1
Trouble with virtual generated columns
Updated 2 months ago6
Ghostferry binlog streamer lag with large source write volume due to misconfigured BinlogEventBatchSize?
Updated 7 months ago1
Config value for copydb `ReplicatedMasterPositionQuery` with vanilla MysQL replication
Updated 8 months ago
refactor ErrorHandler in ferry
Updated a year ago1
Ghostferry control server's webui shows the time taken as now - start
Updated a year ago
Potentially "overlocking" in cursor?
Updated a year ago
InlineVerifyer: invalid memory address or nil pointer dereference
Updated 2 years ago
Alternate exit criteria for DataIterators
Updated 2 years ago
TestThrottlerThrottlesAndUnthrottles flaky
Closed 2 years ago
Add tests for database connection with RunFerryFromReplica
Closed 2 years ago
Ensure BinlogEventBuffer Channel is initialized before running the BinlogStreamer
Updated 2 years ago
BinlogStreamerLag in ControlServer's progress API is not seconds as it says it is
Closed 2 years ago1
Dynamic reconfiguration
Closed 3 years ago2
Ghostferry misses data for PK values of <= 0
Updated 3 years ago4
Ghostferry should abort if it sees a DDL command that can compromise data integrity
Updated 3 years ago
Cannot follow tutorial due to incompatible docker-compose.yml
Closed 3 years ago1
Investigate removing the cursor and merge it back with the DataIterator
Updated 3 years ago
unsigned mediumint value through binlog streamer wrongfully parsed
Updated 3 years ago2
mediumint not recognized as numeric type
Closed 3 years ago2
MariaDB SHOW SLAVE HOSTS output differs from MySQL
Closed 3 years ago
MariaDB binlog events differ from MySQL
Updated 3 years ago
tried to advance to a zero log position
Closed 3 years ago1
Ghostferry documentations review 2021
Updated 3 years ago
Problems with migrating data with foreign key constraints.
Updated 3 years ago
Data corruption with composite PK and non-unique ID column with ghostferry-sharding only
Closed 3 years ago5
Possible deadlock issue in logrus, or deadlock between the data writer and the DataIterator
Updated 3 years ago
Add sanity check to InlineVerifier and TargetVerifier
Updated 3 years ago
estimated row size in bytes performance issues
Updated 3 years ago2
Return first error from `GetEvents`
Closed 3 years ago
ControlServer ETA is negative after "Done"
Closed 3 years ago3
Refactor ControlServer to use Progress instead of StatusDeprecated
Closed 3 years ago
ReportProgress at the beginning of cutover
Closed 4 years ago1
InlineVerifier.currentRowCount tracks the number of rows total, not the number of entries in the BinlogVerifyStore due to pk duplication
Closed 4 years ago
Expand the Progress struct to contain more detailed information
Updated 4 years ago
Ghostferry should queue tables in the DataIterator based on estimated table size
Closed 4 years ago
Support replicating from READ-ONLY DB
Updated 4 years ago13
Does ghostferry have Docker images?
Closed 4 years ago2
`DatabaseRewrites` Config option to rename schema is incorrect
Updated 4 years ago
Setting Config.ProgressReportFrequency to a small or negative number would be bad
Updated 4 years ago2
binlog format for source db is not compatible error
Closed 4 years ago7
Ghostferry Idempotence
Closed 4 years ago2
Documentation for SkipTargetVerification is out of date and logging
Updated 4 years ago
RowData.GetUint64() crashes on uint64 value
Closed 4 years ago19
UI incomplete render
Closed 4 years ago1
Resuming ghostferry may fail due to inactive verifier
Closed 4 years ago2
Ghostferry current ignores statement based replication events, which may cause data corruption if binlog_format and binlog_row_image is set temporarily to the wrong value
Updated 4 years ago
Target Verification against Data Corruption
Closed 4 years ago1
Fewer RowQueryEvents than expected observed in BinlogStreamer
Updated 4 years ago
copydb: State export on stdout makes resuming difficult, if tool invocation fails
Updated 4 years ago7