Large scale data migration
opened this issue · comments
I have a very large sqlite db (150,000 records). I need to migrate to MySQL. When I run the script with chunk size of 100, it migrates only one table and abruptly stops printing the message 'origin'.
Could you please help ?
Hey!
Well, I can tell you for a fact that I've migrated millions of rows in the past without a glitch, so the scale of the database should not be an issue.
What happens if you do not use chunking?
Also, would you mind giving me your system info by running the tool with only --version?
Hi. First of all sorry for the late response.
This is the output of the version command.
`| software | version |
|------------------------|--------------------------------------------------------------------------------|
| sqlite3-to-mysql | 1.3.10 |
| | |
| Operating System | Darwin 19.5.0 |
| Python | CPython 3.7.9 |
| MySQL | mysql Ver 14.14 Distrib 5.6.30, for osx10.11 (x86_64) using EditLine wrapper |
| SQLite | 3.33.0 |
| | |
| click | 7.1.2 |
| mysql-connector-python | 8.0.22 |
| pytimeparse | 1.1.8 |
| simplejson | 3.17.2 |
| six | 1.15.0 |
| tabulate | 0.8.7 |
| tqdm | 4.52.0 |
`
Have you tried not using chunking?