Set up database migrations
jasonaowen opened this issue · comments
Some of the changes we need to make in reviving niceties include changes to the database schema. We will need to make those changes carefully, and preserve the existing data in the production instance. The term of art for this is database schema migrations; we should find and adopt a migration tool.
I've looked for such tools in other languages on other projects, and found it very helpful to figure out what I was looking for ahead of time. Here are some of the goals I would consider if I were to do this research:
- in the Python ecosystem: I'd prefer not to introduce a dependency on another language's toolchain
- uses simple SQL files: I've found that native SQL code is much easier to work with and reason about than a tool-specific DSL, XML, or arbitrary code mixed with SQL strings
- can be run and/or verified at service start: ideally, the code can ensure that the database schema is as it expects at runtime, so that we can avoid the category of errors that stem from such a mismatch
Non-goals:
- strong rollback support: I've found that I have rarely, if ever, needed to undo a database migration; I would be fine with a tool that lacked the ability to migrate down, and I don't intend for us to write the down scripts if it does offer that support
- database agnostic support: we're using PostgreSQL; it does not matter to me if the tool supports more databases than that
We are currently using SQLAlchemy to talk to the database. I am not very familiar with it; it may well be that it has built-in migration support, or that there are tools that build on top of SQLAlchemy. If so, we should use that; if not, let's be careful not to introduce another ORM if we can help it.
I have found this related reading to be a good starting point:
I did some research and quickly found lots of advice on migrating SQLAlchemy under Flask using Flask-Migrate, a wrapper around Alembic (written by the same ppl as SQLAlchemy). I followed the developer's instructions for adding Flask-Migrate to an existing project. I'll update the README and submit the changes next.
@mjec the next step would be to set the production database up in a similar vein. Since you're more familiar with that environment, I wanted to make sure you were looped in on the plan. Are there any particularly good or bad times for me to work on this? Would you rather handle it? I found the process fairly straightforward.