Cannot test migrations if there is a collation in the migrations
jrobichaud opened this issue · comments
I have a migration with a collation:
Ex:
# Generated by Django 4.1.5 on 2023-02-04 09:26
from django.contrib.postgres.operations import CreateCollation
from django.db import migrations
class Migration(migrations.Migration):
initial = True
dependencies = []
operations = [
CreateCollation(
"french",
provider="icu",
locale="fr-x-icu",
)
]
When running the migration tests I have this error:
Error
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.DuplicateObject: collation "french" already exists
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/django_test_migrations/contrib/unittest_case.py", line 37, in setUp
self.old_state = self._migrator.apply_initial_migration(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django_test_migrations/migrator.py", line 61, in apply_initial_migration
return self._migrate(targets, plan=plan)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django_test_migrations/migrator.py", line 84, in _migrate
return self._executor.migrate(migration_targets, plan=plan)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/migrations/migration.py", line 130, in apply
operation.database_forwards(
File "/usr/local/lib/python3.11/site-packages/django/contrib/postgres/operations.py", line 223, in database_forwards
self.create_collation(schema_editor)
File "/usr/local/lib/python3.11/site-packages/django/contrib/postgres/operations.py", line 199, in create_collation
schema_editor.execute(
File "/usr/local/lib/python3.11/site-packages/django/db/backends/base/schema.py", line 199, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: collation "french" already exists
I found this workaround:
class TestMyMigration(MigratorTestCase):
databases = ("default", "my_db",)
database_name = "my_db"
# ...
def setUp(self) -> None:
sql.drop_models_tables(self.database_name)
sql.flush_django_migrations_table(self.database_name)
with connections["my_db"].cursor() as cursor:
cursor.execute("DROP COLLATION french")
super().setUp()
def test_my_migration(self):
pass
hi 👋
Creating a collation is tricky because it's not a data migration, it's just changing the DB configuration/metadata. Probably the best solution will be to add some hook (something like, migrator.apply_*_migration(..., run_before_apply=drop_collaction)
), so the developers can define their own code that will be run before applying initial/target migration. What do you think about it @sobolevn and @jrobichaud?
Yes, seems like a good idea. Something like setup=
and teardown=
Sounds great!
@jrobichaud would you like to send a PR? :)
I could give it a try.
I have trouble finding meaningful names.
For the TestCase it could work:
- setUpBeforeApply
- tearDownAfterApply
For the migrator:
- run_before_apply (its not clear its for forward migration)
- run_after_apply (its not clear its for reverse migration)
In theory someone could want to run 4 different things depending if its before/after apply in forward/reverse migration.
I only need before apply in forward migration.
Any thoughts about this?
Is anything needed for the migrator at all considering the user call it himself directly? He could just add the just before the function.
I believe only the TestCase needs new functions.
yes, that's true that when the developers are using the Migrator
instance directly they can add custom code before/after the apply_*_migration()
, but I think that for apply_initial_migration
we will usually want to run "before apply" hook/code just after dropping tables and flushing Django migrations table - so exactly here, then adding arguments to apply_*_migration
makes sense. We can add identical arguments to the apply_target_migration
to make the interface more intuitive.
So I would add following arguments to apply_*_migration
and pass it down to _migrate()
:
before_hook
: Callable[[PlanType], None](executed in
_migrate` before doing migration)-
after_hook
: Callable[[PlanType], None](executed in
_migrate` after doing migration)
and for the unittest
test case just 1 method (optionally 2), taking plan as well (like you proposed):
run_before_initial_migration
Optionally also run_after_initial_migration
, but this can be replaced by adding the required code to prepare
(of course assuming the migration plan is not important).