Bad behavior when trying to run migrate with sqlite3
spapas opened this issue · comments
Hello,
I tried installing wagtaildemo using sqlite3 as a backend database.
running python manage.py syncdb works fine, however running python manage.py migrate produces a series of exception. More specifically, I can see the following (I will copy just the important bits):
The first time the exception is because of:
File "..\..\wagtail\wagtail\wagtailcore\migrations\0002_initial_data.py", line 11, in forwards model='page', app_label='wagtailcore', defaults={'name': 'page'})
The second time I run migrate, the exception is because of:
File "..\..\wagtail\wagtail\wagtaildocs\migrations\0002_initial_data.py", line 11, in forwards model='document', app_label='wagtaildocs', defaults={'name': 'document'})
The third time I run migrate, the exception is because of:
File "..\..\wagtail\wagtail\wagtailimages\migrations\0002_initial_data.py", line 11, in forwards model='image', app_label='wagtailimages', defaults={'name': 'image'})
The fourth time, the migrate completes without problems!
In all cases, the error I see is the following:
- Migration 'wagtailimages:0002_initial_data' is marked for no-dry-run.
! Error found during real run of migration! Aborting.
! Since you have a database that does not support running
! schema-altering statements in transactions, we have had
! to leave it in an interim state between migrations.
! You *might* be able to recover with: (migration cannot be dry-run; cannot discover commands)
! The South developers regret this has happened, and would
! like to gently persuade you to consider a slightly
! easier-to-deal-with DBMS (one that supports DDL transactions)
! NOTE: The error which caused the migration to fail is further up.
followed by a stacktrace (originating from the place I wrote before) and ending with:
"Your database backend doesn't behave properly when "
django.db.transaction.TransactionManagementError: Your database backend doesn't behave properly when autocommit is off. Turn it on before using 'atomic'.
Does anybody know what is happening ? This is very confusing and I believe that it leaves my database in a not so good state :-(
Some more insight at this issue:
It seems that south wants to do the migration of each different application in the same transaction. This leads to the error mentioned above
Since you have a database that does not support running schema-altering statements in transactions, we have had to leave it in an interim state between migrations.
After some search I found out that sqlite3 actually supports schema changes in transactions but it's not supported because of how python issues statements (http://south.aeracode.org/ticket/1027).
In any case, with the above, the bad behavior can be explained:
When the migrate of wagtailcore starts, the migration 0001 with the schema definition will be executed, but the migration 0002 with the initial data won't be executed. When we re-run migrate, the 0002 migration of wagtailcore will be executed, the transaction will close and a new one will start with the wagtaildocs, for which only the 0001 transaction will be executed etc.
Now, I wasn't able to find a way (through an option for instance) to tell south to close transactions between different migrations (maybe somebody else is more experienced with south than me). However, I found out a simple way to do the migrations (that will throw no exceptions with sqlite3):
After issuing python manage.py syncdb execute the following two statements:
python manage.py migrate 0001 --all
python manage.py migrate
(instead of python manage.py migrate).
The first migrate will execute only the 0001 migration to all apps (so schema definitions won't be mixed with inserts) and the second statement will execute any remaining migrations.
I think we could add a notice to the docs clarifying this behavior for sqlite3 users.
I think we could add a notice to the docs clarifying this behavior for sqlite3 users.
I agree.
Note added to wagtaildemo's readme in 08f8c9c . Will leave this ticket open/unassigned in case someone can come up with a fix that makes manage.py migrate work on its own.
Fixed by #60 - readme note now removed accordingly, in wagtail-deprecated/wagtaildemo@5788826