Fresh database migrations fail with: `relation "exercises_exercisebase" does not exist`
Inlustra opened this issue · comments
Hi guys,
Currently trying to get this all spun up, unfortunately I'm seeing a migration issue on a fresh install.
In particular, get the following error when the container starts, and also when I try to manually migrate from a clean DB.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "exercises_exercisebase" does not exist
LINE 1: ... COUNT("exercises_exercise"."id") AS "count" FROM "exercises...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/wger/src/manage.py", line 25, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 96, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 97, in handle
self.check(databases=[database])
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 475, in check
all_issues = checks.run_checks(
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/registry.py", line 88, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/model_checks.py", line 36, in check_all_models
errors.extend(model.check(**kwargs))
File "/home/wger/src/wger/exercises/models/base.py", line 155, in check
no_translations = cls.no_translations.all().count()
File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 621, in count
return self.query.get_count(using=self.db)
File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/query.py", line 559, in get_count
return obj.get_aggregation(using, ["__count"])["__count"]
File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/query.py", line 544, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 102, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "exercises_exercisebase" does not exist
LINE 1: ... COUNT("exercises_exercise"."id") AS "count" FROM "exercises...
This error happens on startup when it performs the migrations automatically, and also if I try to clean the DB and run the following command:
docker exec wger python3 manage.py migrate
This is from a clean start:
Traceback (most recent call last):
File "/home/wger/.local/bin/wger", line 33, in <module>
sys.exit(load_entry_point('wger', 'console_scripts', 'wger')())
File "/home/wger/.local/bin/wger", line 22, in importlib_load_entry_point
for entry_point in distribution(dist_name).entry_points
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 957, in distribution
return Distribution.from_name(distribution_name)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 548, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for wger
Performing database migrations
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
psycopg2.errors.UndefinedTable: relation "exercises_exercisebase" does not exist
LINE 1: ... COUNT("exercises_exercise"."id") AS "count" FROM "exercises...
^
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/wger/src/manage.py", line 25, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python3.10/dist-packages/django/core/management/__init__.py", line 440, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 402, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 448, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 96, in wrapped
res = handle_func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 97, in handle
self.check(databases=[database])
File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 475, in check
all_issues = checks.run_checks(
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/registry.py", line 88, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/usr/local/lib/python3.10/dist-packages/django/core/checks/model_checks.py", line 36, in check_all_models
errors.extend(model.check(**kwargs))
File "/home/wger/src/wger/exercises/models/base.py", line 155, in check
no_translations = cls.no_translations.all().count()
File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 621, in count
return self.query.get_count(using=self.db)
File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/query.py", line 559, in get_count
return obj.get_aggregation(using, ["__count"])["__count"]
File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/query.py", line 544, in get_aggregation
result = compiler.execute_sql(SINGLE)
File "/usr/local/lib/python3.10/dist-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 102, in execute
return super().execute(sql, params)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 67, in execute
return self._execute_with_wrappers(
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
return executor(sql, params, many, context)
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 84, in _execute
with self.db.wrap_database_errors:
File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "exercises_exercisebase" does not exist
LINE 1: ... COUNT("exercises_exercise"."id") AS "count" FROM "exercises...
It's worth noting that DJANGO_DEBUG=True
Hi! Are you using the docker compose on its own or with something like portainer? From what I have heard from some other users, it is possible that the db container takes too much time to start so that during the initial bootstrap the database isn't correctly initialised (even if it's marked that the application should wait for it). Dropping everything and manually starting the database container first and then the rest should do the trick (only during the first run, later it won't matter)
I am using docker-compose, I've even manually started each part of the application.
I've also tried dropping, deleting volumes and restarting.
It's worth noting that changing to v2.1-dev does get further and migrations are applied, but I end up with:
Applying nutrition.0010_logitem_meal... OK
Applying nutrition.0011_alter_logitem_datetime... OK
Applying sessions.0001_initial... OK
Applying sites.0001_initial... OK
Applying sites.0002_alter_domain_unique... OK
Applying weight.0001_initial... OK
Applying weight.0002_auto_20150604_2139... OK
Applying weight.0003_auto_20160416_1030... OK
Set site URL to https://exercise.****.com
Using gunicorn...
[2023-07-12 12:16:18 +0000] [10] [INFO] Starting gunicorn 20.1.0
[2023-07-12 12:16:18 +0000] [10] [INFO] Listening at: http://0.0.0.0:8000 (10)
[2023-07-12 12:16:18 +0000] [10] [INFO] Using worker: sync
[2023-07-12 12:16:18 +0000] [11] [INFO] Booting worker with pid: 11
Internal Server Error: /en/software/features
Traceback (most recent call last):
File "/home/wger/src/wger/utils/language.py", line 54, in load_language
language = Language.objects.get(short_name=used_language)
File "/usr/local/lib/python3.10/dist-packages/django/db/models/manager.py", line 85, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/django/db/models/query.py", line 435, in get
raise self.model.DoesNotExist(
wger.core.models.language.Language.DoesNotExist: Language matching query does not exist.
Which I believe has been fixed by other users by deleting and manually firing each container up which I will give a go.
let me try to spin up a fresh install, just in case
It's almost instant, this is with wger/server:latest.
Also thanks for the quick responses, let me know if I can do anything else to help
yeah, you're right, something is not right. But why? We didn't do anything(tm)
Will take a look
I think I got something. It seems the celery service tried to sync the exercises before the application was ready (the time when this happens is randomly selected). Changing the order of the services fixed this for me (and in any case, this is more correct anyway)
celery_worker:
depends_on:
web:
condition: service_healthy
celery_beat:
depends_on:
celery_worker:
condition: service_healthy
however it looks like yours breaks down even before that, mhhh
Will give that a go, though in my testing, I'm pretty sure I didn't even startup the celery_worker or celery_beat
Edit: Definitely getting the issue without any of the celery_* running
Also can't get 2.1-dev running for some other completely different migration issues! But then I think the compose might not be valid for 2.1.
wger.core.models.language.Language.DoesNotExist: Language matching query does not exist.
I won't push this one as obviously it's old code that has potentially already been fixed.
In the meantime - do you have any older digest ids of the :latest tag in docker that I could pull?
Happy to test out multiple previous digests to find the error-causing one if necessary. Assume they'll be in Github Actions?
yeah, 2.1 is really old and I'm not sure if it would work with the current compose file. I also tried the current setup with a mac, linux and and arm server from hetzner without any problems so I'm not sure what could be happening. Are you using a somewhat new docker? And yes, all the info about the images are in github actions https://github.com/wger-project/wger/actions/workflows/docker.yml
Ah right, so to clarify, you're not seeing the issue at all? Definitely my end?
Exactly, everything works fine
Ok no worries, I will plough on through! Thanks for taking the time. Will update this issue if I have time to fix it! :)
I'm actually using docker-compose version 2.4 so, fairly old as I'm slotting this into the rest of my docker-compose setup.
Super weird though as it definitely looks like an application/setup issue as opposed to a docker internal issue.
This is the very first error that occurs:
Traceback (most recent call last):
File "/home/wger/.local/bin/wger", line 33, in <module>
sys.exit(load_entry_point('wger', 'console_scripts', 'wger')())
File "/home/wger/.local/bin/wger", line 22, in importlib_load_entry_point
for entry_point in distribution(dist_name).entry_points
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 957, in distribution
return Distribution.from_name(distribution_name)
File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 548, in from_name
raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for wger
Ok as it turns out, it was my own fault due to being lazy with permissions, but it does kind of raise another question.
I had added user: 0:0 to the docker-compose for the database and wger_server, and removing that got it building again, so it didn't like that for whatever reason. This got the migrations running correctly etc.
However and unfortunately, I was now getting permission denied errors during the collectstatic step.
docker exec wger python3 manage.py collectstatic
------
PermissionError: [Errno 13] Permission denied: '/home/wger/static/django_extensions'
Given this config:
wger:
image: wger/server:latest
container_name: wger
depends_on:
wger_db:
condition: service_healthy
wger_cache:
condition: service_healthy
env_file:
- .env.wger
volumes:
- ${CONFIG_DIR}/wger/static:/home/wger/static
- ${CONFIG_DIR}/wger/media:/home/wger/media
healthcheck:
test: wget --no-verbose --tries=1 --spider http://localhost:8000
interval: 10s
timeout: 5s
retries: 5
restart: unless-stopped
A quick mkdir ${CONFIG_DIR}/wger/static && chmod 777 ${CONFIG_DIR}/wger/static (Obviously don't copy me here) does actually fix this error and get the collectstatic method running, but it does raise some notes for me:
- The collectstatic step fails silently during
docker-compose up - I don't know why it is getting permission denied errors, as not a single other container in my (Fairly large) setup, needs any permission modifications. Though I will fully admit that this is most likely due to my lack of knowledge.
One last note is that other containers I'm using (In particular the linuxserver images) allow setting PGID and PUID for this kind of control.
Thanks again for the hard work on this software and responding to my issue. Looking forward to pretending I exercise alot!
alright, wasn't expecting this to be a permissions problem!
but yeah, the volumes are there for a reason 😇 Specially under windows regular mappings were a real pain and the volumes hide everything neatly away. One of these days I'll sit down and see which permissions are needed to use regular folders (in the end we don't do anything crazy with them, specially the static files are just a collection of js and css files that need to be writable by one container and readable by another) and document everything
Looking forward to pretending I exercise alot!
this is the way
(closing here, if you run into other problems, just open a new issue)