Docker setup
Sorakurai opened this issue · comments
Hello
I'm experiencing an issue when using the docker-compose method to setup an environment.
I cloned the repo, navigated to the docker directory and made the following changes to the Dockerfile
| Old | New |
|---|---|
| EXPOSE 80 | EXPOSE 8080 |
| EXPOSE 443 | EXPOSE 8443 |
And the accompanying changes to the nginx.site.conf
| Old | New |
|---|---|
| Listen 80 | Listen 8080 |
| Listen 443 | Listen 8443 |
When running the docker-compose up --build command from the docker directory, everything builds fine 'till the last step:
Step 35/35 : ENTRYPOINT docker/entrypoint.sh
---> Running in d96e8f66a319
Removing intermediate container d96e8f66a319
---> a80221dc1c0b
Successfully built a80221dc1c0b
Successfully tagged docker_dfirtrack:latest
Recreating docker_db_1 ... done
Recreating docker_dfirtrack_1 ... done
Attaching to docker_db_1, docker_dfirtrack_1
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2021-01-04 16:32:42.097 UTC [1] LOG: starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2021-01-04 16:32:42.108 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2021-01-04 16:32:42.108 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2021-01-04 16:32:42.111 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2021-01-04 16:32:42.119 UTC [24] LOG: database system was shut down at 2021-01-04 16:17:43 UTC
db_1 | 2021-01-04 16:32:42.128 UTC [1] LOG: database system is ready to accept connections
dfirtrack_1 | Starting nginx: nginx.
dfirtrack_1 | Operations to perform:
dfirtrack_1 | Apply all migrations: admin, auth, contenttypes, dfirtrack_artifacts, dfirtrack_config, dfirtrack_main, django_q, sessions
dfirtrack_1 | Running migrations:
dfirtrack_1 | No migrations to apply.
dfirtrack_1 | 16:32:47 [Q] INFO Q Cluster oxygen-papa-fillet-maine starting.
db_1 | 2021-01-04 16:32:47.330 UTC [34] ERROR: relation "dfirtrack_async_messages" does not exist at character 45
db_1 | 2021-01-04 16:32:47.330 UTC [34] STATEMENT: SELECT "cache_key", "value", "expires" FROM "dfirtrack_async_messages" WHERE "cache_key" IN (':1:django_q:dfirtrack:cluster')
The Error message just keeps repeating. Any idea what's going wrong?
Hello,
Yeah, the problem arises because of missing python manage.py createcachetable for some changes within DFIRTrack's settings. I updated Ansible some days ago but apparently forgot docker. 😉 I updated the master branch just now. Please try again and give feedback if there is still something missing.
thx
Hello
Your fix solved the issue! ... but another pops up now :(
When browsing to the instance, I receive a "Bad Request (400)" error. I assume something is wrong with the nginx configuration?
Last lines from the Docker output:
Attaching to docker_db_1, docker_dfirtrack_1
db_1 |
db_1 | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1 |
db_1 | 2021-01-05 16:26:28.822 UTC [1] LOG: starting PostgreSQL 13.1 (Debian 13.1-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
db_1 | 2021-01-05 16:26:28.830 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
db_1 | 2021-01-05 16:26:28.830 UTC [1] LOG: listening on IPv6 address "::", port 5432
db_1 | 2021-01-05 16:26:28.831 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1 | 2021-01-05 16:26:28.837 UTC [24] LOG: database system was shut down at 2021-01-04 16:32:59 UTC
db_1 | 2021-01-05 16:26:28.865 UTC [1] LOG: database system is ready to accept connections
dfirtrack_1 | Starting nginx: nginx.
dfirtrack_1 | Operations to perform:
dfirtrack_1 | Apply all migrations: admin, auth, contenttypes, dfirtrack_artifacts, dfirtrack_config, dfirtrack_main, django_q, sessions
dfirtrack_1 | Running migrations:
dfirtrack_1 | No migrations to apply.
dfirtrack_1 | 16:26:35 [Q] INFO Q Cluster london-failed-dakota-jig starting.
dfirtrack_1 | 16:26:35 [Q] INFO Process-1:1 ready for work at 48
dfirtrack_1 | 16:26:35 [Q] INFO Process-1 guarding cluster london-failed-dakota-jig
dfirtrack_1 | 16:26:35 [Q] INFO Process-1:2 monitoring at 49
dfirtrack_1 | 16:26:35 [Q] INFO Process-1:3 pushing tasks at 50
dfirtrack_1 | 16:26:35 [Q] INFO Q Cluster london-failed-dakota-jig running.
dfirtrack_1 | Container started
dfirtrack_1 | !!!! You may run docker/setup_admin.sh from the host system to create a new superuser !!!!
I'll check this for myself. Yesterday with the out-of-the-box config everything worked.
In the meantime:
Did you rebuild the containers (guess so) after updating the repo? What is the Gunicorn logfile telling you (should be located in /var/log/gunicorn.log)? Logfiles of nginx may also helpful (located under /var/log/ngingx/)?
Did you adjust the port mapping in docker-compose.yml (I'm not sure at the moment whether this is necessary)?
Can you test a plain DFIRTrack instance without changing the ports first?
The most educated guess would be to see what the logfiles tell us and to cross check an instance without any changes.
Just for the record: the current master (6dd1f5d) without any changes using Ubuntu should work so far.
Yes, containers were rebuild after updating the repo.
Gunicorn logs:
/docker$ docker exec docker_dfirtrack_1 tail -n 20 /var/log/gunicorn.log
[2021-01-05 16:26:32 +0000] [37] [INFO] Starting gunicorn 20.0.4
[2021-01-05 16:26:32 +0000] [37] [INFO] Listening at: http://127.0.0.1:5000 (37)
[2021-01-05 16:26:32 +0000] [37] [INFO] Using worker: sync
[2021-01-05 16:26:32 +0000] [41] [INFO] Booting worker with pid: 41
[2021-01-05 16:26:32 +0000] [42] [INFO] Booting worker with pid: 42
[2021-01-05 16:26:32 +0000] [43] [INFO] Booting worker with pid: 43
[2021-01-05 16:26:32 +0000] [44] [INFO] Booting worker with pid: 44
Nginx error.log is empty
Nging access.log:
docker exec docker_dfirtrack_1 tail -n 20 /var/log/nginx/access.log
192.168.XXX.XXX - - [05/Jan/2021:16:30:06 +0000] "GET / HTTP/1.1" 400 154 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
192.168.XXX.XXX- - [05/Jan/2021:16:46:11 +0000] "GET / HTTP/1.1" 400 154 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36"
And yes, the port mapping was adjusted in the docker-compose.yml file. I'll try running a regular Docker images without adjustments tomorrow to see if that's where the issue lies.
Thanks for the quick responses ;)
Okay. I'm curious.
Another idea I had that could be far more helpful:
Turn on the debug mode by switching DEBUG to True in /dfirtrack/dfirtrack/local_settings.py. Maybe you have to restart the services (nginx and / or gunicorn) afterwards or simply restart the container. Should give us the best idea what is wrong.
Nginx seems to work. If Gunicorn were unavailable there would be an error 502. And if there was an issue with the backend, an error 500 should raise.
Running the docker container without any port modifications works ;)
When activating the DEBUG mode and restarting the container, I get the following error when browsing:
DisallowedHost at /
Invalid HTTP_HOST header: 'X.X.X.X'. You may need to add 'X.X.X.X' to ALLOWED_HOSTS.
...
Exception Type: DisallowedHost
Adding my "host" to the ALLOWED_HOSTS in the local_settings.py file and restarting the container solved the issue. Guess this is normal behaviour as my test without port modifications was using the "localhost" url, which is in the ALLOWED_HOSTS list.
A new problem arises though (it can never be easy right?). After creating the superuser I try to log in but the CSRF kicks in:
Forbidden (403)
CSRF verification failed. Request aborted.
Help
Reason given for failure:
Referer checking failed - https://X.X.X.X:8443/login/ does not match any trusted origins.
I'm not exactly familiar with Django and the solution I found googling don't seem to work:
Adding a CSRF_TRUSTED_ORIGINS variable to the local_settings.py file (https://stackoverflow.com/questions/38841109/csrf-validation-does-not-work-on-django-using-https)
Okay...
Yeah first of all: ALLOWED_HOSTS... I thought something like that. Docker is just prepared for local testing. And when setting up DFIRTrack for production or remote, Ansible sets the ALLOWED_HOSTS via {{ fqdn }}. But it's great when turning on debug has shown that. 😄
About your second question:
I'm not familiar with the implications of CSRF_TRUSTED_ORIGINS, but I think the problem stems from another point. You said, you added this to local_settings.py? Then I hope I have the solution, even without caring for the CSRF problem 😉
As you can see, settings.py imports only some local settings from local_settings.py. I hope by switching CSRF_TRUSTED_ORIGINS to settings.py it should work (if the suggestion from stackoverflow was right).
That solved the issue indeed!
The container is fully up and running now, many thanks ;)
Glad to hear that! 😄
Have fun and happy hunting!