Need Help on existing HTTP server conversion (on docker-compose)

[Ng Yong Yau (Hugo)]

Guys, i am new to web dev and of course web deployment. I have recently deployed a Flask web app on AWS linux 2 on docker. I have also pointed my domain name to an Elastic IP (static). So far ... my web would work on HTTP. But i wanted to make it SSL encrypted, so i tried to follow this guide

I am stucked for 2 days now, i really had been searching up solution .... so here i am, please help T_T

I followed the guide up until where i need to use the init-letsencrypt.sh part, where i ran into problem.

`[ec2-user@ip-172-31-22-95 FlaskWebapp]$ sudo ./init-letsencrypt.sh
Existing data found for myDomain.com. Continue and replace existing certificate? (y/N) y

Creating dummy certificate for myDomain.com ...

Creating network "flaskwebapp_flask_network" with the default driver
Creating network "flaskwebapp_default" with the default driver
Creating flaskwebapp_certbot_run ... done
Generating a RSA private key
..........................................................++++
....++++
writing new private key to '/etc/letsencrypt/live/myDomain.com/privkey.pem'

Starting nginx ...

Creating flaskwebapp_proxy_1 ... done

Deleting dummy certificate for myDomain.com ...

Creating flaskwebapp_certbot_run ... done

Requesting Let's Encrypt certificate for myDomain.com ...

Creating flaskwebapp_certbot_run ... done
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator webroot, Installer None
Requesting a certificate for myDomain.com and www.myDomain.com
Performing the following challenges:
http-01 challenge for myDomain.com
http-01 challenge for www.myDomain.com
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain myDomain.com
Challenge failed for domain www.myDomain.com
http-01 challenge for myDomain.com
http-01 challenge for www.myDomain.com

Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: myDomain.com
Type: connection
Detail: Fetching http://www.myDomain.com/.well-known/acme-challenge/M6BFdnhcEStqhn0oXAB125ZK29z3vbRfyJR-BaTHs30: Connection refused

Domain: www.myDomain.com
Type: connection
Detail: Fetching http://www.myDomain.com/.well-known/acme-challenge/vogXpAFZSjJAudjK-C11EMB23AdJkW3FicLl3d2qA9U: Connection refused

Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.

Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
ERROR: 1

Reloading nginx ...`

Below is my docker-compose.yml file

`version: "3.7"
services:
web:
build: web
restart: always
environment:
- DEBUG=0
networks:
- flask_network
volumes:
- /etc/localtime:/etc/localtime:ro

proxy:
build: proxy
restart: always
ports:
- "80:80"
- "443:443"
networks:
- flask_network
volumes:
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot

certbot:
image: certbot/certbot
volumes:
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
db:
image: postgres:13
restart: always
networks:
- flask_network
environment:
- POSTGRES_USER=**
- POSTGRES_PASSWORD=**
- POSTGRES_DB=myDB
volumes:
- postgres_data:/var/lib/postgresql/data

networks:
flask_network:
volumes:
postgres_data:`

And this is my Proxy (nginx) configuration.

`server {

listen           80;
server_name      myDomain.com www.myDomain.com;

location /.well-known/acme-challenge/ {
    root /var/www/certbot;
}

location / {
    return 301 https://$host$request_uri;
}

}

server {

listen           443 ssl;
server_name      myDomain.com www.myDomain.com;

ssl_certificate        /etc/letsencrypt/live/myDomain.com/fullchain.pem;
ssl_certificate_key    /etc/letsencrypt/live/myDomain.com/privkey.pem;
include                /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam            /etc/letsencrypt/ssl-dhparams.pem;


location / {
    proxy_pass         http://web:5000;
    proxy_set_header   Host                 $host;
    proxy_set_header   X-Real-IP            $remote_addr;
    proxy_set_header   X-Forwarded-For      $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Proto    $scheme;d-Proto    $scheme;
}

}`

Below is my Directory Structure

Flask_Webapp/
├─ web/
│ ├─ Dockerfile
│ ├─ Bunch of other backend (not related)
├─ proxy/
│ ├─ conf
│ ├─ Dockerfile
├─ db/
docker-compose.yml

[Ng Yong Yau (Hugo)]

These error is found in letsencrypt.log

`2021-12-31 07:10:01,745:ERROR:certbot._internal.log:Error while running apachectl restart.
httpd not running, trying to start

(98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80`

[Jarno]

I fixed this by adding these two rules to the security group of my ec2.

[Ng Yong Yau (Hugo)]

I fixed mine by following this guide https://mindsers.blog/post/https-using-nginx-certbot-docker/

I fixed this by adding these two rules to the security group of my ec2.

My AWS already had those opened up last time, so i believe my problem is not related to this.

Since this matter had been awhile i cannot recall exactly what was the problem but i roughly remember that it was because tried to execute some of the process outside of docker (tho not intended, but by accident due to inexperience)