Unable to send Verification email - Self Hosted

Question

Unable to send Verification email - Self Hosted

mrehanabbasi opened this issue 7 months ago · comments

Muhammad Rehan Abbasi commented 7 months ago

I have followed all the instructions for self-hosted and even used the simplelogin/app:4.5.6-beta image along with using the -traditional flag for DKIM keys. However, when registering my first account, I get this error:

Invalid email, are you sure the email is correct?

Upon checking the DB, I see that a new user has been added to the users table. However, upon checking the logs of sl-app, I see that the verification email is not being sent:

2023-12-03 22:12:24,371 - SL - DEBUG - 7 - "/code/app/auth/views/register.py:82" - register() -  - create user myemail@yahoo.com
2023-12-03 22:12:24,761 - SL - DEBUG - 7 - "/code/app/email_utils.py:298" - send_email() -  - send email to myemail@yahoo.com, subject 'Just one more step to join SimpleLogin'
2023-12-03 22:12:24,841 - SL - ERROR - 7 - "/code/app/mail_sender.py:166" - _send_to_smtp() -  - Could not send message to smtp server 10.0.0.1:25
Traceback (most recent call last):
  File "/code/app/mail_sender.py", line 123, in _send_to_smtp
    with SMTP(config.POSTFIX_SERVER, smtp_port) as smtp:
  File "/usr/local/lib/python3.10/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.10/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.10/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
  File "/usr/local/lib/python3.10/socket.py", line 845, in create_connection
    raise err
  File "/usr/local/lib/python3.10/socket.py", line 833, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

I also have enabled ufw to allow ports 22, 25, 80 and 443.

Is there anything I am doing wrong or are the instructions not complete/correct?

Maxime Labelle · Answer 1 · Mon Dec 04 2023 21:25:00 GMT+0800 (China Standard Time)

The instructions are entirely correct but assume that:

You are running Simple Login as Docker containers
You are running Postfix on the host.

From that perspective, what you are seing is that:

The sl-app is trying to connect to the host via the well-known docker brigde network gateway 10.0.0.1 on port 25.
There are no firewall issues as this is running entirely within the box itself.

Does that match your setup?

Muhammad Rehan Abbasi · Answer 2 · Mon Dec 04 2023 21:29:13 GMT+0800 (China Standard Time)

@springcomp Yes that is exactly my setup.

Maxime Labelle · Answer 3 · Mon Dec 04 2023 21:29:40 GMT+0800 (China Standard Time)

On another note, you mention using 4.6.5-beta.
Did you also update the required instruction for running sl-migration ?

  docker run --rm \
      --name sl-migration \
      -v $(pwd)/sl:/sl \
      -v $(pwd)/sl/upload:/code/static/upload \
      -v $(pwd)/dkim.key:/dkim.key \
      -v $(pwd)/dkim.pub.key:/dkim.pub.key \
      -v $(pwd)/simplelogin.env:/code/.env \
      --network="sl-network" \
-    simplelogin/app:3.4.0 flask db upgrade
+    simplelogin/app:4.6.5-beta alembic upgrade head

Muhammad Rehan Abbasi · Answer 4 · Mon Dec 04 2023 21:31:38 GMT+0800 (China Standard Time)

@springcomp I tried with both 3.4.0 and 4.6.5-beta (with updated command for migration).

Maxime Labelle · Answer 5 · Mon Dec 04 2023 22:08:06 GMT+0800 (China Standard Time)

Can you try and report the following troubleshooting steps:

Postfix must be running on the host

Please, try to connect to the local SMTP server:

> telnet localhost 25
Trying ::1...
Connected to localhost.
Escape character is '^]'.
220 app.domain.tld ESMTP Postfix (Ubuntu)
QUIT
221 2.0.0 Bye
Connection closed by foreign host.
>

Please, make sure you can successfully connect to the local SMTP server.
You will need to issue the QUIT command to exit.
Please, make sure that the SMTP banner mentions the fully qualified name of you self hosted domain like app.domain.tld.

Check the network configuration

Please, run the following command:

docker inspect network simplelogin_default

You should get a JSON-formatted output with network configuration.
Make sure that the gateway is indeed specified:

    "IPAM": {
        "Driver": "default",
        "Options": null,
        "Config": [
            {
                "Subnet": "10.0.0.0/24",
                "Gateway": "10.0.0.1"
            }
        ]
    }

Please, check that the running containers all share the same network.
you can lookup the output to locate the Containers array and see if you have all running containers with a matching IP address from the same network.

My setup reports the following configuration:

sl-app running on 10.0.0.3/24
sl-email running on 10.0.0.2/24
sl-job-runner running on 10.0.0.4/24
sl-db running on 10.0.0.6/24

Muhammad Rehan Abbasi · Answer 6 · Mon Dec 04 2023 23:33:49 GMT+0800 (China Standard Time)

Postfix is running and I was successfully able to connect to localhost on port 25 using telnet.
My network name is sl-network and its configuration is correct. I can also see that all the containers are connected via this network and all of them have IPs assigned to them in the 10.0.0.x range.

Muhammad Rehan Abbasi · Answer 7 · Mon Dec 04 2023 23:45:20 GMT+0800 (China Standard Time)

Here's the complete network config:

[
    {
        "Name": "sl-network",
        "Id": "8cec3bd46e9b11bacee64dc78bce420cba24da11eae89b2c19b6741289bf703b",
        "Created": "2023-12-04T09:42:08.000606397Z",
        "Scope": "local",
        "Driver": "bridge",
        "EnableIPv6": false,
        "IPAM": {
            "Driver": "default",
            "Options": {},
            "Config": [
                {
                    "Subnet": "10.0.0.0/24",
                    "Gateway": "10.0.0.1"
                }
            ]
        },
        "Internal": false,
        "Attachable": false,
        "Ingress": false,
        "ConfigFrom": {
            "Network": ""
        },
        "ConfigOnly": false,
        "Containers": {
            "130efb6112097084a2a6b8824632f410d841a980641e813928ea0af523833f30": {
                "Name": "sl-job-runner",
                "EndpointID": "e00ecba8377355420af7cf5e149d633d30d494e2c7d3af637de7f314224645a4",
                "MacAddress": "02:42:0a:00:00:03",
                "IPv4Address": "10.0.0.3/24",
                "IPv6Address": ""
            },
            "346a1367145e71a15a0122c413fc748fd3548348809d65239995b752b65f7e40": {
                "Name": "sl-email",
                "EndpointID": "bee71577a8b9109ae466b0f23bc1ec1bfb5d1285820ae0f45ff1c5527a644f1f",
                "MacAddress": "02:42:0a:00:00:02",
                "IPv4Address": "10.0.0.2/24",
                "IPv6Address": ""
            },
            "4b94e28d23e1a0180ed6ee8234c258c9e749c30fad0fb7bece7fe7b1b4cfa40a": {
                "Name": "sl-db",
                "EndpointID": "d9bb04d3cc43a94fab3966ffa56fb182002741c82158c55219660fcc0f47ad00",
                "MacAddress": "02:42:0a:00:00:05",
                "IPv4Address": "10.0.0.5/24",
                "IPv6Address": ""
            },
            "5d07dac7c4af4a1ed9b17f400489c4eb4af866c44619e8c05c7296fad2ef889f": {
                "Name": "sl-app",
                "EndpointID": "1cf92077bae94a6e66ae197c2f3ee8a636b6ff6904c5398ece9d46d92a371f84",
                "MacAddress": "02:42:0a:00:00:04",
                "IPv4Address": "10.0.0.4/24",
                "IPv6Address": ""
            }
        },
        "Options": {},
        "Labels": {}
    }
]

Maxime Labelle · Answer 8 · Mon Dec 04 2023 23:54:09 GMT+0800 (China Standard Time)

So, for some reason, when inside a docker container, you cannot reach the host… 🤔

Can you try connecting to sl-app and issueing a telnet command:

> docker exec -it sl-app bash
root@5d07dac7c4a:/code# apt update && apt install -y inetutils-telnet
root@5d07dac7c4a:/code# telnet 10.0.0.1 25
Trying 10.0.0.1 ...
Connected to 10.0.0.1.
Escape character is '^]'.
220 app.domain.tld ESMTP Postfix (Ubuntu)
QUIT
221 2.0.0 Bye
Connection closed by foreign host.
root@5d07dac7c4a: /code#

If that works, I’m out of ideas 😥

Muhammad Rehan Abbasi · Answer 9 · Tue Dec 05 2023 01:16:46 GMT+0800 (China Standard Time)

I'm not able to connect to host from within the container:

root@705a1b7af81c:/code# telnet 10.0.0.1 25
Trying 10.0.0.1...
telnet: Unable to connect to remote host: Connection refused

Maxime Labelle · Answer 10 · Tue Dec 05 2023 02:00:09 GMT+0800 (China Standard Time)

Can you share the output of sudo ufw status ?

I would suggest disabling the firewall for troubleshooting.
Just out of curiosity, can you please confirm that your host IP is not in 10.0.0.x network ?

Muhammad Rehan Abbasi · Answer 11 · Tue Dec 05 2023 10:11:02 GMT+0800 (China Standard Time)

I did turn off the firewall to see if that was causing some issue but that was not the case.

Also, two of my host's interfaces are in the range 10.15.0.x and 10.104.0.x respectively. However, even changing the IP range in the docker network to 192.168.0.x did not work.

Muhammad Rehan Abbasi · Answer 12 · Tue Dec 05 2023 10:16:00 GMT+0800 (China Standard Time)

Also, now when I try to create an account or resend the verification email, I get "502 Bad gateway" error from nginx and this in the logs:

2023-12-05 02:13:54,643 - SL - DEBUG - 337 - "/code/app/email_utils.py:298" - send_email() -  - send email to myemail@yahoo.com, subject 'Just one more step to join SimpleLogin'
[2023-12-05 02:14:10 +0000] [1] [CRITICAL] WORKER TIMEOUT (pid:337)
[2023-12-05 02:14:10 +0000] [337] [INFO] Worker exiting (pid: 337)
[2023-12-05 02:14:11 +0000] [343] [INFO] Booting worker with pid: 343

Ed · Answer 13 · Sat Jan 13 2024 09:12:08 GMT+0800 (China Standard Time)

I'm also getting the same as @mrehanabbasi. Same errors, same issue. I got around the account registration by modifying the users table like so...

UPDATE users SET activated = 't' where email = 'your@email.com';

But I'm not able to send an email to my alias and have it forwarded as I'm getting connection refused

2024-01-13 00:53:38,854 - SL - ERROR - "email_handler.py:1625" - handle_DATA() - 2c99d4b9-0cec-4888-b178-d96d9aadb8f7 - email handling fail test@email-1.com -> ['simplelogin-newsletter.readjusts@mydomain.xyz']
Traceback (most recent call last):
  File "email_handler.py", line 1619, in handle_DATA
    ret = self._handle(envelope)
  File "email_handler.py", line 1643, in _handle
    ret = handle(envelope)
  File "email_handler.py", line 1604, in handle
    for is_delivered, smtp_status in handle_forward(envelope, msg, rcpt_to):
  File "email_handler.py", line 540, in handle_forward
    user,
  File "email_handler.py", line 736, in forward_email_to_mailbox
    is_forward=True,
  File "/code/app/email_utils.py", line 1132, in sl_sendmail
    smtp = SMTP(POSTFIX_SERVER, POSTFIX_PORT_FORWARD)
  File "/usr/local/lib/python3.7/smtplib.py", line 251, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.7/smtplib.py", line 336, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.7/smtplib.py", line 307, in _get_socket
    self.source_address)
  File "/usr/local/lib/python3.7/socket.py", line 728, in create_connection
    raise err
  File "/usr/local/lib/python3.7/socket.py", line 716, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Running on Raspberry Pi 4
simplelogin/app 3.4.0
postgres 12.1

Muhammad Rehan Abbasi · Answer 14 · Sat Jan 13 2024 14:13:28 GMT+0800 (China Standard Time)

@edleeman17 I wasn't able to get simplelogin to run. However using this helped me get it up and running. I'd highly recommend it.

Ed · Answer 15 · Sat Jan 13 2024 16:10:18 GMT+0800 (China Standard Time)

Thank you! I'll take a look