http_proxy from /etc/environment not used on Debian

Question

http_proxy from /etc/environment not used on Debian

laroche opened this issue 2 years ago · comments

Running certbot on Debian and if you have "http_proxy" set in /etc/environment, certbot
does not use the prox server, but direct http/https connections.

best regards,

Florian La Roche

alexzorin · Answer 1 · Mon May 30 2022 04:18:20 GMT+0800 (China Standard Time)

I couldn't reproduce this on Debian 11, installing Certbot from a snap.

Some notes:

/etc/environment is only loaded by PAM on login. It's not re-read when you execute processes like Certbot. If you modified it, you need to reboot your session.
http_proxy only affects HTTP connections (for Certbot, only OCSP revocation checks are done over HTTP). You need to set https_proxy if you want to proxy Certbot's traffic with the ACME server.
(Specifically about the Certbot renewal timer) I'm not exactly sure when systemd reads /etc/environment. I think on boot only. You may actually need to reboot your system to make it see changes to that file, I don't know.

Florian La Roche · Answer 2 · Mon May 30 2022 14:31:57 GMT+0800 (China Standard Time)

This happens for me on Debian testing with the normal Debian package installed, no snap package.
http_proxy and https_proxy are both set and the machine is rebootet.

best regards,

Florian La Roche

alexzorin · Answer 3 · Mon May 30 2022 15:01:31 GMT+0800 (China Standard Time)

I still can't reproduce this, sorry. It works as expected.

If you try this, does it pick up your proxy environment?

# curl -vvv https://google.com
* Uses proxy env variable https_proxy == 'http://127.0.0.1:1234'
*   Trying 127.0.0.1:1234...
<snip>

What about:

python3 -c 'import urllib.request; print(urllib.request.getproxies())'

Florian La Roche · Answer 4 · Mon May 30 2022 15:18:18 GMT+0800 (China Standard Time)

Yes, the rest of the system works ok with proxy setup.

Your test case gives:

root@knorke2:/# curl -vvv https://google.com
* Uses proxy env variable no_proxy == 'localhost'
* Uses proxy env variable https_proxy == 'http://192.168.123.73:3128/'
*   Trying 192.168.123.73:3128...
* Connected to 192.168.123.73 (192.168.123.73) port 3128 (#0)
...

and

# python3 -c 'import urllib.request; print(urllib.request.getproxies())'
{'no': 'localhost', 'ftp': 'http://192.168.123.73:3128/', 'https': 'http://192.168.123.73:3128/', 'http': 'http://192.168.123.73:3128/'}

Can you try original Debian packages or could this be due to Debian<->snap differences?

best regards,

Florian La Roche

alexzorin · Answer 5 · Mon May 30 2022 15:34:02 GMT+0800 (China Standard Time)

Yes, I had tried:

root@5dab532a92ab:/# cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux bookworm/sid"
NAME="Debian GNU/Linux"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@5dab532a92ab:/#
root@5dab532a92ab:/# python3 -c 'import urllib.request; print(urllib.request.getproxies())'
{'https': 'http://192.168.123.73:3128/', 'http': 'http://192.168.123.73:3128/'}
root@5dab532a92ab:/#
root@5dab532a92ab:/# dpkg-query --showformat='${Version}\n' --show python3-certbot
1.25.0-1
root@5dab532a92ab:/#
root@5dab532a92ab:/# certbot register --staging --register-unsafely-without-email --agree-tos
Saving debug log to /var/log/letsencrypt/letsencrypt.log
An unexpected error occurred:
requests.exceptions.ConnectTimeout: HTTPSConnectionPool(host='acme-staging-v02.api.letsencrypt.org', port=443): Max retries exceeded with url: /directory (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0xffff9c712740>, 'Connection to 192.168.123.73 timed out. (connect timeout=45)'))
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
root@5dab532a92ab:/#

Florian La Roche · Answer 6 · Mon May 30 2022 15:52:32 GMT+0800 (China Standard Time)

Oh, that was unclear from me: The renew cron scripts will not pick up the env entries correctly.

Seems this is in /lib/systemd/system/certbot.service. I'll try adding proxy settings there. Not sure
if that should source /etc/environment directly or how that should work out.

best regards,

Florian La Roche

Florian La Roche · Answer 7 · Mon May 30 2022 15:56:14 GMT+0800 (China Standard Time)

I've added "EnvironmentFile=-/etc/environment" for this, not sure a better change is possible.

best regards,

Florian La Roche

alexzorin · Answer 8 · Mon May 30 2022 15:57:39 GMT+0800 (China Standard Time)

In my testing earlier, systemd did pick up environment variables from /etc/environment on boot, and propagated them to the Certbot renewal timer task. i.e. I do not think it should be necessary to make this change on a Debian system.

If you intend to modify that file, I suggest using overrides (systemctl edit certbot.service) and setting [Service]\nEnvironmentFile=... there, rather than changing the file directly. Otherwise, your changes may get wiped out by package upgrades.

Florian La Roche · Answer 9 · Mon May 30 2022 16:08:51 GMT+0800 (China Standard Time)

Oh, thanks for this information.

I'll further check this locally. Could very well be, that /etc/environment wasn't setup when certbot registration was run.
I assumed I can change /etc/environment at any time and this could be picked up after reboot.

Thanks a lot,

Florian La Roche

Florian La Roche · Answer 10 · Tue May 31 2022 04:29:15 GMT+0800 (China Standard Time)

Current status: The above change to add "EnvironmentFile=-/etc/environment" to
/lib/systemd/system/certbot.service helps. This can also easily be verified in
/var/log/letsencrypt/letsencrypt.log.
I'd love to see this in the default debian configuration, so no extra adjustment is needed for this.

I found another source for direct connections to r3.o.lencr.org: nginx is configured for this:
## Enables OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;
I'll dig into this the next few days to review the nginx configuration.

best regards and special thanks for the quick help here,

Florian La Roche

Florian La Roche · Answer 11 · Tue May 31 2022 05:05:31 GMT+0800 (China Standard Time)

My goal was to check if all outgoing connections from the web-server could only be allowed
via the extra proxy server. As extra firewall configuration and security pre-caution.

Not sure I'll keep this goal up and persue it. Then also nginx would need to connect to the proxy server.

best regards,

Florian La Roche

github-actions · Answer 12 · Wed May 31 2023 10:01:32 GMT+0800 (China Standard Time)

We've made a lot of changes to Certbot since this issue was opened. If you still have this issue with an up-to-date version of Certbot, can you please add a comment letting us know? This helps us to better see what issues are still affecting our users. If there is no activity in the next 30 days, this issue will be automatically closed.

github-actions · Answer 13 · Sat Jul 01 2023 10:06:57 GMT+0800 (China Standard Time)

This issue has been closed due to lack of activity, but if you think it should be reopened, please open a new issue with a link to this one and we'll take a look.