Certbot ACME Client embedded/IoT integration utility
====================================================
Certbot is a most powerful ACME client for Let's Encrypt certificate
authority with lot of domain authentication and service configuration
plugins.
Written in Python with a lot of dependencies it might be unsuitable
for use directly in embedded and IoT world.
On the other hand it might be undesirable for large IoT deployments to
directly contact Let's Encrypt servers playing at the edge of their
rate limits.
This utility tries to address both of the above and present POSIX shell
helper that uses certbot to talk with Let's Encrypt and presents proxy
infrastructure to issue certs for thousands of IoT devices.
Features
--------
o Single/Multiple account(s) access to Let's Encrypt services from
controllable set of hosts acting as proxy for IoT devices
o Minimal dependencies for proxy client on target devices:
- POSIX compatible /bin/sh
- install
- mktemp
- mv
- rm
- chmod
- cmp
- openssl
- wget/curl
most of which may be available through busybox or similar toolboxes.
o Standard certbot utility command line interface, no extra options.
o Uses DNS-01 authenticator and can be integrated with any DNS service
hosting providers (not only with explicitly supported by certbot
plugins) through CNAMEs pointing to different domain or delegated
subdomain that supports rfc2136 compliant dynamic updates.
o Run Certbot ACME Client under specified unprivileged user
(letsencrypt) on proxy host for improved security
o Supplementary group (certbot) whose members can execute certbotsh
to allow set of unprivileged users to access certbot instance under
specified user
o All in one script containing installer and client/server applets
to simplify deployments at both client (e.g. IoT device) and servers
Requirements
------------
Domain or subdomain must be up and running either on hosting provider
provided authoritative nameservers or privately controlled (delegated
domain).
While proxy server side configuration tested only on RHEL/CentOS 7.x and
8.x releases it should work as well on other Linux (or BSD) systems
possibly with slight modifications.
Client side has it's minimal requirements described in Features section.
Description
-----------
There are multiple usage scenarious possible (e.g. simple certbot
wrapper to run under unprivileged user) with this utility but most
common is a proxy between Let's Encrypt and own services consuming
authority issued certificates.
Consider following diagram with example.com as domain:
+----------------------------+ CNAMEs -> TXTs
| Hosting provider | _acme-challenge.www -> www._acme-le
| authoritative DNS servers | ...
| managed by customer via | _acme-challenge -> self._acme-le
| some UI (e.g. web portal) |
| | +------------------------------+
| +-----------------+ CNAME | | Let's Encrypt infrastructure |
| | ns1.hosting.net |<.......o.. | |
| +-----------------+ | . | +-------------------+ |
| +-----------------+ CNAME | .......o..>| DNS-01 Validation | |
| | ns2.hosting.net |<.......o.. . | +---------^^--------+ |
| +-----------------+ | . | || |
+----------------------------+ . | +---------vv--------+ |
DNS . | | ACME frontend | |
. | +-------------------+ |
+--------------------------------+ . | ^ |
| certbotsh server (proxy) | . +--------------o---------------+
| | . .
| +-------------------------+ TXT| . .
| | ns._acme-le.example.com |<...o... .
| +-------------------------+bind| .
| +----------------------------+ | HTTP/ACME .
| | acme-le.gw.api.example.com |<o....................
| +----------------------------+ | certbot
| ^ lighttpd |
+--------------o-----------------+
.
...................
.
+--------------------------------o----------------------------------+
| certbotsh client(s) . |
| . HTTP GET over TLS/SSL (HTTPS) |
| .................................................... |
| . . . . . . |
| +---v---+ +---v---+ +--v--+ +---v----+ +--v--+ +--v---+ |
| | dev-1 | | dev-2 | | www | | portal | | ftp | | smtp | |
| +-------+ +-------+ +-----+ +--------+ +-----+ +------+ |
| ph1 ph2 ph3 ph4 ph5 ph6 |
| |
| Fetch & unpack PKCS#12 encrypted with per client passphrase (phX) |
| file with privkey, chain, cert to /etc/letsencrypt/live/<fqdn> |
| to retain compatibility with certbot filesystem pathes and naming |
| |
+-------------------------------------------------------------------+
In this diagram certbotsh server side running on RHEL/CentOS 7.x or 8.x
with following components:
ns._acme-le.example.com
~~~~~~~~~~~~~~~~~~~~~~~
ISC BIND 9.11.x supporting rfc2136 dynamic updates
authoritative master (only) DNS name server serving
_acme-le.example.com subdomain zone.
Example of zone file, named(8) config snippet among with
README file with instructions on how to configure BIND
can be found in ~${runas}/extra/bind subdirectory.
acme-le.gw.api.example.com
~~~~~~~~~~~~~~~~~~~~~~~~~~
Let's Encrypt ACME gateway for domain example.com running
certbot to get, update revoke certificates.
Single LE account for whole domain created by certbot
on this host.
Deploy hook (certbotsh-publish) is used to create PKCS#12
with privkey, chain and cert files created by certbot.
Either certbot-renew.timer or cron job takes care on
updating certificates that is near to expiry and not
revoked.
Each PKCS#12 file encrypted with per client passphrase
that is only known to this gateway/proxy and client this
certificate intended for. Strong cryptography algorithms
(AES256-CBC with SHA256) used for this by default and
can be adjusted in specific config files (publish.cfg)
if necessary.
At certificate issue time client hostname (domain) should
be known and resolvable to all IPv4 and IPv6 addresses as
specific configuration for lighttpd implemented to ensure
that client can only download it's own PKCS#12. Download
is done using HTTP GET method over TLS/SSL secure channel
with acme-le.gw.api.example.com certificate issued by
trusted certificate authority (e.g. Let's Encrypt).
From certbotsh client side there is two actions required
1) Download and unpack PKCS#12 files to /etc/letsencrypt/live/<fqdn>
subdirectory (see certbotsh-update). There is crontab(5) file to
run certbotsh-update periodically to pull updated certificates
from gateway/proxy.
2) Execute pre, post and deploy hooks (see certbotsh-hook) to
stop service, install certificates and start service that
uses them.
In most cases certbotsh-hook $runas root to make sure it
have privileges necessary to restart service. It is started
by certbotsh-update and switched to root using sudo(8).
Security
--------
Proxy configuration shipped with secure by default.
However additional configurations like firewall and HTTP
authentication might be implemented if desired (however last one
has very little impact on security as compromise of single
certbotsh client would give access to HTTP authentication
credentials).
Whenever possible least privilege principle used (e.g. certbot
is running as unprivileged user to perform all actions).
PKCS#12 files encrypted with strong cryptography algorithms
(AES256-CBC with SHA256 for MAC by default) using per client
passphrase shared only between proxy and client so that compromise
of single client passphrase does not compromise the rest.
PKCS#12 files can only be downloaded by clients they intended to.
This check is done by lighttpd based on network (IPv4 and/or IPv6)
address of connecting client. Therefore whole TCP session needs to
be spoofed to access PKCS#12 of other client.
Lighttpd configured with minimal set of modules with HTTP to HTTPS
redirect and HTTP Strict Transport Security. Server certificate
must be issued by trusted by certbotsh clients authority (e.g.
Let's Encrypt). Only static content is served by lighttpd.
Digest authentication might be enabled to enhance protection
in environments where more specific network address prefix can be
injected (way to spoof TCP session) to make possible for unauthorized
parties to download PKCS#12 file. Password for digest authentication
is shared among all trusted hosts therefore it's compromise on single
host will reveal access from others.
Where appropriate (e.g. hooks running by certbotsh-update) code
injection protection mechanisms applied.
Usage
-----
usage: [vars] certbot.sh install [client|server|all]
If argument to install is not specified 'client' is assumed.
Accepted environment variables (e.g. in [vars]) with their defaults are
runas=letsencrypt
User to install (and run) services as
certmgr=certbot
Group whose members allowed to run services
httpd_domain=example.com
Domain used for default (wildcard) server
httpd_hostname=acme-le.gw.api.$httpd_domain
Virtual host name to configure in httpd
Report bugs to http://github.com/serhepopovych/certbotsh
Examples
--------
(1) Deploy gateway/proxy acme-le.gw.api.example.com on CentOS 7.x or 8.x
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Install EPEL
$ sudo yum install -y epel-release && sudo yum update -y
# Install dependencies
$ sudo yum install -y diffutils psmisc curl lighttpd certbot
# Install certbot-dns-rfc2136 plugin on
# CentOS 7.x (python2) or CentOS 8.x (python3)
$ sudo yum install -y python2-certbot-dns-rfc2136 ||
sudo yum install -y python3-certbot-dns-rfc2136
# Fetch certbotsh
$ curl -s -o ~/certbot.sh \
https://raw.githubusercontent.com/serhepopovych/certbotsh/master/certbot.sh &&
chmod 0755 certbot.sh
# Install certbotsh
$ sudo \
httpd_domain='example.com' \
httpd_hostname='acme-le.gw.api.example.com \
~/certbot.sh install server
# Patch dns_rfc2136.py to support CNAME/DNAME lookups
# CentOS 7.x (python2) or CentOS 8.x (python3)
$ if cd /usr/lib/python2.7/site-packages/certbot_dns_rfc2136/_internal ||
cd /usr/lib/python3.6/site-packages/certbot_dns_rfc2136/_internal
then
sudo patch <${s}${n}
cd - >/dev/null
fi
# Needed to recompile .pyo/.pyc as we run as unprivileged user later
sudo /usr/bin/certbot -h all
# Make sure to configure at least keyname and secret in
/var/lib/letsencrypt/.config/letsencrypt/rfc2136.ini
with values from step(2)
(2) Deploy _acme-le.example.com on CentOS 7.x or CentOS 8.x
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Install ISC BIND with chrooted configuration
$ sudo yum install -y bind-chroot
# Get /var/lib/letsencrypt/extra/bind/*
scp acme-le.gw.api.example.com:/var/lib/letsencrypt/extra/bind/* ~
# Generate TSIG key
$ sudo \
sh -c 'tsig-keygen -a hmac-sha256 acme-le.gw.api.example.com-key >>/etc/named.tsig.key'
# Install named(8) config snippet
$ sudo install \
-m 640 -o root -g named \
~/named._acme-le.zones /etc/named._acme-le.zones
# Add named.tsig.key and named._acme-le.zones to /etc/named-chroot.files
$ grep '/etc/named\.tsig\.key' /etc/named-chroot.files ||
sudo sh -c 'echo "/etc/named.tsig.key" >>/etc/named-chroot.files'
$ grep '/etc/named\._acme-le\.zones' /etc/named-chroot.files ||
sudo sh -c 'echo "/etc/named._acme-le.zones" >>/etc/named-chroot.files'
# Make sure to add following configuration to /etc/named.conf
options {
/* This is needed for dns-rfc2136 plugin patch to resolve CNAMEs */
allow-recursion { <host -t a acme-le.gw.api.example.com>; };
recursion yes;
};
include "/etc/named.tsig.key";
include "/etc/named._acme-le.zones";
# Install named(8) zone file and adjust Resource Records (RRs) to point
# to correct IP addresses instead of 127.0.1.1 (edit named._acme-le.example.com)
$ sudo install -m 0644 -o named -g named \
~/named._acme-le.example.com \
/var/named/named._acme-le.example.com
# Make sure dynamic zone updates can be commited by named(8)
$ sudo chown root:named /var/named && sudo chmod 1770 /var/named
# Start named(8)
$ sudo systemctl enable --now named-chroot
# At this point you can configure delegation of _acme-le.example.com
# from your hosting provider to ns._acme-le.example.com.
(3) Request trusted certificates for acme-le.gw.api.example.com and start lighttpd
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# Note that when certbot is running as root it will switch to unprivileged
# user as certbot is an "alias" installed in /usr/local/bin/certbot and
# /usr/local/sbin/certbot.
#
# This will work only when /usr/local/bin and/or /usr/local/sbin preceeding
# /usr/bin, /bin and /usr/sbin, /sbin in PATH variable.
#
# If this fails to obtain certificates check _acme-le.example.com BIND settings,
# subdomain _acme-le.example.com delegation from your hosting provider and
# ~letsencrypt/.config/letsencrypt/rfc2136.ini. Make sure CNAMEs for
# _acme-challenge.acme-le.gw.api.example.com and _acme-challenge.example.com
# setup correctly and point to (non-existent as certbot will create them with
# dynamic updates) TXT records in _acme-le.example.com domain.
# Request certificates from Let's Encrypt for proxy/gateway (including wildcard)
$ sudo certbot -d acme-le.gw.api.example.com -d '*.example.com'
# Point lighttpd certificate dirs to /etc/letsencrypt/live
$ sudo ln -sf ../../letsencrypt/live/example.com /etc/lighttpd/pki/
$ sudo ln -sf ../../letsencrypt/live/acme-le.gw.api.example.com /etc/lighttpd/pki/
$ sudo ln -sf example.com /etc/lighttpd/pki/wildcard
# Adjust SELinux settings for lighttpd
$ sudo yum install -y policycoreutils
$ sudo setsebool -P httpd_setrlimit 1
# Start lighttpd(8)
$ sudo systemctl enable --now lighttpd
(4) Install certbotsh client side
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
It is fairly simple but depends on client type. At least you need to copy
certbot.sh to client, run installation and configure hooks:
# Fetch certbotsh
$ curl -s -o ~/certbot.sh \
https://raw.githubusercontent.com/serhepopovych/certbotsh/master/certbot.sh &&
chmod 0755 certbot.sh
# Install certbotsh
$ sudo ~/certbot.sh install client
# Use example ~letsencrypt/.config/letsencrypt/update.cfg to configure hooks.