Let’s Encrypt is an excellent service offering the ability to generate free SSL certs. The way it normally works is using http-01 challenge… to respond to the Let’s Encrypt challenge the client (typically Certbot) puts an answer in the webroot. Let’s Encrypt makes an http request and if it finds the response to the challenge it issues the cert.
Update: 2023-03-21: Updated the post to change a few links that no longer existed to more relevant ones.
Certbot is great for public web-servers.
Generating Intranet SSL Certs Using DNS-01 Challenge
But what if you’re generating an SSL certificate for a mail server, or mumble server, or anything but a web server? You don’t want to spin up a web server just for certificate verification.
Or what if you’re trying to generate an SSL certificate for an intranet server Many homelabs, organizations and businesses need publicly signed SSL certs on internal servers. You may not even want external A records for these services, much less a web-server for validation.
ACME DNS Challenge
Fortunately, Let’s Encrypt introduced the DNS-01 challenge in January of 2016. Now you can respond to a challenge by creating a TXT record in DNS.
Lukas Schauer wrote dehydrated (formerly letsencrypt.sh) which can be used to automate the process.
Here’s a quick guide on Ubuntu 16.04, but it should work on any Linux distribution (or even FreeBSD).
Install dehydrated / letsencrypt.sh
sudo su mkdir /etc/dehydrated cd /etc/dehydrated mkdir certs accounts cd /opt git clone https://github.com/lukas2511/dehydrated.git cd dehydrated cp docs/examples/config /etc/dehydrated/ cp docs/examples/domains.txt /etc/dehydrated ln -s /opt/dehydrated/dehydrated /usr/local/bin/
Hook for DNS-01 Challenge
At this point, you need to install a hook for your DNS provider. If your DNS provider doesn’t have a hook available, you can write one against their API or switch to a provider that has one.
If you need to pick a new provider with a proper API my favorite DNS Providers are Cloudflare DNS (free) and Amazon Route53 (small cost). Cloudflare is what I use for b3n.org. It is always the fastest DNS provider on DNSPerf, gets consistently low latency lookup times, and it’s free. Route53 is one of the most advanced DNS providers. It’s not free but usually ends up cheaper than most other options and is extremely robust. The access control, APIs, and advanced routing work great. I’m sure there are other great DNS providers but I haven’t tried them.
Here’s how to set up a CloudFlare hook as an example:
cd /opt/ git clone https://github.com/kappataumu/letsencrypt-cloudflare-hook apt install python3 python3-pip pip3 install -r letsencrypt-cloudflare-hook/requirements.txt
In letsencrypt-cloudflare-hook/hook.py change the top line to point at python3:
#!/usr/bin/env python3
Config File
Edit the “/etc/dehydrated/config” file… add or uncomment the following lines:
CHALLENGETYPE="dns-01" CERTDIR="${BASEDIR}/certs" ACCOUNTDIR="${BASEDIR}/accounts" HOOK=/opt/letsencrypt-cloudflare-hook/hook.py [email protected] export CF_EMAIL='[email protected]' export CF_KEY='your_cloudflare_API_key'
domains.txt
Create an /etc/dehydrated/domains.txt file, something like this:
gitlab.b3n.org emby.b3n.org stor1.b3n.org stor2.b3n.org b3n.org www.b3n.org dev.b3n.org www-dev.b3n.org
The first four lines will each generate their respective certificates, the last line creates a multi-domain or SAN (Subject Alternate Name) cert with multiple entries in a single SSL certificate.
Finally, run
dehydrated -c
The first time you run it, it should get the challenge from Let’s Encrypt, and provision a DNS TXT record with the response. When validated the certs will be placed under the certs directory and from there you can distribute them to the appropriate applications. The certificates will be valid for 90 days.
For subsequent runs letsencrypt.sh will check to see if the certificates have less than 30 days left and attempt to renew them.
Automate
It would be wise to run dehydrated -c from cron once or twice a day and let it renew certs as needed.
To deploy the certs to the respective servers I suggest using an IT Automation tool like Ansible. I have a dedicated VM that runs Ansible. You can configure an ansible playbook to run from a daily cron job to copy updated certificates to remote servers and automatically reload services if the certificates have been updated. Here’s an example of an Ansible Playbook which could be called daily to copy certs to all web-servers and reload nginx if the certs were updated or renewed:
Create a file web-servers-nginx.yml
- hosts: web-servers-nginx tasks: - name: Copy SSL certificates copy: dest: /etc/dehydrated/certs/ src: "/etc/dehydrated/certs/{{ item }}" mode: 0600 follow: yes with_items: "{{ dehydrated_ssl_certs }}" register: sslcerts - name: Reload Nginx when certs change service: name=nginx state=reloaded when: sslcerts.changed
Add the below to your Ansible inventory file (mine is namned ‘production’). “b3n.org” matches the primary name of the certificate, found in /etc/dehydrated/certs/
[web-servers-nginx] nyc2.b3n.org dehydrated_ssl_certs='["b3n.org"]'
Execute the playbook with:
ansible-playbook -i production web-nginx-servers.yml
(note that the user that runs this needs to have permissions to read the certificates that dehydrated generated. Easiest way to do that is to use the same user account to run dehydrated as you do for Ansible. Also Ansible will need public/private key authentication setup to connect to the remote server without a password).
Then obviously you would have something like this in nginx:
ssl on; ssl_certificate /etc/dehydrated/certs/b3n.org/fullchain.pem; ssl_certificate_key /etc/dehydrated/certs/b3n.org/privkey.pem; ssl_stapling on; ssl_stapling_verify on; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH'; ssl_dhparam /etc/nginx/ssl/dhparam.pem;
(for the ssl_dhparam to work you’ll need to run the below command once on the web server):
cd /etc/ssl/certs openssl dhparam -out dhparam.pem 4096
And after that nginx needs to be restarted.
If this is a public server, I strongly suggest testing with SSLLabs to make sure your chaining and security are setup correctly.
Traceback (most recent call last):
File “/srv/dehydrated/letsencrypt-cloudflare-hook/hook.py”, line 203, in
main(sys.argv[1:])
File “/srv/dehydrated/letsencrypt-cloudflare-hook/hook.py”, line 199, in main
ops[argv[0]](argv[1:])
File “/srv/dehydrated/letsencrypt-cloudflare-hook/hook.py”, line 167, in create_all_txt_records
create_txt_record(args[i:i+X])
File “/srv/dehydrated/letsencrypt-cloudflare-hook/hook.py”, line 105, in create_txt_record
zone_id = _get_zone_id(domain)
File “/srv/dehydrated/letsencrypt-cloudflare-hook/hook.py”, line 83, in _get_zone_id
return r.json()[‘result’][0][‘id’]
IndexError: list index out of range
Is not support for intranet ?
thanks
Sorry I didn’t see your comment sooner, ducpm. Did you get it working? This is most likely fixed by specifying a particular version of python on the shebang (first line) or the code may need to be updated to work with the version of Python you’re using (you may also want to report the issue on the GitHub repo for the author of the code, I believe most have already updated their code to work with later versions of python so you may need to specify python3). If you still need help let me know your distro, version of python, and point me at the source of the hook and I’ll take a look.
The solution only works if the domain name is actually managed by the DNS service used for the text records. Of course this still makes it of limited use to people that want to generate SSL certificates for servers on internal servers that have (internal) DNS names that follow the same pattern as external addresses – e.g. my-internal-server.mypublicdomain.com. But not for servers that have addresses like http://my-internal-server/
Also, there’s a typo in the ‘how to set up a CloudFlare hook’ example, ‘isntall’ where it should say ‘install’, but possibly an issue for the copy/paste-crowd.
Thanks for the heads up on the typo! That’s correct. But nobody should be running servers on http://my-internal-server. Best practice is servers should always be accessed using a registered domain, and always accessed using the fqdn (not just the hostname). One reason to register a domain is you’ll guarantee that your intranet domain will never conflict with a public domain in the future (especially now that ICANN is allowing lots of tlds). The second reason is if anyone other than you (say a consultant or a friend) is on your intranet and needs to access an intranet server; if it’s not a real domain the only way they can validate your SSL certs is they would have to install and trust your internal CA cert on their computer which is a security risk for them since it essentially allows you create fake certs that would let you MITM any domain.
Hello, thnks for this very nice article.
How can this process be semiautomated by manually inserting the TXT instead of using a hook?
Thanks