Getting HTTPS with SSL/TLS certificates from LetsEncrypt using Docker

What’s up

For this Long Ass post I’ll assume a basic understanding of docker and nginx. I will be using docker version 18.09.3, docker-compose version 1.18.0, and nginx (free edition) version 1.15.9. I will be using a prebuilt version of nginx which incorporates letsencrypt from linuxserver.io that can be found at the docker hub to generate SSL server certificates.

What and Why

HTTPS establishes a secure, encrypted line of communication between your browser and the server it is connecting to and sharing information with. That information, whether it be credit card information or a recipe for keto bread, should by default be scrambled in such a way as to be not readily available to anyone who might intercept it.

Or so says the folks who make the internet happen. HTTPS is a basic, standard element of the web’s infrastructure and implementing it has become a requirement for any website or webapp that is communicating with the public and has become very easy to do thanks to letsencrypt.

Even if you are just publishing a basic blog that you don’t expect many people to visit and you won’t be doing anything like asking them for their email, using HTTPS is kind of like covering your mouth when you cough. It’s easy, free, and makes everyone a little safer.

The Method I Use

As I describe in Multiple Websites and Webapps Served with Nginx and Docker as Non Root User, I prefer to use a primary nginx docker instance to handle only requests and therefore HTTPS communication and then secondary nginx containers to proxy to various servers, such as the one running this hexo blog instance.

Though under the hood this seems to add some unnecessary overhead in terms of process, it ends up being much easier to build and deploy as well as troubleshoot. The secondary nginx and server instances are very modular as well. And the certificates are all kept in a single volume attached to the main nginx instance.

So for this method you will need a docker volume, which we will call ssl-vol, that is used in production to store certificates so they can be used by the main nginx instance. In this volume there are seperate directories for your domains and subdomains that are being served. I’ll call the directories blog and wiki, which are subdomains of different domains, so they must be verified by letsencrypt and handled by nginx seperately. Finally we have a volume called letenc which is used only with the linuxserverio instance to hold the downloaded files.

When obtaining certificates I therefore use the linuxserver.io container as a secondary instance, as the graphic above shows, temporarily bypassing the actual site or app that is being secured. I am not so concerned about running this temporary container as root, so only the environment variables will need to be changed, although this does mean we will have to be careful about permissions when everything is done.

Overview

  1. Make letsencrypt.conf site configuration file for main nginx instance.

  2. Edit docker-compose file for linuxserver/letsencrypt docker container with the correct environment parameters and staging set to true

  3. Run compose then reload main nginx container to make sure the linuxserverio container is reachable from the address we are obtaining the certs for.

  4. Erase staging files, comment staging out, and run container for reals.

  5. Transfer files to correct directory, fix permissions, change ssl.conf to point to the directory

  6. Make site configuraton file for main nginx instance that redirects HTTP to HTTPS, points to correct directory for SSL certs, and proxies to the correct server.

  7. Revel in your awesomeness!!!!!

The Primary Config

The first step is making an site config file for the primary nginx instance that will proxy to the linuxserver.io container. This will direct the request from letsencrypt as they verify your ownership of the site.

I have a file called letsencrypt.config that I keep as letsencrypt.config.bak when not in use and just change the config’s server_name to whatever site name I am certifying. For the sake of example I’ll use this site’s address.

LetsEncrypt Configview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
server {

listen 80;
server_name blog.digitalap3.com;

access_log /var/log/nginx/nginx-access.log;
error_log /var/log/nginx/nginx-error.log;

location \ {
proxy_set_header Host $host;
proxy_pass http://linuxserverio/; #name of container on same network!
}

# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root ./static/;
}
}

Note we are making this file on a running instance and NOT yet reloading it, since the linuxserverio server it is referring to is not up and so reloading our main nginx would cause an error.

The LinuxServerio Container

The people at linuxserver.io have made this process so much easier for those of us who love to use docker. All we need to do is write up a simple docker-compose script, fill in some parameters, and let their setup do the heavy lifting. You could just as easily use a docker run command, but I prefer the organization of the docker-compose yml format. The primary element of note is the environment section. The one below is for the subdomain that this blog is served on.

Linuxserverio Docker-Composeview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
version: '3'

services:
linuxserverio:
image: linuxserver/letsencrypt
container_name: linuxserverio
volumes:
- "letenc:/config" # files will be created here
- "ssl-files:/sslfiles"
- "./index.html:/usr/share/nginx/html/index.html"
- "./default.conf:/etc/nginx/conf.d/default.conf"
- "./nginx.conf:/etc/nginx/nginx.conf"
expose:
- "80"
networks:
- dockernetwork
environment:
- PUID=1000
- PGID=1000
- EMAIL=hello@digitalap3.com
- URL=digitalap3.com
- SUBDOMAINS=blog
- TZ=America/NewYork
- VALIDATION=dns
# - DNSPLUGIN=cloudflare # for wildcard subdomain
- STAGING=true

volumes:
letenc:
external: true
ssl-files:
external: true
networks:
dockernetwork:
external: true

Note that the STAGING is set to true. This is IMPORTANT! It tells letsencrypt that we are just checking things out because we haven’t been able to proxy their request to the linuxserverio container yet. Also be aware of the environment section and change the relevant parameters. There are the obvious ones to change and some that may need to be added or subtracted.

Also you can see how three files are inserted into the container in the volume section. I find this easier than copying them into a new image. For the sake of brevity, you can open them in a new tab and copy paste if you need to, or just change the indicated lines:

default.conf : the server_name is set to the generic underscore since this file is referred to by proxy.

nginx.conf : I kept having an issue with the the pid file not being referenced correctly - so I added line 17 to correct the problem.

index.html : this inserts the nginx welcome message to be referred to by default.conf, thus showing us that we are able to access the domain.

So let’s get crazy and spin it up:

docker-compose -f compose.letsencrypt.yml up

The glorious flow of code will commence and the server will populate the letenc volume mounted to the linuxserverio container as the config directory with relevant but non functioning, ie staged, files. You should have an error as well since the main nginx hasn’t been reloaded and letsencrypt was unable to verify your domain.

So before we go further, exec into the main nginx container as root and nginx -t then nginx -s reload, then point your browser at the domain or subdomain you are securing and make sure you get the default nginx page. This ensures that letsencrypt is able to verify your ownership.

Start and stop the linuxserverio container and watch for any error messages.

If all looks well then edit compose.letsencrypt.yml to comment out staging:

#STAGING= true

It’s probably not necessary but I like to rm -rf all the staging files and start over with a clean directory. Run docker-compose... again and you have the files you need!

Since both volumes are mounted to the linuxserver container, docker exec -it in and move the files to the blog dir.

However, though these files are in the right directory, since they were formed by a root process we need to change the permissions. I just exec into the main nginx container as root and run the necessary chown commands.

Take the linuxserverio container down, rename the config file to ...bak and let’s make an nginx config file for the subdomain that redirects to HTTPS.

## The Configuration Files
Here is what mine looks like. Again, this is the main nginx server config file so it is proxying to another nginx instance on the same docker network.

Main Blog Confview raw
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
server {

listen 8080; # remember we redirect 80 to 8080
server_name blog.digitalap3.com;
return 301 https://$host$request_uri;
}

server {
listen 4443 ssl; # remember we redirect 443 to 4443
server_name blog.digitalap3.com;

include /ssl-files/blog/nginx/proxy-confs/*.subfolder.conf;

include /ssl-files/blog/nginx/ssl.conf;

client_max_body_size 4G;

access_log /var/log/nginx/nginx-access.log;
error_log /var/log/nginx/nginx-error.log;

location / {
proxy_set_header Host $host;
proxy_pass http://hexonginx/;
}

# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root ./static/;
}
}

But wait!! We’re not done! There’s always one more thing to do. Sometimes development work is like zeno’s arrow - always getting halfway to the target.

We need to change the default paths in the ssl-files/blog directory referred to in the config file above. Specifically in the file /nginx/ssl.conf. Here are the 3 relevant lines changed to match the directory structure reflected in the above nginx config.

SSL Conf Changesview raw
1
2
3
4
5
6
7
8
9
10
11
12
## Version 2018/05/31 - Changelog: https://github.com/linuxserver/docker-letsencrypt/commits/master/root/defaults/ssl.conf



# Diffie-Hellman parameter for DHE cipher suites
ssl_dhparam /ssl-files/blog/nginx/dhparams.pem;

# ssl certs
ssl_certificate /ssl-files/blog/keys/letsencrypt/fullchain.pem;
ssl_certificate_key /ssl-files/blog/keys/letsencrypt/privkey.pem;

# protocols

With all this done test and reload nginx then point your browser at your domain without https:// to ensure the redirect and bask in your heroic glory for you have helped make the internet a safer place!!

Then go over to letsencrypt and give ‘em a couple of bucks for the great work they are doing.