reverse proxy self-hosted services with https, wildcard dns, and wildcard ssl certificate, without port forwarding and tunneled from a vps.

2020-07-05

Introduction

Even though most of my services are hosted in Virtual Private Servers (VSP) or even dedicated servers around, which are placed in environments that are designed for providing services publicly (data centers, networks, and providers), I enjoy self-hosting some services at home, from a Synology (but this could be any computer, including a Raspberry Pi). The advantages in terms of privacy are straightforward: your data is at home. Sometimes it is even useful to have these services limited to your Local Area Network (LAN) only. But sometimes, you really want them to be public facing. There are several techniques for accessing your LAN contents from the outside, e.g, using a Virtual Private Network (VPN), some of which are very interesting like ZeroTier. But sometimes you just want an easy way: you provide a URL, and that is it. One way to achieve this is to use a Dynamic DNS and port forwarding: your router is responsible to take all requests to your home IP and a certain port, to route them to the Synology and back. But, having open ports to the Internet is not always a good idea. I, for once, do not enjoy having my home public IP exposed, even if pointed by an obscure domain. Furthermore, many people are not in the position of being able to port forward properly. Some Internet Service Providers (ISP) have particular routing systems (such as Carrier-Grade NAT, CGNAT), where different users share the same public IP address. Who gets traffic to a shared IP? Not an easy answer. There is another way, which is the one I have explored recently: tunnel traffic from a remote machine, say, a VPS with its public IP address, to my Synology. This principle is not really new. The Secure Shell, or SSH, is capable to tunnel and forward TCP requests to-and-from different hosts, to reach its final destination. Some systems were built around this, for example Serveo (often down) and Localhost.run. Perhaps the most famous system that follows this tunnel concept, even if more elaborated, is ngrok. These systems allow you to keep your ports closed, but they work only with TCP, not with UDP. They sometimes cost money. And, they often work only for basic cases.

What this tutorial enables

After this tutorial, you will be able to have your several dockerized services (they do not have to be dockerized, actually), which listen locally to 127.0.0.1:port on your Synology and nowhere else, accessible through your domain name with form http://appname.domain.com, through HTTPS, with a Wildcard Let’s Encrypt SSL certificate and a Wildcard CNAME (no need to manually create a DNS record for all services you have), tunneled from a virtual private server anywhere on Earth. This without forwarding any port on your router.

Enter frp

frp (fast reverse proxy) is a combination of tunneling and reverse proxy system, so described:

A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

It supports both TCP and UDP tunnels, and for TCP tunnels it provides special wrappers for HTTP and HTTPS. There is also a Peer-to-Peer (P2P) mode. We will use it to act as a tunnel and reverse proxy, from a VPS to a Synology (or any other computer at home). It is pretty awesome! Go read all its use cases.

Graphical summary of results

Assumptions for the tutorial

For brevity, I will assume several details, here summarized. You might find them useful for adapting my settings to your use case.

Your Synology features several docker containers that listen to 127.0.0.1:port (any port), on the Synology itself.
You own a domain domain.com that you are going to use for accessing your apps.
You are using Synology Nginx reverse proxy (Control panel → Application Portal → Reverse Proxy), which listens at http://localhost:80 and https://localhost:443 and have your containers listed there in the form:
- Source: https://app1.domain.com
- Destination: http://localhost:port
You have a VPS with a public IP, with Docker installed.
You are setting a Wildcard DNS and SSL certificate:
- Your DNS has the following settings (here illustrated for Cloudflare)
- A RECORD, name domain.com, Content 111.111.111.111, TTL Auto, Proxy status does not matter (white cloud or orange cloud)
- CNAME Record, name * (which stands for *.domain.com), content @ (shown by Cloudflare as domain.com), Proxy status does not matter (white cloud or orange cloud)
You create a Wildcard Let’s Encrypt certificate with acme.sh (illustrated below)

SSL with acme.sh

Synology DSM is pretty nice as an easy to use operating system. DSM even handles certificates with Let’s Encrypt, but it does not support Wildcard certificates yet. acme.sh is an awesome utility, written in Bash, which acts as a client for the Let’s Encrypt system. One of its greatest features is the creation and renewal of Wildcard SSL certificates via DNS challenge. A DNS challenge works by dynamically changing DNS records instead of starting HTTP servers. This approach has two advantages for our case:

It does not require port forwarding.
It is compatible with Cloudflare proxy (orange cloud), in case you wish to use it. acme.sh talks to Cloudflare via its APIs.

Another plus: acme.sh can deploy the certificate to Synology DSM, which will see it as a normal system certificate.

Synology Installation

You can safely follow acme.sh installation instructions for Synology DSM. The quick steps:

wget https://github.com/acmesh-official/acme.sh/archive/master.tar.gz
tar xfvz master.tar.gz
cd acme.sh-master/
./acme.sh --install --nocron --home /usr/local/share/acme.sh --accountemail "[email protected]"
source /root/.profile

I will report just some tips.

Once you SSH to your Synology, obtain administration privileges using sudo su, not su -s. acme.sh will complain otherwise.
Create an admin user, let’s call it cert, with a strong password, and without 2-Factor authentication. This is the easiest way. You can keep 2-FA but things start to get complicated as you need to sniff cookies.
Related to the previous second one, is: Control Panel → User → Advanced and untick the option Enforce 2-step verification for the following users. This is not mentioned in acme.sh tutorial, and I wasted a lot of time before finding out that this option must be disabled, as well.
At the end of the installation, acme.sh will install itself at /usr/local/share/acme.sh.
You need to configure your DNS validation method for acme.sh. acme.sh supports several providers. Here we assume Cloudflare, as many take advantage of its free nameservers and proxy service.

DNS Settings (Cloudflare example)

For Cloudflare, create a token at https://dash.cloudflare.com/profile/api-tokens. Give the token Zone:Edit and DNS:Edit permissions. Take note of your token. You will also need your Cloudflare account ID, which can be seen on the right sidebar after accessing any of your Cloudflare domains.

Issue your Wildcard certificate

You can use the following bash script (fill in your token, account ID, and domain name).

#!/bin/bash
    
export CF_Token="YOUR_CLOUDFLARE_TOKEN"
export CF_Account_ID="YOUR_ACCOUNT_ID"

/usr/local/share/acme.sh/acme.sh \
--issue \
--dns dns_cf \
-d *.domain.com -d domain.com

At the end of the operation, you will find your certificates at /usr/local/share/acme.sh/*.domain.com/*.domain.com.cer.

Deploy certificate to Synology DSM

Deploy your certificate by following the official acme.sh instructions for Synology DSM. Quick steps illustrated here:

#!/bin/bash
export SYNO_Username='synologyadminuser'
export SYNO_Password='synologyadminpassword'
export SYNO_Scheme='http'
export SYNO_Hostname='localhost'
export SYNO_Port='5000'
export CERT_DOMAIN='*.domain.com' # This is the file to be looked for at /usr/local/share/acme.sh/
export SYNO_Certificate='domain.com' # This is the name of the certificate in DSM Security -> Certificate
export SYNO_Create=1
/usr/local/share/acme.sh/acme.sh --deploy -d ${CERT_DOMAIN} -d "*.${CERT_DOMAIN}" --deploy-hook synology_dsm

Please note that, if you use a port for Synology DSM different from the default one—which is a recommended security setting—you need to change the value for SYNO_Port to the port you use.

Have Synology DSM use the certificate

Under Control Panel → Security → Certificate, first verify that you have a new entry for *.domain.com. Hit the Configure button. For all your docker services, select the *.domain.com certificate. Hit OK.

Setup certificate renewal

Follow the official guide to set up certificate renewal. The gist of it: Control Panel → Task Scheduler → Create → Scheduled Task → User-defined script by user root, once per week, with the following command:

/usr/local/share/acme.sh/acme.sh --cron --home /usr/local/share/acme.sh/

Reverse proxy and tunnel with frp

You can download a compiled version of frp..or not. You can also just use a dockerized version of frp. I picked snowdreamtech/frps and its counterpart snowdreamtech/frpc because I like the way frps and frpc are configured. The images are also kept in sync with frp source code.

VPS side (front)

On your VPS (111.111.111.111), create the directory /root/frps. Inside it, create two files: frps.ini

[common]
bind_addr = 123.123.123.123
bind_port = 7000
vhost_http_port = 80
vhost_https_port = 443
subdomain_host = domain.com

start.sh

#!/bin/bash
/usr/bin/docker run --restart=always --network host -d -v ${PWD}/frps.ini:/etc/frp/frps.ini --name frps snowdreamtech/frps 

Run start.sh (give it execution permissions with chmod +x start.sh first). Verify with docker logs frps that frp is running fine.

Synology side (back)

Notice that we are using frpc here instead of frps. Create the directory /root/frpc. Inside it, create two files: frpc.ini

[common]
server_addr = 123.123.123.123
server_port = 7000

[app1]
type = https
use_encryption = false
use_compression = true
local_port = 443
local_ip = 127.0.0.1
subdomain = app1

[app2]
type = https
use_encryption = false
use_compression = true
local_port = 443
local_ip = 127.0.0.1
subdomain = app2

[app3]
type = https
use_encryption = false
use_compression = true
local_port = 443
local_ip = 127.0.0.1
subdomain = app3

Follow the template with all services you are hosting. Yes, use_encryption should be set to false, because HTTPS is already encrypted, no need for frp to further encrypt.

start.sh

#!/bin/bash
/usr/local/bin/docker run --restart=always --network host -d -v ${PWD}/frpc.ini:/etc/frp/frpc.ini --name frpc snowdreamtech/frpc

Run start.sh (give it execution permissions with chmod +x start.sh first). Verify with docker logs frpc that frp is running fine. What confused me at first was that I approached this with a reverse proxy mentality. I attempted to set up an SSL certificate on the front side, the VPS, to “reverse proxy” the HTTPS request. frp (and similar systems) do not use certificates on the front side. frps simply forwards the request to the receiving end, frpc, which forwards it to the endpoint, in this case Synology DSM nginx localhost server, at port 443. As frp forwards HTTP(S) headers, too, the receiving end, nginx, knows to whom forward the request to.

Profit

Access your services by visiting URLs like https://app1.domain.com. They will appear to come from your VPS (or even from Cloudflare IPs if you use Cloudflare proxy, the orange cloud). In a sense, they are, through frp. But, they run in your Synology at home. End-to-end encrypted. Without port forwarding. Magic! By using a system such as frp instead of simpler systems such as Serveo, I am also ready to forward non-HTTP TCP requests as well as UDP requests. Please donate to the authors of these two wonderful, free and open source, pieces of software.

Donate to acme.sh.
Donate to frp.

I do not use a commenting system anymore, but I would be glad to read your feedback. Feel free to contact me.