has_many :codes

Building DynaSite in public: custom domains support

Published  

Update: see the next post on how I built a landing page and waitlist for DynaSite, to capture email addresses of people who might be interested in the product while I build it.

Today I am going to talk about how I implemented custom domains support in DynaSite. See the previous post where I talk about my plans to relaunch DynaBlogger as DynaSite, with new features including an AI-enhanced page builder.

Since I launched DynaBlogger, one of the most common questions I get asked is "how did you implement custom domains?"

The main reason I get asked this is TLS/SSL and certificates. To implement custom domains in your SaaS you need to somehow provision certificates for these domains.

Also the way you provision certificates must be able to scale regardless of how many customers (and thus how many custom domains) your SaaS needs to handle. This is a problem even with popular PaaS like Heroku because there's a limit with the number of custom domains that they can support for a single app. In some cases you can contact support to raise these limits but not sure up to what extent.

An alternative is to build something yourself based on Let's Encrypt to be able to offer certificates for free, especially if you manage the apps on your servers, but it can get complicated as your infrastructure grows and you need to manage and use certificates with many servers and load balancers.

For DynaBlogger, the platform was hosted on a Kubernetes cluster in Hetzner Cloud (created with my open source tool https://github.com/vitobotta/hetzner-k3s), therefore the natural choice for me to manage free Let's Encrypt certificates was cert-manager.

The way the basic functionality works is quite simple: 

  • user adds a custom domain to their blog
  • the app creates an ingress resource in Kuberrnetes for that domain with support for unencrypted traffic on port 80
  • the app exposes a special, unique and difficult to guess path at http://domain.com/verification/<some long UUID>
  • the app polls every few seconds that path and checks the response. If the response contains a matching key, then it flags the domain as verified because it means that the domain is correctly pointing to our app. The domain remains in pending verification state in the UI until this condition is met
  • once the domain is verified, the ingress resource is "upgraded" with support for encrypted connections, meaning that it adds to the ingress resource the required annotation to trigger a certificate issuance request to Let's Encrypt  via cert-manager
  • once the certificate is issued successfully, the domain is marked as ready and the app can start serving the user's content from their custom domain
  • when a custom domain is removed by the user, the relevant ingress resource is also deleted

You might ask, why perform a verification ourselves if Let's Encrypt already does this? The reason is that there are some rate limits that take into account also verification failures. So before even letting cert-manger request a certificate, I make sure that the domain is in fact pointing to my app.

I might also add that to make the verification more robust, and ensure that the user who is currently trying to use the domain owns it, you might want to use some sort of DNS based verification instead, which can help prevent a form of attacks called "subdomain takeover".

This process is quite simple and doesn't require much coding and can scale a lot with a Kubernetes cluster regardless of how many nodes etc you have; once you reach the limit of how many virtual hosts the ingress controller can handle, you can scale with multiple ingress controller, and so on.

The next question I get asked often is, "how do I do implement this stuff if I don't use Kubernetes?"

These past few evenings I have started to work on the DynaBlogger relaunch as DynaSite, but until the actual launch I won't be setting up a Kubernetes cluster and will use a single server to cut costs until proper scaling is required.

At the moment I am using Dokku on this single server to host the app, and I had to figure out how to handle custom domains while I keep this simpler setup. I evaluated a few options, and then finally settled on Cloudflare for SaaS since I already use Cloudflare and it's a brilliantly simple solution. I am probably going to keep this even when I launch since I really like it as a solution and doesn't require my users to use Cloudflare for their domains. 100 hostnames are free and after that it's just $0.10 per hostname.


Cloudflare for SaaS

If you don't have a Cloudflare account yet, you'll need to create one (there's a free plan) and migrate your domain's DNS configuration to Cloudflare.

Once the domain is ready, you need to configure a DNS record pointing to your SaaS. Cloudflare will forward any traffic for custom hostnames to the hostname you configure in this record. Make sure the proxy is enabled for this record (orange cloud).


 
cloudflare-proxy.png 5.1 KB




Next, head to

cloudflare-custom-hostnames-01.png 20 KB



and enable the feature, then add a "fallback origin":

fallback-origin.png 82.6 KB



Once it's active, you can start adding custom domains for your users. For simplicity, I use the HTTP verification method:


new-hostname.png 186 KB


Give it a few moments, and then the hostname will appear as ready:

hostname-ready.png 9.63 KB



Dokku


To complete the configuration, we need to remove the default site virtual host configured in Nginx:

rm /etc/nginx/sites-enabled/default
dokku nginx:stop
dokku nginx:start

And create a new virtual host that points to your SaaS app, which will be a "catch-all" for domains not directly configured here. Create this file at /etc/nginx/conf.d/00-default-vhost.conf:

server {
  listen 80 default_server;
  listen [::]:80 default_server;
  listen 443 ssl default_server;
  listen [::]:443 ssl  default_server;
  server_name _;

  access_log  /var/log/nginx/dynasite-access.log;
  error_log   /var/log/nginx/dynasite-error.log;

  ssl_certificate           /home/dokku/dynasite/tls/server.crt;
  ssl_certificate_key       /home/dokku/dynasite/tls/server.key;
  ssl_protocols             TLSv1.2 TLSv1.3;
  ssl_prefer_server_ciphers off;
  keepalive_timeout   70;

  location    / {
    gzip on;
    gzip_min_length  1100;
    gzip_buffers  4 32k;
    gzip_types    text/css text/javascript text/xml text/plain text/x-component application/javascript application/x-javascript application/json application/xml  application/rss+xml font/truetype application/x-font-ttf font/opentype application/vnd.ms-fontobject image/svg+xml;
    gzip_vary on;
    gzip_comp_level  6;

    proxy_pass  http://dynasite-3000;

    proxy_http_version 1.1;
    proxy_read_timeout 60s;
    proxy_buffer_size 4096;
    proxy_buffering on;
    proxy_buffers 8 4096;
    proxy_busy_buffers_size 8192;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $http_connection;
    proxy_set_header Host $http_host;
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Port $server_port;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header X-Request-Start $msec;
  }
}

Of course replace "dynasite" with the name of your app, and "3000" with its correct container port.

Finally, reload nginx:

sudo nginx -t
sudo nginx -s reload

That's it!

Wrapping up

As you can see setting up custom domains for your SaaS is ridiculously easy (and pretty cheap too) using Cloudflare. You don't need to worry about scale or anything really. Like I mentioned I might keep this solution instead of using cert-manager again because it's portable and I like it a lot. I didn't show here how to you use the API to create custom hostnames in Cloudflare, but it's very easy and there are libraries for many programming languages. Let me know in the comments what you think!

Update: see the next post on how I built a landing page and waitlist for DynaSite, to capture email addresses of people who might be interested in the product while I build it.

© Vito Botta