has_many :codes

Storing Mastodon assets in super cheap object storage iDrive e2

Published  

When I first set up my personal Mastodon instance (here's how I did it and here is a follow up on how to set up a crossposter between Mastodon and Twitter), I used a Wasabi bucket to store my instance's media and assets. Wasabi is an awesome service, pretty fast and reliable, but I came across a newer service called iDrive e2 which is even cheaper and since Mastodon can use a lot of storage, I decided to try it. There is still an introductory offer for 1 whole Terabyte for just $4 for the first year, which is awesome. After 1 year it's $40/year, which is still cheaper than Wasabi. I am not aware of a cheaper object storage service compatible with the S3 API. 

So I opened an account and I was impressed right away by the control panel but also the performance of the bucket operations. I haven't done any benchmarks or other measurements but iDrive e2 seems even faster than Wasabi. Also iDrive is a company with a long experience with storage services like backup, so it should be reliable as well.

At first, however, I couldn't get an iDrive e2 bucket to work with Mastodon out of the box because the files must be readable with public, non expiring URLs, and this was not possible with this new object storage service. So temporarily I used a simple Nginx container as a proxy in front of the bucket and that worked. 

After spending some more time on it and also with some help from iDrive's support (which gave me good first impressions btw), I managed to get iDrive e2 working properly with Mastodon without a proxy container and behind Cloudflare, so I don't have to worry about egress bandwidth (iDrive e2, like Wasabi, has some limitations in this regard).

Creating the bucket


First things first, you need to create an account with iDrive e2, and enable the region where you want to create your bucket for Mastodon. In my case, the instance is in Germany, so I created a bucket in Germany. I recommend that you name the bucket with the subdomain you will use with Cloudflare, if you want to use their free CDN like I do.  This saves bandwidth in e2 and improves performance because assets are served from locations that are closer to other instances and users, as opposed to serving all files from a single location.  In my case, I am serving media files from the files.botta.social hostname as configured in Cloudflare, therefore I named the bucket files. There is a reason for this concerning iDrive and TLS configuration, which we'll see later.   

Allowing public reads


I should mention that it is possible to solve the problem with public URLs mentioned earlier by contacting support. Upon request (for now, but they are going to make this possible in the control panel soon), they can make your bucket public, but don't do that. This makes the bucket public for both reads and writes, meaning that third parties would also be able to modify the content of your bucket, which is probably not what you want.

What you want to do instead, is to configure a bucket policy that only allows public reads, while requiring authentication and authorization for any other action. Documentation about iDrive e2 is still not great, but luckily I found that it supports S3 bucket policies, so that's what I recommend you do.

First, you need to install the AWS CLI on your computer and configure credentials (see docs in the link); then create a file somewhere (say /tmp/policy.json) with the following content:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowPublicRead",
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "*"
        ]
      },
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3:::bucket-name/*"
      ]
    }
  ]
}

Make sure you specify the correct name for your bucket. To apply the policy, run:

aws s3api --endpoint-url https://<your iDrive e2 endpoint> put-bucket-policy --policy file:///tmp/policy.json --bucket <name of your bucket>

You can find the iDrive endpoint for your bucket in the control panel, for the bucket's region.

You can then check that the policy has been applied with the command

aws s3api --endpoint-url https://<iDrive e2 endppoint> get-bucket-policy --bucket <name of your bucket>

Once this is done, try accessing a file you have uploaded to the bucket with the URL https://<iDrive endpoint>/path/to/file. It should work just fine.

CDN with Cloudflare

This is the part that requires a bit more steps. If you don't use Cloudflare yet, create an account and follow the simple instructions to add your domain to Cloudflare. This means that hereinafter you will be using Cloudflare as nameservers for your domain.

Once you have added the domain, go to the DNS tab in Cloudflare and add a wildcard CNAME record pointing to your iDrive e2 endpoint. Like I said earlier, for this to work correctly you should have given the bucket the same name that you want to use as subdomain. So in my case the bucket is named files because I use the subdomain files.botta.social to serve my Mastodon assets.

So now you should have a CNAME record for *.your-domain.com pointing to whatever endpoint iDrive has assigned to you for the region you have enabled.

Next, while still in Cloudflare, head to the SSL/TLS tab > "Origin Server" and click on "Create Certificate". You'll need to add both the root domain and the wildcard domain, so for me it was botta.social and *.botta.social. Click on "Create".

You will be presented with the "Origin Certificate" as well as a "Private Key". Click "OK", then enable the toggle under "Authenticated Origin Pulls". Take note of both.

Next, go your iDrive account, and click on the "Settings" button under the region of your bucket. This will open a panel on the right side. Click on "Add CNAME",  then fill in this way:

- CNAME: use the wildcard domain, in my case it was *.botta.social
- TLS certificate: enter the certificate and the private key you generated with Cloudflare.

Confirm. It will take a little while and several attempts depending on DNS propagation, but eventually it should add the CNAME correctly.

The last bit is setting the S3_ALIAS_HOST environment variable for Mastodon to the bucket-name.your-domain.com hostname, with your custom domain instead of iDrive's endpoint. Of course you need to configure the other settings for iDrive. Follow the previous post I linked to in the beginning for instructions on how to set up Mastodon with Docker.

That's it! Now you should be able to serve the Mastodon assets from the hostname bucket-name.your-domain.com.

Let me know in the comments if you run into issues and I will try to help.
© Vito Botta