has_many :codes
Apple • Docker • macOS

Using Docker on Apple silicon with a remote Docker engine

Published  

If you already own a Mac powered by the new M1 SoC designed by Apple ("Apple silicon"), or are interested in getting one, you probably know by now that Docker doesn't work on these machines yet, nor does virtualization in general (Docker is a Linux technology of course, so it requires virtualization to run on other operating systems).

I bought an M1 Mac Mini a week ago and couldn't be happier. But this means that I had to find a workaround for Docker since I depend on it quite a bit. Docker for Desktop cannot run on these Macs yet, but someone got Docker working in a virtual machine somehow.

I did try that solution, but the VM was somehow unstable, resulting in frequent kernel panics so I couldn't actually work with it. Luckily there is another option we can use as workaround for now, and that option is a remote Docker engine.

This is pretty easy to do and all you need is a Linux machine somewhere; I use a Hetzner Cloud VPS (referral link, we both receive credits) for this - it's very cheap and I can use a VPS in the Helsinki data center, so the latency is minimal for me) but you can use any Linux server. The only requirement is that it has Docker installed.

While the Docker engine doesn't work on M1, the Docker CLI does, you can install it e.g. with brew:

brew install docker

(You likely need to use the version of brew installed under Rosetta).

If you try any Docker commands after this you will only get errors, which is expected. So to use our remote Docker engine all we need to do is set the DOCKER_HOST environment variable to the location of that Docker instance:

export DOCKER_HOST=ssh://user@host

You can add this to your shell's config and you are good to go. Now Docker commands will work as usual. What will NOT work out of the box is bind mounts for volumes, which we usually use to have a "live" link from a container to a local folder for development. This is expected, because of course the Docker engine is on another computer and any path we specify as the origin path for the bind mount is interpreted by Docker as a path on the remote server. So any changes we make to local files won't be reflected in the remote container. 

I found a workaround for this, which involves a very little setup but works pretty well, offering an experience as much similar to the normal Docker experience as possible while we wait for native Docker support for M1 Macs. This workaround consists of a two way synchronization between the local directory and the remote container, in a very similar way to what recent builds of Docker (for Intel Macs) do behind the scenes.

File synchronization between a local directory and a remote container

To implement the active two way sync I am using syncthing, a free and open source alternative to services like Dropbox and Google Drive which does not require a central server and also offers a handy command line version which suits our needs perfectly.

We are going to use an SSH tunnel for the synchronization so to keep things secure.

syncthing on remote server

I will show here the commands you need to set up syncthing on a Ubuntu server, so these may differ if you use another Linux distro. To install the latest version, run these commands on the remote server:

sudo apt install curl apt-transport-https
curl -s https://syncthing.net/release-key.txt | sudo apt-key add -
echo "deb https://apt.syncthing.net/ syncthing release" | sudo tee /etc/apt/sources.list.d/syncthing.list
sudo apt-get update
sudo apt-get install syncthing

 Double check with:

syncthing -version

syncthing locally

You could install syncthing with brew on your Mac, but I found that version (running under Rosetta) quite unstable as it crashes randomly. So I am using an ARM build of syncthing which I found on Github in a thread about syncthing and Apple silicon. You can download this build for ARM from here.

I like to add binaries to /usr/local/bin like Homebrew does.

Configuration

We need some configuration for syncthing both locally and on the remote server. I keep these config files in a directory named syncthing in my project. You can generate these configuration files and the certificates (required for the encryption) by running the following command both locally and on the remote server:

syncthing -generate

Then move these files in the syncthing/local and syncthing/remote directories in your project, for the local syncthing and the remote server respectively. 

Next, find out the device id for both by running the following command on each:

syncthing -device-id

Take note of these IDs since you will need them next.

Edit syncthing/local/config.xml and change it as follows:

<configuration version="32">
    <folder id="<project name>" label="Default Folder" path="./" type="sendreceive" rescanIntervalS="2" fsWatcherEnabled="true" fsWatcherDelayS="10" ignorePerms="false" autoNormalize="true">
        <filesystemType>basic</filesystemType>
        <device id="<local device id>" introducedBy="">
            <encryptionPassword></encryptionPassword>
        </device>
        <device id="<remote device id>" introducedBy="">
            <encryptionPassword></encryptionPassword>
        </device>
        <minDiskFree unit="%">1</minDiskFree>
        <versioning>
            <cleanupIntervalS>3600</cleanupIntervalS>
        </versioning>
        <copiers>0</copiers>
        <pullerMaxPendingKiB>0</pullerMaxPendingKiB>
        <hashers>0</hashers>
        <order>random</order>
        <ignoreDelete>false</ignoreDelete>
        <scanProgressIntervalS>0</scanProgressIntervalS>
        <pullerPauseS>0</pullerPauseS>
        <maxConflicts>-1</maxConflicts>
        <disableSparseFiles>false</disableSparseFiles>
        <disableTempIndexes>false</disableTempIndexes>
        <paused>false</paused>
        <weakHashThresholdPct>25</weakHashThresholdPct>
        <markerName>.stfolder</markerName>
        <copyOwnershipFromParent>false</copyOwnershipFromParent>
        <modTimeWindowS>0</modTimeWindowS>
        <maxConcurrentWrites>2</maxConcurrentWrites>
        <disableFsync>false</disableFsync>
        <blockPullOrder>standard</blockPullOrder>
        <copyRangeMethod>standard</copyRangeMethod>
        <caseSensitiveFS>false</caseSensitiveFS>
        <junctionsAsDirs>true</junctionsAsDirs>
    </folder>
    <device id="<local device id>" name="local" compression="metadata" introducer="false" skipIntroductionRemovals="false" introducedBy="">
        <address>tcp://127.0.0.1:2000</address>
        <paused>false</paused>
        <autoAcceptFolders>false</autoAcceptFolders>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <maxRequestKiB>0</maxRequestKiB>
        <untrusted>false</untrusted>
    </device>
    <device id="<remote device id>" name="remote" compression="metadata" introducer="false" skipIntroductionRemovals="false" introducedBy="">
        <address>tcp://127.0.0.1:20001</address>
        <paused>false</paused>
        <autoAcceptFolders>false</autoAcceptFolders>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <maxRequestKiB>0</maxRequestKiB>
        <untrusted>false</untrusted>
    </device>
    <gui enabled="true" tls="false" debugging="false">
        <address>0.0.0.0:3500</address>
        <apikey>cnd</apikey>
        <theme>default</theme>
    </gui>
    <ldap></ldap>
    <options>
        <listenAddress>tcp://127.0.0.1:20000</listenAddress>
        <globalAnnounceServer>default</globalAnnounceServer>
        <globalAnnounceEnabled>false</globalAnnounceEnabled>
        <localAnnounceEnabled>false</localAnnounceEnabled>
        <localAnnouncePort>21027</localAnnouncePort>
        <localAnnounceMCAddr>[ff12::8384]:21027</localAnnounceMCAddr>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <reconnectionIntervalS>5</reconnectionIntervalS>
        <relaysEnabled>false</relaysEnabled>
        <relayReconnectIntervalM>10</relayReconnectIntervalM>
        <startBrowser>false</startBrowser>
        <natEnabled>false</natEnabled>
        <natLeaseMinutes>60</natLeaseMinutes>
        <natRenewalMinutes>30</natRenewalMinutes>
        <natTimeoutSeconds>10</natTimeoutSeconds>
        <urAccepted>1</urAccepted>
        <urSeen>3</urSeen>
        <urUniqueID><some unique id></urUniqueID>
        <urURL></urURL>
        <urPostInsecurely>false</urPostInsecurely>
        <urInitialDelayS>1800</urInitialDelayS>
        <restartOnWakeup>true</restartOnWakeup>
        <autoUpgradeIntervalH>12</autoUpgradeIntervalH>
        <upgradeToPreReleases>false</upgradeToPreReleases>
        <keepTemporariesH>24</keepTemporariesH>
        <cacheIgnoredFiles>false</cacheIgnoredFiles>
        <progressUpdateIntervalS>2</progressUpdateIntervalS>
        <limitBandwidthInLan>false</limitBandwidthInLan>
        <minHomeDiskFree unit="%">1</minHomeDiskFree>
        <releasesURL></releasesURL>
        <overwriteRemoteDeviceNamesOnConnect>false</overwriteRemoteDeviceNamesOnConnect>
        <tempIndexMinBlocks>10</tempIndexMinBlocks>
        <unackedNotificationID>authenticationUserAndPassword</unackedNotificationID>
        <trafficClass>0</trafficClass>
        <defaultFolderPath>~</defaultFolderPath>
        <setLowPriority>true</setLowPriority>
        <maxFolderConcurrency>0</maxFolderConcurrency>
        <crashReportingURL>https://crash.syncthing.net/newcrash</crashReportingURL>
        <crashReportingEnabled>false</crashReportingEnabled>
        <stunKeepaliveStartS>180</stunKeepaliveStartS>
        <stunKeepaliveMinS>20</stunKeepaliveMinS>
        <stunServer>default</stunServer>
        <databaseTuning>auto</databaseTuning>
        <maxConcurrentIncomingRequestKiB>0</maxConcurrentIncomingRequestKiB>
        <announceLANAddresses>true</announceLANAddresses>
        <sendFullIndexOnUpgrade>false</sendFullIndexOnUpgrade>
    </options>
</configuration>

Make sure you configure the parameters/attributes in <> correctly.

Similarly, edit syncthing/remote/config.xml and change it as follows:

<configuration version="32">
    <folder id="<project name>" label="Default Folder" path="<path on the remote server>" type="sendreceive" rescanIntervalS="2" fsWatcherEnabled="true" fsWatcherDelayS="10" ignorePerms="false" autoNormalize="true">
        <filesystemType>basic</filesystemType>
        <device id="<local device id>" introducedBy="">
            <encryptionPassword></encryptionPassword>
        </device>
        <device id="<remote device id>" introducedBy="">
            <encryptionPassword></encryptionPassword>
        </device>
        <minDiskFree unit="%">1</minDiskFree>
        <versioning>
            <cleanupIntervalS>3600</cleanupIntervalS>
        </versioning>
        <copiers>0</copiers>
        <pullerMaxPendingKiB>0</pullerMaxPendingKiB>
        <hashers>0</hashers>
        <order>random</order>
        <ignoreDelete>false</ignoreDelete>
        <scanProgressIntervalS>0</scanProgressIntervalS>
        <pullerPauseS>0</pullerPauseS>
        <maxConflicts>-1</maxConflicts>
        <disableSparseFiles>false</disableSparseFiles>
        <disableTempIndexes>false</disableTempIndexes>
        <paused>false</paused>
        <weakHashThresholdPct>25</weakHashThresholdPct>
        <markerName>.stfolder</markerName>
        <copyOwnershipFromParent>false</copyOwnershipFromParent>
        <modTimeWindowS>0</modTimeWindowS>
        <maxConcurrentWrites>2</maxConcurrentWrites>
        <disableFsync>false</disableFsync>
        <blockPullOrder>standard</blockPullOrder>
        <copyRangeMethod>standard</copyRangeMethod>
        <caseSensitiveFS>false</caseSensitiveFS>
        <junctionsAsDirs>true</junctionsAsDirs>
    </folder>
    <device id="<local device id>" name="local" compression="metadata" introducer="false" skipIntroductionRemovals="false" introducedBy="">
        <address>tcp://127.0.0.1:20001</address>
        <paused>false</paused>
        <autoAcceptFolders>false</autoAcceptFolders>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <maxRequestKiB>0</maxRequestKiB>
        <untrusted>false</untrusted>
    </device>
    <device id="<remote device id>" name="remote" compression="metadata" introducer="false" skipIntroductionRemovals="false" introducedBy="">
        <address>tcp://127.0.0.1:20000</address>
        <paused>false</paused>
        <autoAcceptFolders>false</autoAcceptFolders>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <maxRequestKiB>0</maxRequestKiB>
        <untrusted>false</untrusted>
    </device>
    <gui enabled="true" tls="false" debugging="false">
        <address>0.0.0.0:3501</address>
        <apikey>cnd</apikey>
        <theme>default</theme>
    </gui>
    <ldap></ldap>
    <options>
        <listenAddress>tcp://127.0.0.1:20000</listenAddress>
        <globalAnnounceServer>default</globalAnnounceServer>
        <globalAnnounceEnabled>false</globalAnnounceEnabled>
        <localAnnounceEnabled>false</localAnnounceEnabled>
        <localAnnouncePort>21027</localAnnouncePort>
        <localAnnounceMCAddr>[ff12::8384]:21027</localAnnounceMCAddr>
        <maxSendKbps>0</maxSendKbps>
        <maxRecvKbps>0</maxRecvKbps>
        <reconnectionIntervalS>5</reconnectionIntervalS>
        <relaysEnabled>false</relaysEnabled>
        <relayReconnectIntervalM>10</relayReconnectIntervalM>
        <startBrowser>false</startBrowser>
        <natEnabled>false</natEnabled>
        <natLeaseMinutes>60</natLeaseMinutes>
        <natRenewalMinutes>30</natRenewalMinutes>
        <natTimeoutSeconds>10</natTimeoutSeconds>
        <urAccepted>1</urAccepted>
        <urSeen>3</urSeen>
        <urUniqueID>b</urUniqueID>
        <urURL></urURL>
        <urPostInsecurely>false</urPostInsecurely>
        <urInitialDelayS>1800</urInitialDelayS>
        <restartOnWakeup>true</restartOnWakeup>
        <autoUpgradeIntervalH>12</autoUpgradeIntervalH>
        <upgradeToPreReleases>false</upgradeToPreReleases>
        <keepTemporariesH>24</keepTemporariesH>
        <cacheIgnoredFiles>false</cacheIgnoredFiles>
        <progressUpdateIntervalS>2</progressUpdateIntervalS>
        <limitBandwidthInLan>false</limitBandwidthInLan>
        <minHomeDiskFree unit="%">1</minHomeDiskFree>
        <releasesURL></releasesURL>
        <overwriteRemoteDeviceNamesOnConnect>false</overwriteRemoteDeviceNamesOnConnect>
        <tempIndexMinBlocks>10</tempIndexMinBlocks>
        <unackedNotificationID>authenticationUserAndPassword</unackedNotificationID>
        <trafficClass>0</trafficClass>
        <defaultFolderPath>~</defaultFolderPath>
        <setLowPriority>true</setLowPriority>
        <maxFolderConcurrency>0</maxFolderConcurrency>
        <crashReportingURL>https://crash.syncthing.net/newcrash</crashReportingURL>
        <crashReportingEnabled>false</crashReportingEnabled>
        <stunKeepaliveStartS>180</stunKeepaliveStartS>
        <stunKeepaliveMinS>20</stunKeepaliveMinS>
        <stunServer>default</stunServer>
        <databaseTuning>auto</databaseTuning>
        <maxConcurrentIncomingRequestKiB>0</maxConcurrentIncomingRequestKiB>
        <announceLANAddresses>true</announceLANAddresses>
        <sendFullIndexOnUpgrade>false</sendFullIndexOnUpgrade>
    </options>
</configuration>

In my case the directory I am synchronizing on the remote server is /home/deploy/app. 

Before we start the synchronization, I recommend you run the following command on the remote server otherwise you might see some errors with the file system watching:

echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

Next, on your Mac you need to run the following command to set up a couple of SSH tunnels between your local machine and the remote server:

ssh -f -N -i ~/.ssh/id_rsa -L 3501:127.0.0.1:3501 -L 127.0.0.1:20001:127.0.0.1:20000 -R 127.0.0.1:20001:127.0.0.1:20000 user@host

With the command above we create three tunnels:

  • the first one allows us to connect from our local machine to the GUI for syncthing running on the remote server at http://127.0.0.1:3501. This can be useful for debugging if the synchronization isn't working for some reason. You can open the GUI for the local syncthing at http://127.0.0.1:3500.
  • the second and third tunnels allow the local machine and the remote server to communicate with each other; on both, 127.0.0.1:20000 corresponds to its own running syncthing, while 127.0.0.1:20001 is the other device. You can see that we have used these in the config.xml files.

We need to do a first sync from your Mac and the remote server, and we can do this easily with rsync:

rsync -av --delete ./ user@host:/remote/path

Now that we have the tunnels in place and the content on the two devices is initially synced, we are ready to start the two way synchronization. On the remote server run:

syncthing -config=/home/deploy/app/syncthing/remote -data=/tmp -verbose -logfile /tmp/syncthing.log -log-max-old-files=0

Change the path of the config if needed. Then on your Mac run:

syncthing -config=syncthing/local -data=/tmp -verbose -logfile /tmp/syncthing.log -log-max-old-files=0

You will see from the logs on each that they connect each to the other device, and you can test the synchronization by creating and editing files on each side.

Conclusion

All in all I am very happy with the purchase of my M1 Mac Mini, and although I've had to find temporary workarounds for a few things, it's not been a particular problem so far. It's definitely been better and easier than I was expecting it to be in the beginning as far as software compatibility is concerned. As for Docker, the workaround described in this post is IMO easier than setting up a virtual machine on the Mac as described in the article I linked to, and it just works, without having to deal with kernel panics and such that make the other solution more difficult to actually do work with. Let me know in the comments if this works for you or if you have a better solution while we wait for a working Docker for Mac.

Previous: Kubernetes in Hetzner Cloud with Rancher Part 2 - Node Driver
Next: Self hosted Nextcloud in Kubernetes with S3 as primary storage
© Vito Botta