Setting up a Mastodon-Twitter crossposter

Published

In the previous post I described how I set up a personal Mastodon instance to be in control of my destiny on the "fediverse". In this follow up, I will share how to set up a couple of "cross posters" so that you can have your tweets posted to Mastodon and your toots posted to Twitter, automatically.
Why would you want to do that? If you, like me, are not 100% ready to let go of Twitter yet and are waiting to see how the situation evolves, you might want to use both at the same time for now. In that case, it's nice to be able to cross post automatically so when you for example post a toot to Mastodon, your Twitter followers can also see the same content. Same thing for Twitter to Mastodon crossposting.
Now there are two options for crossposters that I am aware of, and both are also available as "public" services:
- https://crossposter.masto.donte.com.br/
- https://moa.party/
Both are very easy to configure and use. All you need to do is link your Mastodon and Twitter accounts and configure some options (such as whether you want to crosspost retweets and things like that) and that's it. From that moment on your content will be crossposted automatically.
The problem with these public services is that they are very slow. There's a huge delay between when you post content on one platform and when the same content is crossposted to the other platform. The reason is that lots of people are using these apps, so they are likely being heavily rate-limited by Twitter (as we'll see, you need to set up a Twitter application in order to interact with the Twitter platform in an automated way).
For this reason, and since I am already self hosting Mastodon anyway, I decided to also self host a cross poster so that my content is crossposted right away without any delays. Both the options I mentioned above are available as open source and therefore we can self host any of them. In this post I will show you how to self host either of them so you can choose which one you prefer.
Why would you want to do that? If you, like me, are not 100% ready to let go of Twitter yet and are waiting to see how the situation evolves, you might want to use both at the same time for now. In that case, it's nice to be able to cross post automatically so when you for example post a toot to Mastodon, your Twitter followers can also see the same content. Same thing for Twitter to Mastodon crossposting.
Now there are two options for crossposters that I am aware of, and both are also available as "public" services:
- https://crossposter.masto.donte.com.br/
- https://moa.party/
Both are very easy to configure and use. All you need to do is link your Mastodon and Twitter accounts and configure some options (such as whether you want to crosspost retweets and things like that) and that's it. From that moment on your content will be crossposted automatically.
The problem with these public services is that they are very slow. There's a huge delay between when you post content on one platform and when the same content is crossposted to the other platform. The reason is that lots of people are using these apps, so they are likely being heavily rate-limited by Twitter (as we'll see, you need to set up a Twitter application in order to interact with the Twitter platform in an automated way).
For this reason, and since I am already self hosting Mastodon anyway, I decided to also self host a cross poster so that my content is crossposted right away without any delays. Both the options I mentioned above are available as open source and therefore we can self host any of them. In this post I will show you how to self host either of them so you can choose which one you prefer.
First option: @[email protected]'s cross poster
This crossposter, whose source code is available on Github, is a Ruby on Rails app just like Mastodon itself. Like Mastodon, it's pretty easy to self host using Docker. The instructions here assume you have set up Mastodon as described in the previous post, so if you haven't you will need to adapt these steps accordingly.
First things first, edit mastodon/docker-compose.yml and add the following:
First things first, edit mastodon/docker-compose.yml and add the following:
crossposter-db: restart: always image: postgres:14-alpine container_name: "crossposter-db" healthcheck: test: pg_isready -U postgres environment: POSTGRES_HOST_AUTH_METHOD: trust volumes: - /home/vito/apps/mastodon/crossposter/postgres:/var/lib/postgresql/data crossposter-redis: restart: always image: redis:6.0-alpine container_name: "crossposter-redis" healthcheck: test: redis-cli ping volumes: - /home/vito/apps/mastodon/crossposter/redis:/data crossposter-web: restart: always build: https://github.com/renatolond/mastodon-twitter-poster.git#main image: mastodon-twitter-poster container_name: "crossposter-web" env_file: - /home/vito/compose/mastodon/crossposter.env environment: ALLOWED_DOMAIN: "botta.social" DB_HOST: crossposter-db REDIS_URL: "redis://crossposter-redis" depends_on: - crossposter-db - crossposter-redis crossposter-sidekiq: restart: always build: https://github.com/renatolond/mastodon-twitter-poster.git#main image: mastodon-twitter-poster container_name: "crossposter-sidekiq" env_file: - /home/vito/compose/mastodon/crossposter.env environment: ALLOWED_DOMAIN: "botta.social" REDIS_URL: "redis://crossposter-redis" DB_HOST: crossposter-db command: bundle exec sidekiq -c 5 -q default -q high healthcheck: test: ps aux | grep '[s]idekiq\ 6' || false depends_on: - crossposter-db - crossposter-redis
You may also share the Postgres and Redis containers between Mastodon and the crossposter, but here for simplicity I am using separate containers. Like Mastodon, the cross poster requires both a Postgres database and Redis, and consists of two processes: one web process that allows us to configure the "bridge" between Mastodon and Twitter, and a background worker that actually handles the crossposting using Sidekiq jobs. Be sure to replace the username "vito" with yours in the YAML above. Also set the ALLOWED_DOMAIN env variable for the web container to the domain you use for your Mastodon instance, to prevent other people from using your cross poster (unless you are fine with that).
We also need to create the file /home/vito/compose/mastodon/crossposter.env with some environment variables:
LIBRATO_EMAIL= LIBRATO_TOKEN= TWITTER_CLIENT_ID=... TWITTER_CLIENT_SECRET=... SECRET_KEY_BASE=... RAILS_ENV=production RACK_ENV=production RAILS_LOG_TO_STDOUT=enabled # The name that's going to be displayed in the page, but also in the bottom of toots as the app name (optional) CROSSPOSTER_APP_NAME=Mastodon Twitter Crossposter # The domain where the app can be accessed CROSSPOSTER_DOMAIN=https://crossposter.domain.com # The repo, if it's a fork (optional) CROSSPOSTER_REPO=https://github.com/repo-to/crossposter # Optional: URL to a custom privacy policy # CROSSPOSTER_PRIVACY_URL=https://example.com/privacy # The stats page, if exists (optional) CROSSPOSTER_STATS=https://grafana.example.com # The fediverse account where announcements about this instance of the crossposter can be found CROSSPOSTER_FEDI_ACCOUNT_ADDRESS=https://instance.example.com/@crossposter [email protected] # Twitter account of the admin of this instance, if exists (optional) CROSSPOSTER_ADMIN_TWITTER=... # Fediverse account that users can use to reach the admin in case of issues CROSSPOSTER_ADMIN_FEDI_ADDRESS=https://your.mastodon/@adminuser [email protected] # If you are using statsd to save metrics, you need the three following variables STATSD_ENABLED=false STATSD_HOST=127.0.0.1 STATSD_PORT=8125 # You should only enable one of the following lines at a time ALLOWED_DOMAIN=your-mastodon-domain BLOCKED_DOMAINS=evil.corp,bad.instance # Database settings # Postgres DB_HOST=crossposter-db DB_PORT=5432 DB_NAME=crossposter DB_USER=postgres DB_PASS= # Redis only for Sidekiq REDIS_URL=redis://crossposter-redis:6379/0
You can use any random string for SECRET_KEY_BASE but for the Twitter Client ID and the Client Secret you need to create a Twitter application. So head to https://developer.twitter.com and sign up with your Twitter account, then create an application of type web; you will be asked a few questions such as whether your app needs to pull people's data from Twitter and things like that. Answer no to those since you don't need to do those things. All you need is to be able to post tweets on behalf of the user, and nothing else. The only critical configuration options you'll need to specify are the callback URL in the format "https://crossposter.domain.com/users/auth/twitter/callback" and the cross poster website URL. You will need to wait for Twitter to approve your app before you can do anything with it (perhaps it's better you don't mention Mastodon in the description of the app). Be sure to give the app both read and write permissions. Once the app is ready, you will need the consumer key (which you will set as TWITTER_CLIENT_ID in the env vars file above) and the consumer secret (TWITTER_CLIENT_SECRET).
Finally, to create the crossposter's containers run
compose/start.sh
Then run the following command to prepare the database:
docker compose run --rm crossposter-web bundle exec rake db:setup
Next, you'll need to configure the DNS for a subdomain like crossposter.yourdomain.com or something like that so that it points to the server. Then open the Nginx Proxy Manager control panel and add a new proxy host for that domain pointing to the crossposter-web container at port 3000. Be sure to request a new SSL certificate for it.
Once ready, visit the crossposter domain in the browser and proceed with linking your Mastodon/Twitter accounts, as well as configuring the available options to your linking. That's it! Now tweets should be crossposted to Mastodon and toots to Twitter, automatically.
Second option: moa.party
Unlike the first crossposter, moa.party is a Python app rather than Rails. I am not familiar with Python so I must admit it took me a while to get it working but this is actually what I am using at the moment since I like the configuration options more.
For starters, clone the source code from https://gitlab.com/fedstoa/moa, then create a Dockerfile in the root with the following content:
For starters, clone the source code from https://gitlab.com/fedstoa/moa, then create a Dockerfile in the root with the following content:
FROM python:3.7 RUN apt-get update -qq \ && DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends build-essential git-core \ && rm -rf /var/lib/apt/lists/* COPY requirements.txt requirements.txt RUN pip3 install -r requirements.txt COPY . . CMD ["python3", "app.py"]
Also create the file worker.sh which will run the worker. The worker doesn't seem to be a persistent process; instead, it checks whether there are bridges to process and when it's done crossposting it exits. For this reason we are using a simple loop to repeat this every 30 seconds:
#!/bin/bash while true; do python3 -m moa.worker sleep 30 done
Make the script executable:
chmod +x worker.sh
Next, build the image:
docker build --platform linux/amd64 -t moa .
If you are building directly on the server then you don't really need to push the image to a registry so you can name/tag it as you wish.
Next, create the compose/mastodon/moa/config.py file as follows:
from defaults import DefaultConfig class DevelopmentConfig(DefaultConfig): DEBUG = False DEVELOPMENT = True TESTING = False CSRF_ENABLED = True # secret key for flask sessions http://flask.pocoo.org/docs/1.0/quickstart/#sessions SECRET_KEY = '...' SQLALCHEMY_TRACK_MODIFICATIONS = False TWITTER_CONSUMER_KEY = '...' TWITTER_CONSUMER_SECRET = '...' INSTAGRAM_CLIENT_ID = '' INSTAGRAM_SECRET = '' # define in config.py # SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://moa:moa@localhost/moa' SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://root:mariadbpassword.@mariadb/moa?charset=utf8mb4' SEND = False # This option prevents Twitter replies and mentions from occuring when a toot contains @[email protected]. This # behavior is against Twitter's rules. SANITIZE_TWITTER_HANDLES = True SEND_DEFERRED_EMAIL = False SEND_DEFER_FAILED_EMAIL = False MAINTENANCE_MODE = False STATS_POSTER_BASE_URL = None STATS_POSTER_ACCESS_TOKEN = None TRUST_PROXY_HEADERS = False WORKER_JOBS = 1 SERVER_NAME = 'crossposter.domain.com' PREFERRED_URL_SCHEME = 'https'
You already know what the Twitter consumer key and secret are. Choose a random string for the SECRET_KEY, AND IN SQLALCHEMY_DATABASE_URI write the correct password for MariaDB as per the previous post.
Open a MySQL shell:
mysql -uroot -p<mariadb password> -h 127.0.0.1
And create the database for the crossposter:
CREATE DATABASE moa;
Next, add the following to the Mastodon docker-compose.yml file:
moa-web: restart: always image: moa container_name: "moa-web" environment: MOA_CONFIG: config.DevelopmentConfig FLASK_DEBUG: "0" volumes: - /home/vito/apps/mastodon/moa:/data - /home/vito/compose/mastodon/moa/config.py:/config.py moa-worker: restart: always image: moa container_name: "moa-worker" environment: MOA_CONFIG: DevelopmentConfig command: "/worker.sh" volumes: - /home/vito/apps/mastodon/moa:/data - /home/vito/compose/mastodon/moa/config.py:/config.py
As usual replace the username etc.
Start the containers:
compose/start.sh
And run the following command to prepare the database:
docker run --rm -it -e MOA_CONFIG=DevelopmentConfig -v /home/vito/compose/mastodon/moa/config.py:/config.py --network apps vitobotta/moa-crossposter:v8 python3 -m moa.models
Then create a proxy host in Nginx Proxy Manager using the moa-web container and the port 5000 this time. Also set the callback URL in the Twitter app to https://crossposter.domain.com/twitter_oauthorized.
Open the crossposter domain in the browser, connect the Mastodon and Twitter accounts, and configure the crossposting. Done.
Wrapping up
If you have set up Mastodon as per the previous post, the instructions above should be straightforward, but you don't need to change them much otherwise. Let me know in the comments if you run into any issues and I will try to help.