Rails: Signing out from devices

In an app I’m working on, I wanted users to be able to sign out from any device they are signed in on, by invalidating logins. There’s a gem called authie that does this so you may want to check it out; here I’ll show a very simple implementation I went with which works well enough for me. The goal is to:

  • create a login whenever a user signs in, with IP address, user agent and a unique device ID;
  • at each request, check whether a login exists for the given user/device ID combination and if it doesn’t, force sign in;
  • update the login at each authenticated request just in case the IP address (thus the location) changes while a session is active (optional);
  • delete the login when the user signs out from the device;
  • list all the active logins in the user’s account page with browser/OS info, IP address, and approximate location (city & country);
  • allow the user to delete any of those logins to sign out from the respective device.

I like doing authentication from scratch (see this Railscast) so that’s what I am using here but if you use something like Devise instead, it won’t be very different.

The first thing we need for this simple implementation is to generate a Login model:

rails g model Login user:belongs_to ip_address user_agent device_id:index

The Login model will be basically empty as it will only do persistence:

class Login < ApplicationRecord
  belongs_to :user
end

Then in the create action of my SessionsController I have something like this:

  def create
    @sign_in_form = SignInForm.new

    if user = @sign_in_form.submit(params[:sign_in_form])
      device_id = SecureRandom.uuid

      if params[:sign_in_form][:remember_me]
        cookies.permanent[:auth_token] = user.auth_token
        cookies.permanent[:device_id]  = device_id
      else
        cookies[:auth_token] = user.auth_token
        cookies[:device_id]  = device_id
      end

      user.logins.create!(ip_address: request.remote_ip,
                          user_agent: request.user_agent,
                          device_id: device_id)

      redirect_to ...
    else
      redirect_to sign_in_path, alert: "Invalid email or password."
    end
  end

So each time a user successfully signs in from a device we create a login with a unique device ID.

In the ApplicationController, I have:

  def current_user
    @current_user ||= begin
      if cookies[:auth_token].present? and cookies[:device_id].present?
        if user = User.find_by(auth_token: cookies[:auth_token])
          if login = user.logins.find_by(device_id: cookies[:device_id])
            # optional
            login.update!(ip_address: request.remote_ip, user_agent: request.user_agent, updated_at: Time.now.utc)
            user
          end
        end
      end
    end
  end
  helper_method :current_user

  def authenticate
    redirect_to sign_in_path unless current_user
  end

I didn’t bother here but perhaps you can prettify the current_user method. So, in order to assume the user is successfully authenticated for the request, we expect:

  • both the auth_token and device_id cookies to be present;
  • the auth_token to be associated with an existing user;
  • a login to exist for the user with the device_id stored in the cookies;

otherwise we redirect the user to the sign in page.

Finally, in the SessionsController I have a destroy action which deletes both the login and the cookies from the browser:

  def destroy
    current_user.logins.find_by(device_id: cookies[:device_id]).destroy
    cookies.delete(:auth_token)
    cookies.delete(:device_id)
    flash.now[:notice] = "Successfully signed out."
    redirect_to sign_in_path
  end

Remember to add a route for the destroy action, e.g.:

resources :logins, only: [:destroy]

Next, we want to list the active logins for the user in their account page so that they can sign out from any of those devices. So that the user can easily tell logins apart I am using:

  • the device_detector gem to identify browser and operating system;
  • the Maxmind GeoIP2 API with the geoip2 gem to geolocate IP addresses so we can display the approximate location for each login. This is just one of many ways you can geolocate IP addresses; I am using Maxmind for other things too so using the Maxmind API works fine for me but you may want to use a different service or a local database (for performance). Also see the geocoder gem for another option.

In the LoginsHelper I have:

module LoginsHelper
    def device_description(user_agent)
        device = DeviceDetector.new(user_agent)
        "#{ device.name } #{ device.full_version } on #{ device.os_name } #{ device.os_full_version }"
    end

    def device_location(ip_address)
        if ip = Ip.find_by(address: ip_address)
            "#{ ip.city }, #{ ip.country }"
        else
            location = Geoip2.city(ip_address)
            if location.error
                Ip.create!(address: ip_address, city: "Unknown", country: "Unknown")
                "Unknown"
            else
                Ip.create!(address: ip_address, city: location.city.names[:en],
                                     country: location.country.names[:en])
                "#{ location.city.names[:en] }, #{ location.country.names[:en] }"
            end
        end
    end
end

I am leaving these methods in the helper but you may want to move them into a class or something. device_description, as you can see, shows the browser/OS info, for example for my Chrome on Gentoo it shows Chrome 52.0.2743.116 on GNU/Linux; then device_location shows city and country like Espoo, Finland if the IP address is in the Maxmind database. If the IP address is invalid or it is something like 127.0.0.1 or a private IP address, the Maxmind API will return an error so we’ll just show “Unknown” instead. This is an example, you may want to avoid the API call (if using an API) when the IP is a private IP address; another optimisation could be performing the geolocation asynchronously with a background job when the user signs in, instead of performing it while rendering the view. Also, you can see another model here, Ip. This is a simple way to cache IP addresses with their locations so we don’t have to make the same API request twice for a given IP address. So next we need to generate this model:

rails g model Ip address:index country city

Again, I am showing here an example, you may want to move the geolocation logic to the Ip model or to a separate class, up to you.

We can now add something like the following to the user’s account page:

<h2>Active sessions</h2>
These are the devices currently signed in to your account:
<table id="logins">
<thead>
<tr>
<th>Device</th>
<th>IP Address</th>
<th>Approximate location</th>
<th>Most recent activity</th>
<th></th>
</tr>
</thead>
<tbody>
    <%= render @logins  %></tbody>
</table>

where @logins is assigned in the controller:

@logins = current_user.logins.order(updated_at: :desc)

The _login.html.erb partial contains:

<tr id="<%= dom_id(login) %>" class="login">
<td><%= device_description(login.user_agent) %></td>
<td><%= login.ip_address %></td>
<td><%= device_location(login.ip_address) %></td>
<td><%= time_ago_in_words(login.updated_at) %></td>
<td>
        <% if login.device_id == cookies[:device_id] %>
            (Current session)
        <% else %>
            <%= link_to "<i class='fa fa-remove'></i>".html_safe, login_path(login), method: :delete, remote: true, title: "Sign out", data: { confirm: "Are you sure you want to sign out from this device?" } %>
        <% end %></td>
</li>

Besides browser/OS/IP/location we also show an X button to sign out from devices unless it’s the current session. It looks like this:

screenshot-from-2016-10-19-18-31-50

Finally, a little CoffeeScript view to actually delete the login when clicking on the X:

$("#login_<%= @login.id %>").hide ->
    $(@).remove()

and the destroy action:

class LoginsController < ApplicationController
    def destroy
        current_user.logins.find(params[:id]).destroy
    end
end

That’s it! Now if the user removes any of the logins from the list, the respective device will be signed out.

Cron-like timers with systemd

Yesterday I configured backups with duplicity on a couple of servers using a 3rd party service, rsync.net, as the backup destination; since I am a little paranoid with backups I also wanted to schedule a daily task on my computer at home to mirror the backups from that service to a local directory, just in case. I could have used cron for this but since I use Gentoo with systemd now I wanted to try systemd timers.

Cron does seem to be a lot easier to use, but there are some advantages to using systemd timers. For example – from what I have read/understood so far:

  • all the events are logged in the systemd journal, so you can easily check for example when a timer last ran and if the task was successful – this is very helpful when debugging;

  • systemd timers are basically services, and as such they are more flexible than cron jobs; among other things you can specify IO scheduling priority, niceness, timeouts, etc. (see this);

  • a timer can be triggered in various ways, even -for example- by hardware state changes;

  • a timer can be configured to depend on another service, for example to mount some remote filesystem before executing the scheduled task.

Configuring a systemd timer

So here’s how to configure a simple timer with systemd. In this example I want to mirror a remote directory to a local directory daily at 4am. For starters, you need to create a .timer file under /etc/systemd/system which looks like this:

[Unit]
Description=Mirror rsync.net backups daily
RefuseManualStart=no
RefuseManualStop=no

[Timer]
Persistent=true
OnCalendar=*-*-* 04:00:00
Unit=rsyncnet.service

[Install]
WantedBy=basic.target

You also need to create a second file with same name but with .service extension in the same location:

[Unit]
Description=Mirror rsync.net backups daily
RefuseManualStart=no
RefuseManualStop=yes

[Service]
User=vito
Type=simple
ExecStart=/usr/bin/rsync -azP --delete ...

To have systemd pick up these files you need to run:

sudo systemctl daemon-reload

Then, to enable the scheduled task now and at startup:

sudo systemctl start rsyncnet.timer
sudo systemctl enable rsyncnet.timer

To list the timers:

sudo systemctl list-timers --all

To trigger the task manually:

sudo systemctl start rsyncnet

To check the log for the task status:

journalctl -f -u rsyncnet.timer

Or you can check the status of both the timer and the service directly:

systemctl status rsyncnet.timer
systemctl status rsyncnet.service

These are just the basics for a daily task which runs at a given time, but systemd timers are really flexible and powerful, I’d suggest you check the man pages for more info.

Gentoo: using Postfix with an external SMTP service

Sometimes I want to have some email notifications sent to my email address by my computer at home, for example to receive reports of some scheduled tasks.

The problem is that if you just install Postfix or other MTA with a default configuration, emails sent from your home computer may be flagged as spam or mailing may not work altogether due to some restrictions ISPs often have also to prevent spam.

One workaround is to configure e.g. Postfix to use an external SMTP service such as SendGrid to send the emails. Here I’ll show how to do this on Gentoo.

First thing you need to do is install Postfix. Edit /etc/portage/package.use and add:

>=mail-mta/postfix-3.1.0-r1 sasl

(of course you may have to specify a different version) Then run:

sudo emerge -av mail-mta/postfix

I also suggest you install mailutils as this includes an utility you can use to test email sending:

sudo emerge -av net-mail/mailutils

Next, you need to edit /etc/postfix/sasl_passwd and add the following line which contains the address and port of the SMTP service and the credentials required for the authentication:

[smtp.sendgrid.net]:587 username:password

You need then to create a db from this file with the following command:

sudo postmap /etc/postfix/sasl_passwd
sudo chown root:root /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db
sudo chmod 0600 /etc/postfix/sasl_passwd /etc/postfix/sasl_passwd.db

Also run:

sudo newaliases
sudo postmap /etc/mail/aliases

Now edit /etc/postfix/main.cf and add the following:

relayhost = [smtp.sendgrid.net]:587
smtp_sasl_auth_enable = yes
smtp_sasl_security_options = noanonymous
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_use_tls = yes
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
myhostname = <hostname>
mydomain = <hostname>

Please note that you need to set a FQDN hostname on your computer that is already validated with the SMTP service.

Finally, restart Postfix:

sudo systemctl restart postfix.service

You can test that mailing works with the mail utility:

echo blah | mail -s "test" <your email address>

To check the logs you can run:

journalctl -f -u postfix

That’s it. All emails sent from your computer will be now sent through the 3rd party SMTP service.

Migrating a Google Analytics property to another account

I’ve had two Google Analytics accounts for a few years now, the first one with just one property – for this blog – and the other one for the other sites/apps I manage. Today I wanted to migrate the blog property to the second account so to keep everything under the same account, so I was happy to see that this is possible now – not sure when they’ve changed things but I had tried this some time ago without success.

I did find some help page by Google about this, but it was confusing as to which exact permissions I had to enable and where, so here’s what I have done in case someone else finds this confusing too.

So, assuming you own a Google Analytics account A and another account B, and want to migrate/move a property from A to B, the first thing you need to do is open the property settings under Admin in account A:

screenshot-from-2016-10-15-13-43-17

Then you have to add the user account B under User Management and enable all the permissions for it. Here’s the confusing part: there is a User Management section for both the property and the account. From reading the Google help pages it wasn’t clear which one was it; it turns out, you want to head to the account‘s user management:

screenshot-from-2016-10-15-13-46-41

Here you need to add the email address for account B and enable all the permissions:

screenshot-from-2016-10-15-13-54-45

Once you’ve done this, head back to Admin > Property Settings and click the Move property button.

screenshot-from-2016-10-15-13-58-10

Select account B from the drop down and confirm the changes. That’s it. Give it (usually) a few minutes and the property will be moved to account B.

Encrypted Gentoo Installation on MacBook Pro

It looks like it’s been a while again since I last posted something… but here I am. About three months ago I was planning to replace my late 2013 iMac 27″ with a Mac Pro; overall I liked the iMac a lot but from time to time I do some video editing/encoding and the iMac got very hot and noisy each time. So I was originally thinking to switch to a Mac Pro mainly for this reason. However there was no sight of new Mac Pros and the ones currently available are still ridiculously expensive considering that we are talking about hardware released in 2013; with much less money you can easily build yourself a much more powerful machine, and so I did. I sold the iMac and with half the amount I’d have spent for a Mac Pro I bought all the parts (plus two 27″ monitors, new keyboard/mouse and webcam!) and built a very powerful machine with recent hardware. It’s very fast and very quiet even overclocked.

I initially thought I’d use the new PC as a Hackintosh and install macOS on it as the primary OS, but having used a Hackintosh in recent past I didn’t want again the hassle of getting the computer to work with macOS knowing that each time there is a big update there is also the risk that the OS could stop working altogether.

So the primary candidate was Ubuntu since I have been using it on servers for many years, but I then decided to install Gentoo Linux instead. IMO the installation isn’t as complicated and difficult as many think it is, so I eventually installed Gentoo on my two MacBook Pros as well as the desktop. I must say that so far I am loving it and I don’t miss OSX/macOS at all since I found and got used to the alternative apps for Linux.

Why Gentoo?

Some of the reasons why I wanted to give Gentoo a try as my primary OS are:

  • you can install binary packages but most software is compiled and thus it is optimised for your hardware, which means it does take longer when you install stuff but you usually get a faster system in return (“Gentoo” is the name of the fastest penguins on earth);
  • you really install only what you want/need. It’s not like most other distros which install a lot of stuff and features that you may never use. Instead with Gentoo you only install what you actually need and just the dependencies required; for example if you use Gnome like me, you can configure the system so that it doesn’t install all the packages required for KDE and so on. With USE flags you can even customise features on a per package basis if you wish;
  • Gentoo differs from other distros also in that it uses a rolling release system, so you can just install the system once and keep it frequently updated with the latest versions of everything, rather than having to perform a bigger upgrade in one go each time a new release is out; you must update your system frequently though for this to work well;
  • documentation is perhaps the best one I’ve seen so far for Linux distributions.

Installing Gentoo on a MacBook Pro with full disk encryption

There are several guides on the Internet (especially the official Gentoo Handbook) which show how to do a typical Gentoo installation, but I thought I’d add here my own notes on how to do this specifically on a MacBook Pro with full disk encryption and LVM, so it can hopefully save some time vs reading several guides to achieve the same. I want to keep this as short as possible so I won’t go into the details for every command, which you can easily find yourself. Here I will just describe the steps necessary to get a system up and running quickly, and will update the post each time I install Gentoo, if needed.

First, a few notes:

  • the two MacBook Pros on which I have installed Gentoo are a mid-2010 and an early-2011, so they are not very recent; you might find you have to tweak the installation process a little if you own a more recent MBP but most of the process will be the same;
  • while learning the installing process I had at times to force eject the installation CD/DVD during boot. I found that you can do this by holding the touch-pad’s left button while the MBP is booting;
  • once you install the system, you may find that your MBP takes around 30 seconds before actually booting and it will seem as if it freezes on the white screen after the startup chime sound; to fix this you will need to boot the system from an OSX/macOS installation media or use the Internet recovery, and lunch the following command from a terminal:
bless --device /dev/disk0s1 --setBoot --legacy

You need to replace /dev/disk0s1 with the correct name for your disk device which you can find with the diskutil list command;

  • during the installation the network interface may not work automatically until you get everything sorted; you can use the
ip link show

command to find the correct name for your network interface, which as we’ll see later you will need to manually activate.

  • you can use either the Gentoo CD or the DVD to install the system. The difference is that the CD only boots in BIOS mode while the DVD can also boot in EFI mode. So if you want to do an installation in EFI mode you will have to use the DVD. In my case, I have chosen to install Gentoo in BIOS mode on both my MBPs, because when the system boots in BIOS mode the integrated Intel graphics card is automatically disabled, forcing you to use the discrete ATI or nVidia card instead; if you want to avoid possible issues which may arise when having both the integrated card and the discrete card enabled, I recommend you also install the system in BIOS mode; it’s just easier. This is what I will show here.

The installation media

So, to get started with the installation first burn the Gentoo CD/DVD image which you can download here, then insert the CD/DVD in the optical drive and turn the MBP on while holding the Alt key, so you can chose to boot the system from the installation media. If you are using the DVD version you will be able to choose whether to boot the system in “Windows” mode or EFI mode. Choose “Windows” mode. You will then see the bootloader screen with some options; press “e” to temporarily edit the boot configuration and add the nomodeset argument to the line which starts with linux. This will avoid some issues with the graphics card during boot. Continue with the boot process making sure you boot into a terminal if you are using the DVD installation disk, otherwise it will load the “Live” version of Gentoo.

Disk and partitions

Next, assuming that you are going to install Gentoo as the only OS or anyway as the first OS (I won’t show here how to install multiple operating systems), you will want to wipe the disk and create the necessary partitions – if you want you can create separate partitions for /home etc but here I will assume you want a single main partition for simplicity. Run

fdisk /dev/sda

Press “p” to see the current partition scheme of the disk; to delete the first partition press “d” followed by the number of the partition you want to delete (starting from 1); repeat this until all the partitions have been removed from the configuration of the disk. Then you need to create the new partitions.

First, create the BIOS partition by pressing “n”, then “p” (to specify that you want to create a primary partition), and then “1” as the partition number; fdisk will now ask for both the first sector and the last sector for this partition; enter “2048” first and then “+2M” so that the size of the partition is 2MB. Next, create the boot partition by pressing “n”, then “p”, “2” (second partition); accept the default value for the first sector and enter “+128M” for the last sector so to have a 128M boot partition. Now press “a” and then “2” to make this partition bootable.

The last partition you need to create is /dev/sda3 which will later be encrypted and contain both the root partition for the OS and the data, and the swap partition. Press “n” again, followed by “p”, then “3”; accept the default values for both the first sector and the last sector so that this partition will take the remaining space on the disk.

If everything is OK you will see something like the following by pressing “p”:

Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00000000

Device Boot Start End Sectors Size Id Type
/dev/sda1 2048 6143 4096 2M 4 FAT16 <32M
/dev/sda2 * 6144 268287 262144 128M 83 Linux
/dev/sda3 268288 468860079 468591792 223.5G 83 Linux

The changes you have made haven’t been written to disk yet, so to confirm these changes and actually wipe the disk and create partitions press “w” then exit fdisk.

Now run

mkfs.vfat -F 32 /dev/sda2

to format the boot partition. Next it’s time to set up the encrypted partition. To activate the kernel modules required for the encryption run

modprobe dm-crypt
modprobe aes (if it returns an error it means that no hardware cryptographic device is present; in this case run "modprobe aes_generic" instead)
modprobe sha256

Next, to set up encryption and LVM run

cryptsetup luksFormat /dev/sda3 (type uppercase YES and enter a passphrase which you will use to unlock the encrypted disk)
cryptsetup luksOpen /dev/sda3 main
pvcreate /dev/mapper/main
vgcreate vg /dev/mapper/main
lvcreate -L 1GB -n swap vg
lvcreate -l 100%FREE -n root vg

Please note that I am using cryptsetup here with the default settings, but you can tweak the luksFormat command if you want to achieve higher security. Please refer to the man pages for more details. Next run vgdisplay to verify that all the space has been allocated to the encrypted partitions, then run:

mkswap /dev/vg/swap
swapon /dev/vg/swap
mkfs.ext4 /dev/vg/root
mount /dev/vg/root /mnt/gentoo
mkdir /mnt/gentoo/boot
mount /dev/sda2 /mnt/gentoo/boot
cd /mnt/gentoo

These commands will prepare and activate the swap partition, format the root partition as ext4 and mount both the boot and root partitions.

Installing the base system

Now you are ready to download the archive which contains the base system and install it. Run

links https://www.gentoo.org/downloads/mirrors/

which will launch a text based browser. Choose a mirror close to your location and download a stage3 archive from releases/amd64/autobuilds. Then run

tar xvjpf stage3-*.tar.bz2 --xattrs

to extract all the files for the base system on the root partition. Next run

nano -w /mnt/gentoo/etc/portage/make.conf

and change the follow settings:

CFLAGS="-march=native -O2 -pipe"
MAKEOPTS="-j5"
USE="mmx sse sse2 -kde gtk gnome dvd alsa cdr emu efi-32 efi-64i -bindist xvmc"
INPUT_DEVICES="evdev synaptics mtrack tslib"
VIDEO_CARDS="nouveau" (for nVidia graphics cards, or "radeon" for ATI cards)

Set MAKEOPTS to the number of cores + 1. Please note that I am assuming here you want to use Gnome, that’s why I have gnome but -kde in the USE setting. If you want to use something else you will have to change the USE setting. Now run

mirrorselect -i -o >> /mnt/gentoo/etc/portage/make.conf

and choose a mirror which will be used to download software from the repos hereinafter. Next,

mkdir /mnt/gentoo/etc/portage/repos.conf
cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf

and then run

cp -L /etc/resolv.conf /mnt/gentoo/etc/

to configure DNS resolution for the installation process. Now run

mount -t proc proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev
mount --make-rslave /mnt/gentoo/sys
mount --make-rslave /mnt/gentoo/dev

after which you are ready to chroot into the new system:

chroot /mnt/gentoo /bin/bash
source /etc/profile
export PS1="chroot $PS1"

It’s time to configure which system “profile” you want to use to configure and install the software. Run

emerge-webrsync
eselect news read
eselect profile list
eselect profile set X (where X is the profile you want to use, I use gnome/systemd)

Now install all the packages required to reflect the system profile you have chosen – as said I will assume you also have chosen gnome/systemd.

emerge --ask --update --deep --newuse @world

This will take some time, so go and enjoy a coffee. Once it’s done, choose your timezone, e.g.:

echo "Europe/Helsinki" > /etc/timezone
emerge --config sys-libs/timezone-data

and configure the locale:

nano -w /etc/locale.gen
locale-gen
eselect locale list
eselect locale set X (choose one)

So that these changes take effect, run

env-update && source /etc/profile && export PS1="(chroot) $PS1"

Configuring and compiling the Kernel

Now download the kernel sources with

emerge --ask sys-kernel/gentoo-sources

To ensure that the kernel will support encryption, run

echo "sys-kernel/genkernel-next cryptsetup" >> /etc/portage/package.use/genkernel-next

Then install genkernel which is a tool you can use to configure and compile the kernel.

emerge --ask sys-kernel/genkernel-next

You need now to edit /etc/fstab to ensure the boot partition is mounted at boot:

nano -w /etc/fstab

and add:

/dev/sda2 /boot ext2 defaults 0 0

Next install LVM:

emerge -av sys-fs/cryptsetup sys-fs/lvm2

Then edit /etc/genkernel.conf and make the following changes:

MRPROPER="no"
MAKEOPTS="-j5"
LVM="yes"
LUKS="yes"
REAL_ROOT="/dev/vg/root"
INITRAMFS_OVERLAY="/boot/overlay"
BUSY_BOX="yes"
MENU_CONFIG="yes"

Here also set MAKEOPTS to the number of cores + 1.

To compile the kernel, run:

genkernel --no-zfs --no-btrfs --install all

Now you can customise the kernel if you wish, or leave the defaults as they are – up to you. As you can see I am passing the –no-zfs –no-btrfs arguments since I don’t use these file system, so the compilation takes a little less time.

Once the kernel has been compiled, edit /etc/fstab once again and add

/dev/sda2 /boot vfat defaults 0 2
/dev/vg/root / ext4 noatime 0 0
/dev/vg/swap none swap sw 0 0
/dev/cdrom /mnt/cdrom auto noauto,user 0 0

Networking

Check which name your network interface has with

ip link show

then edit /etc/conf.d/net and change it so it looks as follows:

config_enp2s0f0="dhcp"

Of course change enp2s0f0 with the name of your network interface. Next, run

cd /etc/init.d
ln -s net.lo net.enp2s0f0 (again, use the name of your network interface here)
rc-update add net.enp2s0f0 default

Miscellaneous

At this stage you may want to set your root password with

passwd

Also install sysklogd with

emerge --ask app-admin/sysklogd

Bootloader

To install the bootloader, run

emerge --ask sys-boot/grub:2

Then edit /etc/default/grub and change the GRUB_CMDLINE_LINUX setting as follows:

GRUB_CMDLINE_LINUX="init=/usr/lib/systemd/systemd crypt_root=/dev/sda3 root=/dev/mapper/vg-root dolvm rootfstype=ext4 nomodeset"

This makes sure the correct settings are used each time you update the bootloader. In this example we specify that systemd, encryption and lvm must be used during boot otherwise it will not be possible to access the encrypted partitions. We also add nomodeset to avoid problems with the graphics card as explained earlier. Next,

grub-install /dev/sda
grub-mkconfig -o /boot/grub/grub.cfg

You should now be able to boot into the new system:

exit
cd
umount -l /mnt/gentoo/dev{/shm,/pts.}
umount /mnt/gentoo{/boot,/sys,/proc}
shutdown -r now

Hopefully the system will start from the disk. If all is OK, run

systemd-machine-id-setup
hostnamectl set-hostname vito-laptop (choose whichever hostname you wish here)

Next edit /etc/systemd/network/50-dhcp.network and change the contents as follow:

[Match]
Name=en*

[Network]
DHCP=yes

To activate networking now and ensure it is activated at startup, run

systemctl enable systemd-networkd.service
systemctl start systemd-networkd.service

At this stage I’d add the main user account with

useradd -m -G users,wheel,audio,video -s /bin/bash vito
passwd vito

Of course use your chosen account name instead of “vito”.

Graphics card and environment

To install the drivers for your graphics card and X, run

emerge --ask --verbose x11-base/xorg-drivers
emerge --ask x11-base/xorg-server
env-update
source /etc/profile

Next, to install Gnome edit /etc/portage/package.use/gnome-session and add

gnome-base/gnome-session branding

Then run

emerge --ask gnome-base/gnome
eselect news read
gpasswd -a vito plugdev (your account name instead of 'vito')

Edit /etc/conf.d/xdm and set GDM as the window manager, then run

echo "exec gnome-session" > ~/.xinitrc
systemctl enable gdm.service
systemctl start gdm.service
shutdown -r now

If all went well, the system will now boot into Gnome.

Touch pad

If the touch pad isn’t working you will need to recompile the kernel. Run

genkernel --no-zfs --no-btrfs --install all

and enable the following settings before saving and exiting – which will trigger recompilation:

EHCI HCD (USB 2.0) support
Root Hub Transaction Translators
Improved Transaction Translator scheduling
Generic EHCI driver for a platform device
Device Drivers --->
Input device support --->
Mice --->
Apple USB BCM5974 Multitouch trackpad support

Keeping the system up to date

As I mentioned earlier, it is recommended you update the system frequently to avoid problems with big updates. To update the system, I usually run the following commands weekly:

emerge --sync
emerge -avuDU --with-bdeps=y @world
emaint --check world
emerge -av --depclean
emerge --update --newuse --deep @world
revdep-rebuild
perl-cleaner --all

Conclusions

I actually had some more notes about using proprietary drivers for the graphics card (instead of the open source nouveau or radeon drivers) and a few more things, but I can’t find them at the moment. I will update the post if I find them or if I go through the installation process again. Anyway the steps described in the post will get you up and running with an encrypted installation with gnome/systemd.

Let me know in the comments if this post has been somehow useful.

How to use Let’s Encrypt certificates with Nginx

Back in early 2011, I wrote a post on the most common reasonswhy SSL isn’t turned on by default for all websites, and one of these reasons at the time was cost.

Standard SSL certificates can be quite cheap these days, yet nothing beats free. According to their website, Let’s encrypt – which entered public beta on December 3 – is

a new Certificate Authority: It’s free, automated, and open.

So this essentially means you can get valid, trusted TLS/SSL certificates for free. Besides the cost, one thing I really like of Let’s Encrypt is that it is so easy and quick to get a new certificate! Normally you’d have to generate a Certificate Signing Request (CSR) and a private key on the server, then send the CSR to a provider/Certificate Authority in order to get the actual certificate. In many cases, the certificate you receive from the provider is a bundle of several certificates that you have to combine into a single certificate you can then install on the server. You need to repeat the process each time you need to renew the certificate.

The process overall isn’t complicated but is made much easier and quicker with Let’s Encrypt. If you use Apache, everything is pretty much automated with the Let’s Encrypt python tools, in that the certificate will be generated and installed in Apache automatically for you. The same level of support for Nginx is still in the works, but generating a certificate you can install with Nginx as well is quite straightforward.

First, you need to clone the git repo which contains the python tools you will use to generate new certificates:

git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt

Next, you need to stop Nginx before proceeding… I know this sounds like it may be a problem, but there is a reason for this will I will explain in a moment.

service nginx stop

Now you can run the python tool which will generate the certificate for you:

./letsencrypt-auto --agree-dev-preview --server https://acme-v01.api.letsencrypt.org/directory auth

This will require that you accept the terms and conditions and enter the domain or domains you need the certificate for. For example, you may want a certificate for a domain with and without the www subdomain.

Once the tool has done its stuff, you will find the new certificate in /etc/letsencrypt/live by default, with a directory for each domain which contains the following files:

cert.pem chain.pem fullchain.pem privkey.pem

The important files which you will use with Nginx are fullchain.pem and privkey.pem.

So open the relevant virtual host file (usually in /etc/nginx/sites-enabled) and add the following lines to the server block:

server {
listen 443 ssl;

server_name <domain name>;

ssl on;
ssl_certificate /etc/letsencrypt/live/<domain name>/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/<domain name>/privkey.pem;

...
}

Of course replace domain name with the actual domain name (or names for the server_name directive if more than one, e.g. with and without www).

These are the minimum settings you need to add in order to enable https for your site, but I recommend you have a look at Mozilla’s SSL config generator for additional settings to improve the security of your setup. For example I’m currently using the following settings:

ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers "ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA";
ssl_prefer_server_ciphers on;

add_header Strict-Transport-Security max-age=15768000;

ssl_stapling on;
ssl_stapling_verify on;

Once you have completed the configuration, reload or restart Nginx and test the configuration with this service.

If all is configured properly you should get a very good score, e.g.:

screen-shot-2015-12-18-at-22-35-31

Optionally, you may also want to redirect all the plain http traffic to the https ‘version’ of your site. To do this, just add another server block to the virtual hosts like the following:

server {
listen 80;
server_name <domain name>;
rewrite ^/(.*) https://<domain name>/$1 permanent;
}

So, why do you need to stop Nginx before generating a certificate with Let’s Encrypt? When you request a certificate with a typical provider, they need to verify that you own the domain and this is done, for example, by sending an email to an email address of that domain with a confirmation link. If you own the domain, of course you have access to that email address and therefore you can proceed with the next steps required to get the certificate.

With Let’s Encrypt, everything is automated but they still need to verify the ownership of the domain first. So when you run letsencrypt-auto, it starts an HTTP server listening to the port 80 and requests a certificate from Let’s Encrypt CA. The CA, in order to verify that you own the domain, makes an HTTP request to your domain, which of course will be served by letsencrypt-auto’s server, confirming that you own the domain. Because this HTTP server runs on the port 80, you can’t run your Nginx server on the port 80 at the same time, so while you generate a certificate with letsencrypt-auto you will need to stop Nginx first. It doesn’t take long to get a certificate but this may be a problem depending on the application, especially considering that -as we’ll see later- Let’s Encrypt certificates must be renewed every 90 days. There is a module for Apache that does all of this automatically without downtime, but as said the same support for Nginx is still in the works so in the meantime you will have to stop Nginx while generating the certificate. Please note that what I described is the easiest way to obtain and install a certificate with Let’s Encrypt, so there may be other ways to do this without downtime even with Nginx. Update: I found this which might be of interest.

Limitations

Unfortunately, Let’s Encrypt certificates come with some limitations:

  • only Domain Validation (DV) certificates are issued, so the browsers will show the padlock as expected. However Organisation Validation and Extended Validation certificates are not available and apparently Let’s Encrypt has no plans to offer these certificates because they require some human intervention and thus they cost money, so the generation of these certificate cannot be fully automated nor offered for free, which are the key features of Let’s Encrypt.
  • wildcard certificates aren’t available either; you can get certificates for multiple subdomains though. This may be a problem with some applications.
  • certificates expire in 90 days, which seems a bit too short. See this for an explanation.
  • there is a limit of 5 certificates for a registered domain in 7 days; this limit should be lifted when Let’s Encrypt is out of beta. So for example if you request separate certificates for mydomain.com, http://www.mydomain.com and mail.mydomain.com these will be counted as 3 certificates for the same domain. But of course you can request a certificate with multiple subdomains at once.
  • all major browsers are supported, but some devices don’t recognise these certificates. See this list for more info.

Even with these limitations, Let’s Encrypt is an exciting initiative and it is likely that things will improve when LE is out of beta. It’s a great service because by offering free certificates that are also easier to obtain, it will surely speed up the adoption of TLS/SSL encryption, making for a more secure web.

I don’t have any particular reasons for enabling encryption on all pages on this blog since it doesn’t manage any user data and I am outsourcing comments to Disqus, but I am planning on switching anyway because another added benefit of https is that it helps increase search engine raking.

So if you haven’t yet, check Let’s Encrypt out!

Setting up a Ubuntu server for Ruby and PHP apps

There are several guides on the Internet on setting up a Ubuntu server, but I thought I’d add here some notes on how to set up a server capable of running both Ruby and PHP apps at the same time. Ubuntu’s latest Long Term Support (LTS) release is 14.04, so this guide will be based on that release.

I will assume you already have a a server with the basic Ubuntu Server Edition installed – be it a dedicated server or a VPS from your provider of choice – with just SSH access enabled and nothing else. We’ll be bootstrapping the basic system and install all the dependencies required for running Ruby and PHP apps; I usually use Nginx as web server, so we’ll be also using Phusion Passenger as application server for Ruby and fastcgi for PHP to make things easier.

First steps

Before anything else, it’s a good idea to update the system with the latest updates available. So SSH into the new server with the IP and credentials you’ve been given and -recommended- start a screen session with

screen -S <session-name>

Now change the root password with

passwd

then open /root/.ssh/authorized_keys with and editor and ensure no SSH keys have already been added other than yours; if you see any keys, I recommend you comment them out and uncomment them only if you ever need to ask your provider for support.

Done that, as usual run:

apt-get update
apt-get upgrade -y

to update the system.

Next, edit /etc/hostname with vi or any other editor and change the hostname with the hostname you will be using to connect to this server; also edit /etc/hosts and add the correct hostname in there as well. Reboot:

reboot now

SSH access

It’s a good idea to use a port other than the default one for SSH access, and a user other than root. In this guide, we’ll be:

  • using the example port 17239
  • disabling the root access and enabling access for the user deploy (only) instead
  • switching from password authentication to public key authentication for good measure.

Of course you can choose whichever port and username you wish.

For convenience, on your client computer (that is, the computer you will be connecting to the server from) edit ~/.ssh.config and add the following content:

Host my-server (or whichever name you prefer)
Hostname <the ip address of the server>
Port 22
User root

So you can more easily SSH into the new server with just

ssh my-server

As you can see for now we are still using the default port and user until the SSH configuration is updated.

Unless your public key has already been added to /root/.ssh/authorized_keys during the provisioning of the new server, still on the client machine run

ssh-copy-id <hostname or ip of the server>

to copy your public key over. You should now be able to SSH into your server without password.

Back on the server, it’s time to setup the user which you will be using to SSH into the server instead of root:

adduser deploy

Edit /etc/sudoers and add:

deploy ALL=(ALL:ALL) ALL

On the client, ensure you can SSH into the server as deploy using your key:

ssh-copy-id deploy@my-server

You should now be able to login as deploy without password.

Now edit /etc/ssh/sshd_config and change settings as follows:

Port 17239
PermitRootLogin no
PasswordAuthentication no
UseDNS no
AllowUsers deploy

This will:

  • change the port
  • disable root login
  • disable password authentication so we are forced to use public key authentication
  • disable DNS lookups so to speed up logins
  • only allow the user deploy to SSH into the system

Restart SSH server with:

service ssh restart

Keep the current session open just in case for now. On the client, open again ~/.ssh/config and update the configuration of the server with the new port and user:

Host my-server (or whichever name you prefer)
Hostname <the ip address of the server>
Port 17239
User deploy

Now if you run

ssh my-server

you should be in as deploy without password. You should no longer be able to login as root though; to test run:

ssh root@my-server date

you should see an error:

Permission denied (publickey).

Firewall

Now that SSH access is sorted, it’s time to configure the firewall to lock down the server so that only the services we want (such as ssh, http/https and mail) are allowed. Edit the file /etc/iptables.rules and paste the following:

# Generated by iptables-save v1.4.4 on Sat Oct 16 00:10:15 2010
*filter
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 ! -i lo -j DROP
-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p tcp -m tcp --dport 80 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 443 -j ACCEPT
-A INPUT -p tcp -m tcp --dport 587 -j ACCEPT
-A INPUT -p tcp -m state --state NEW -m tcp --dport 17239 -j ACCEPT
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables [Positive[False?]: " --log-level 7
-A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -j LOG
-A INPUT -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -j ACCEPT
COMMIT
# Completed on Sat Oct 16 00:10:15 2010
# Generated by iptables-save v1.4.4 on Sat Jun 12 23:55:23 2010
*mangle
:PREROUTING ACCEPT
:INPUT ACCEPT
:FORWARD ACCEPT
:OUTPUT ACCEPT
:POSTROUTING ACCEPT
COMMIT
# Completed on Sat Jun 12 23:55:23 2010
# Generated by iptables-save v1.4.4 on Sat Jun 12 23:55:23 2010
*nat
:PREROUTING ACCEPT
-A PREROUTING -p tcp --dport 25 -j REDIRECT --to-port 587
:POSTROUTING ACCEPT
:OUTPUT ACCEPT
COMMIT
# Completed on Sat Jun 12 23:55:23 2010

It’s a basic configuration I have been using for some years. It locks all incoming traffic apart from SSH access, web traffic (since we’ll be hosting Ruby and PHP apps) and mail. Of course, make sure you specify the SSH port you’ve chosen here if other than 17239 as in the example.

To apply the setting now, run:

iptables-restore < /etc/iptables.rules

and verify with

iptables -L

You should see the following output:

Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
DROP all -- anywhere loopback/8
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere tcp dpt:http
ACCEPT tcp -- anywhere anywhere tcp dpt:https
ACCEPT tcp -- anywhere anywhere tcp dpt:submission
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:17239
LOG all -- anywhere anywhere limit: avg 5/min burst 5 LOG level debug prefix "iptables [Positive[False?]: "
ACCEPT icmp -- anywhere anywhere icmp echo-request
LOG all -- anywhere anywhere LOG level warning
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere

Now if you reboot the server, these settings will be lost, so you need to persist them in either of two ways:

1) open /etc/network/interfaces and add, in the eth0 section, the following line:

post-up iptables-restore < /etc/iptables.rules

So the file should now look similar to the following:

auto eth0
iface eth0 inet static
address ...
netmask ...
gateway ...
up ip addr add 10.16.0.5/16 dev eth0
dns-nameservers 8.8.8.8 8.8.4.4
post-up iptables-restore < /etc/iptables.rules

OR,

2) Run

apt-get install iptables-persistent

Either way, reboot now and verify again with iptables -L that the settings are persisted.

ZSH shell, editor (optional)

If you like me prefer ZSH over BASH and use VIM as editor, first install ZSH with:

apt-get install zsh git-core
curl -L https://raw.github.com/robbyrussell/oh-my-zsh/master/tools/install.sh | sh
ln -s ~/dot-files/excid3.zsh-theme ~/.oh-my-zsh/themes

Then you may want to use my VIM configuration so to have a nicer editor environment:

cd; git clone https://github.com/vitobotta/dot-files.git
cd dot-files; ./setup.sh

I’d repeat the above commands for both the deploy user and root (as usual you can use sudo -i for example to login as root). Under deploy, you’ll need to additionally run:

chsh

and specify /usr/bin/zsh as your shell.

Dependencies for Ruby apps

You’ll need to install the various dependencies required to compile Ruby and install various gems:

apt-get install build-essential curl wget openssl libssl-dev libreadline-dev libmysqlclient-dev ruby-dev mysql-client ruby-mysql xvfb firefox libsqlite3-dev sqlite3 libxslt1-dev libxml2-dev

You’ll also need to install nodejs for the assets compilation (Rails apps):

apt-get install software-properties-common
add-apt-repository ppa:chris-lea/node.js
apt-get update
apt-get install nodejs

Next, as deploy:

Ensure the following lines are present in the shell rc files (.zshrc and .zprofile) and reload the shell so the new Ruby can be “found”:

export PATH="$HOME/.rbenv/bin:$HOME/.rbenv/shims:$PATH"
eval "$(rbenv init -)"

ruby -v should now output the expected version number, 2.2.4 in the example.

Optionally, you may want to install the rbenv-vars plugin for environment variables support with rbenv:

git clone https://github.com/sstephenson/rbenv-vars.git ~/.rbenv/plugins/rbenv-vars
chmod +x ~/.rbenv/plugins/rbenv-vars/bin/rbenv-vars

Dependencies for PHP apps

Install the various packages required for PHP-FPM:

apt-get install php5-fpm php5-mysql php5-curl php5-gd php5-intl php-pear php5-imagick php5-mcrypt php5-memcache php5-memcached php5-ming php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl php5-geoip php5-mcrypt php-apc php5-imap

MySQL

I am assuming here you will be using MySQL – I usually use the Percona distribution. If you plan on using some other database system, skip this section.

First, install the dependencies:

apt-get install curl build-essential flex bison automake autoconf bzr libtool cmake libaio-dev libncurses-dev zlib1g-dev libdbi-perl libnet-daemon-perl libplrpc-perl libaio1
gpg --keyserver hkp://keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A
gpg -a --export CD2EFD2A | sudo apt-key add -

Next edit /etc/apt/sources.list and add the following lines:

deb http://repo.percona.com/apt trusty main
deb-src http://repo.percona.com/apt trusty main

Install Percona server:

apt-get update
apt-get install percona-xtradb-cluster-server-5.5 percona-xtradb-cluster-client-5.5 percona-xtradb-cluster-galera-2.x

Test that MySQL is running:

mysql -uroot -p

Getting web apps up and running

First install Nginx with Passenger for Ruby support (also see this:

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
apt-get install apt-transport-https ca-certificates

Edit /etc/apt/sources.list.d/passenger.list and add the following:

deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main

Update sources:

chown root: /etc/apt/sources.list.d/passenger.list
chmod 600 /etc/apt/sources.list.d/passenger.list
apt-get update

Then install Phusion Passenger for Nginx:

apt-get install nginx-extras passenger

Edit /etc/nginx/nginx.conf and uncomment the passenger_root and passenger_ruby lines, making sure the latter points to the version of Ruby installed with rbenv, otherwise it will point to the default Ruby version in the system. Make the following changes:

user deploy;
worker_processes auto;
pid /run/nginx.pid;

events {
use epoll;
worker_connections 2048;
multi_accept on;
}

http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
server_tokens off;
…
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
passenger_ruby /home/deploy/.rbenv/shims/ruby;
passenger_show_version_in_header off;
}

Restart nginx with

service nginx restart

Test that nginx works by opening http://the_ip_or_hostname in your browser.

For PHP apps, we will be using fastcgi with unix sockets. Create for each app a profile in /etc/php5/fpm/pool.d/, e.g. /etc/php5/fpm/pool.d/myapp. Use the following template:

[<app name>]
listen = /tmp/<app name>.php.socket
listen.backlog = -1
listen.owner = deploy
listen.group = deploy

; Unix user/group of processes
user = deploy
group = deploy

; Choose how the process manager will control the number of child processes.
pm = dynamic
pm.max_children = 75
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 500

; Pass environment variables
env[HOSTNAME] = $HOSTNAME
env[PATH] = /usr/local/bin:/usr/bin:/bin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

; host-specific php ini settings here
; php_admin_value[open_basedir] = /var/www/DOMAINNAME/htdocs:/tmp

To allow communication between Nginx and PHP-FPM via fastcgi, ensure each PHP app’s virtual host includes some configuration like the following:

location / {
try_files $uri /index.php?$query_string;
}

location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass unix:/tmp/<app name>.php.socket;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root/$fastcgi_script_name;
}

Edit /etc/php5/fpm/php.ini and set cgi.fix_pathinfo to 0. Restart both FPM and Nginx:

service php5-fpm restart
service nginx restart

Congrats, you should now be able to run both Ruby and PHP apps.

Backups

There are so many ways to backup a server…. what I usually use on my personal servers is a combination of xtrabackup for MySQL databases and duplicity for file backups.

As root, clone my admin scripts:

cd ~
git clone https://github.com/vitobotta/admin-scripts.git
apt-key adv --keyserver keys.gnupg.net --recv-keys 1C4CBDCDCD2EFD2A

Edit /etc/apt/sources.list and add:

deb http://repo.percona.com/apt trusty main
deb-src http://repo.percona.com/apt trusty main

Proceed with the installation of the packages:

apt-get update
apt-get install duplicity xtrabackup

Next refer to this previous post for the configuration.

Schedule the backups with crontab -e by adding the following lines:

MAILTO = <your email address>

00 02 * * sun /root/admin-scripts/backup/duplicity.sh full
00 02 * * mon-sat /root/admin-scripts/backup/duplicity.sh incr
00 13 * * * /root/admin-scripts/backup/xtrabackup.sh incr

Mailing

  • install postfix and dovecot with
apt-get install postfix dovecot-common mailutils
  • run dpkg-reconfigure postfix and set the following:
  • General type of mail configuration -> Internet Site
  • System mail name -> same as the server’s hostname
  • Root and postmaster email recipient -> your email address
  • Force synchronous updates on mail queue -> no
  • Local networks -> leave default
  • Mailbox size limit (bytes) -> set 10485760 (10MB) or so, to prevent the default mailbox from growing with no limits
  • Internet protocols to use -> all

  • SMTP authentication: run

postconf -e 'home_mailbox = Maildir/'
postconf -e 'smtpd_sasl_type = dovecot'
postconf -e 'smtpd_sasl_path = private/auth'
postconf -e 'smtpd_sasl_local_domain ='
postconf -e 'smtpd_sasl_security_options = noanonymous'
postconf -e 'broken_sasl_auth_clients = yes'
postconf -e 'smtpd_sasl_auth_enable = yes'
postconf -e 'smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination'
  • TLS encryption: run
mkdir /etc/postfix/certificate && cd /etc/postfix/certificate
openssl genrsa -des3 -out server.key 2048
openssl rsa -in server.key -out server.key.insecure
mv server.key server.key.secure
mv server.key.insecure server.key
openssl req -new -key server.key -out server.csr
openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt

postconf -e 'smtp_tls_security_level = may'
postconf -e 'smtpd_tls_security_level = may'
postconf -e 'smtp_tls_note_starttls_offer = yes'
postconf -e 'smtpd_tls_key_file = /etc/postfix/certificate/server.key'
postconf -e 'smtpd_tls_cert_file = /etc/postfix/certificate/server.crt'
postconf -e 'smtpd_tls_loglevel = 1'
postconf -e 'smtpd_tls_received_header = yes'
postconf -e 'myhostname = <hostname>'
  • SASL
  • edit /etc/dovecot/conf.d/10-master.conf, and uncomment the following lines so that they look as follows (first line is a comment so leave it…commented out):

Postfix smtp-auth

unix_listener /var/spool/postfix/private/auth {
mode = 0666
}
* edit /etc/dovecot/conf.d/10-auth.conf and change the setting auth_mechanisms to “plain login”
* edit /etc/postfix/master.cf and a) comment out smtp, b) uncomment submission
* restart postfix: service postfix restart
* restart dovecot: service dovecot restart
* verify that all looks good

root@nl:/etc/postfix/certificate# telnet localhost 587
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 <hostname> ESMTP Postfix (Ubuntu)
ehlo <hostname>
250-<hostname>
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-AUTH PLAIN LOGIN
250-AUTH=PLAIN LOGIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

Test email sending:

echo "" | mail -s "test" <your email address>

There’s a lot more that could be done, but this should get you started. Let me know in the comments if you run into any issues.