Deploying with Kamal

19th Feb 2024

#all #tech #rails

Intro

I’ve been hosting 1500cals.com on fly.io as I wrote about previously, and frankly, its really great. Particularly to get a Rails app up and hosted in minutes, harkening back to the old Heroku days when it was the defacto answer for almost instant free Rails hosting.

However, last year when DHH introduced Kamal my interest was piqued! He described it as

A simple yet battle-tested deployment tool extracted from our cloud exit, which features zero-downtime deploys, rolling restarts, asset bridging, and just about everything else most people will need to deploy and update their web applications. Whether they run on cloud VMs or their own bare metal.

Overview

In short, you can deploy your Rails app with Kamal anywhere: to any server, VPS, cloud VM or your own boxes if you like. It handles all the complexity and well within the grasp of a typical web developer used to PaaS offerings. The built in integration with traefik means no down time when you deploy, and having everything containerised with Docker means reliability and reproducibility is baked in. I love it already!

Here I’ll show you the steps I took to migrate 1500cals.com from fly.io to my own VMs provisioned on Hetzner, and you’ll see just how easy it is! Whats more is, if and when you want to move hosting providers, cloud or even provision another for high availability, it will be trivial to do with a few tweaks to Kamal.

1. Install Kamal

First up, lets install Kamal. Installing system wide makes sense in most cases so:

gem install kamal

Now we initialise our app ready for Kamal, so cd in to your app root directory, and run

kamal init

This will generate a few files to get us started:

• config/deploy.yml
• .env
• .kamal/hooks/*

2. Docker

As mentioned above, Kamal deploys our app and services in Docker Containers, and we let Docker know how to configure a container using the Dockerfile. If you’re new to Docker, this configuration can look a bit strange, but it really is straightforward. I recommend following this Docker 101 tutorial on the docker website to get a quick intro if needed

If you generated your app with Rails 7.1 or later, the default Dockerfile will be there in your root directory. If not, copy the text below into a new file named Dockerfile in your root directory. The only difference below from the default generated by Rails 7.1 is I changed the RUBY_VERSION to 3.2.3 to match the version I’m using, and added the libpq-dev packages necessary to build PostgreSQL and the replaced libsqlite3-0 with the postgresql-client client libraries, so feel free to change these back if you’re just using SQLite.

# syntax = docker/dockerfile:1

# Make sure RUBY_VERSION matches the Ruby version in .ruby-version and Gemfile
ARG RUBY_VERSION=3.2.3
FROM registry.docker.com/library/ruby:$RUBY_VERSION-slim as base

# Rails app lives here
WORKDIR /rails

# Set production environment
ENV RAILS_ENV="production" \
    BUNDLE_DEPLOYMENT="1" \
    BUNDLE_PATH="/usr/local/bundle" \
    BUNDLE_WITHOUT="development"


# Throw-away build stage to reduce size of final image
FROM base as build

# Install packages needed to build gems
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y build-essential git libpq-dev libvips pkg-config

# Install application gems
COPY Gemfile Gemfile.lock ./
RUN bundle install && \
    rm -rf ~/.bundle/ "${BUNDLE_PATH}"/ruby/*/cache "${BUNDLE_PATH}"/ruby/*/bundler/gems/*/.git && \
    bundle exec bootsnap precompile --gemfile

# Copy application code
COPY . .

# Precompile bootsnap code for faster boot times
RUN bundle exec bootsnap precompile app/ lib/

# Precompiling assets for production without requiring secret RAILS_MASTER_KEY
RUN SECRET_KEY_BASE_DUMMY=1 ./bin/rails assets:precompile


# Final stage for app image
FROM base

# Install packages needed for deployment
RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y curl postgresql-client libvips && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

# Copy built artifacts: gems, application
COPY --from=build /usr/local/bundle /usr/local/bundle
COPY --from=build /rails /rails

# Run and own only the runtime files as a non-root user for security
RUN useradd rails --create-home --shell /bin/bash && \
    chown -R rails:rails db log storage tmp
USER rails:rails

# Entrypoint prepares the database.
ENTRYPOINT ["/rails/bin/docker-entrypoint"]

# Start the server by default, this can be overwritten at runtime
EXPOSE 3000
CMD ["./bin/rails", "server"]

3. Configuring Kamal

config/deploy.yml

Now we have a Docker file to build our container, lets open up the config/deploy.yml that was generated earlier by kamal init, and start setting our app up for deployment with Kamal!

Container Image and Registry

Firstly we name our service. This can be whatever you like, in my case it was obviously service: 1500cals. This name will get prepended to your container names so make sense to be the name of your app.

Secondly, we’ll be building our app and pushing the Docker image to a registry that Kamal then uses for deployment. This can be any registry which you specify with the server: property. The default is docker hub so we’ll use that. If you haven’t already, create an account at https://hub.docker.com/ and then update the image property in config/deploy.yml with your Docker Hub username and service name to identify the image . In my case this is tapster/1500cals. We also need to add the username and password for our registry in the appropriate places below. Note: once your image has been generated the first time, make sure to go back to Docker Hub and mark it private if needed, since by default your container and hence source code will be available to be downloaded others.

# config/deploy.yml

# Name of your application. Used to uniquely configure containers.
service: my-app

# Name of the container image.
image: user/my-app

# Credentials for your image host.
registry:
  # Specify the registry server, if you're not using Docker Hub
  # server: registry.digitalocean.com / ghcr.io / ...
  username: my-user

  # Always use an access token rather than real password when possible.
  password:
    - KAMAL_REGISTRY_PASSWORD

Traefik

Kamal uses the excellent Traefik reverse proxy to securely route requests from clients like web browsers or apps to your Rails app within one or more docker containers. In our case we’re only going to accept secure https connections, so we need to let traefik know to publish port 443 in config/deploy.yml as shown below:

traefik:
  options:
    publish:
      - "443:443"
  args:
    entryPoints.websecure.address: ":443"

Servers

Now we need to let Kamal know about our servers and add any required rules for each host that Traefik will be directing our requests to. In the case of 1500cals, I’m running one rails server and one job server that handles our ActiveJobs with SolidQueue as I described in the previous article, so we add these under the servers: key with web: and jobs:.

Under each server there are various configuration options. The first being hosts: which is where we add the IP of our host server. I spun up two basic cloud virtual servers on Hetzner which is a great option to get started a very low cost.

Traefik by default is only connected to servers in the web role, and we can add further configuration rules with labels:. In my case, we’re only allowing https, so we set up traafik routers as shown below that specify the secure entrypoint and also the hostnames we want routed to our web server. You can read more about Traefik config here https://doc.traefik.io/traefik/providers/docker/

Lastly for servers: notice under the job: server we can add a command that is run on startup. In our case bundle exec rake solid_queue:start which will kick off the SolidQueue supervisor, ready to consume and process our jobs! Lovely

servers:
  web:
    hosts:
      - <add-your-web-server-ip-here>
    labels:
      traefik.http.routers.1500cals_secure.entrypoints: websecure
      traefik.http.routers.1500cals_secure.rule: "Host(`1500cals.com`) || Host(`www.1500cals.com`)"
      traefik.http.routers.1500cals_secure.tls: true
  job:
    hosts:
      - <add-your-jobs-server-ip-here>
    cmd: bundle exec rake solid_queue:start

Accessories

Services like a database, redis server or similar are grouped as “accessories” in Kamal. So to set up our PostgreSQL instance, you guessed it, we need to add some configuration in config/deploy.yml under accessories:

accessories:
  database:
    image: postgres:16.2
    host: <add-your-database-server-ip-here>
    port: 5432:5432
    env:
      clear:
        POSTGRES_USER: postgres
        POSTGRES_DB: <add-your-database-name-here>
      secret:
        - POSTGRES_PASSWORD
    directories:
      - data:/var/lib/postgresql/data

Here we’re naming our accessory database, specifying the official PostgreSQL docker image name with image: postgres:16.2. We map the port using the default 5432 and the location of our data file with the directories: key. The PostgreSQL user, db and password are set under the env: which I’ll explain next.

.env

You may have noticed we used KAMAL_REGISTRY_PASSWORD for the password: in config/deploy.yml and also above we have POSTGRES_PASSWORD and some other constants for the database set under an env: key.

Kamal leverages .env files on the servers to manage constants, passwords etc. The base comes from the .env file that was generated with kamal init.

Before we do anything else, make sure we have an entry in .gitignore for .env. Rails will add this by default but always good to check in case you’re upgrading an old app or its been removed. We never want to check this in to our repo, so always good to confirm this before we start.

Now, if you open up the .env file you’ll see placeholders for KAMAL_REGISTRY_PASSWORD and RAILS_MASTER_KEY so lets add these now. KAMAL_REGISTRY_PASSWORD is the password or token for your registry. In our case we used Docker Hub so we can generate a token at https://hub.docker.com/settings/security and copy the value. As for RAILS_MASTER_KEY, you’ll find that in your Rails app config/master.key file.

# .env

KAMAL_REGISTRY_PASSWORD=change-this
RAILS_MASTER_KEY=another-env

This file is on your local machine of course. When deploying, Kamal will take this file, and then inject any entries under env: keys, like those under the database: accessory above and copy to the server. The keys under clear: are as the name suggests, in clear text. This is good for things like URL endpoints, your database name, or any constant you need access to that is not secret. For things like passwords, or any private credentials, keep them in your local .env file and refer to them under the secret: key as above.

4. Deploying our App

The first time we want to deploy our app, we need to use the kamal setup command that will set up all the servers and accessories. This includes installed docker on the servers if not there already.

All being well, assuming you set up your DNS to point your domain to the server IP, you should now be able to access your Rails app at your URL!

When you update your app and need to deploy you simply issue the kamal deploy command which will rebuild your container image as needed and deploy the Rails app. It won’t make any changes to accessory servers.

Wrapping up

Bit of a long post I know, but hopefully there’s enough here to get you started if you were interested like I was to get a bit closer to the metal, deploy my app with Kamal and have a lot more flexibility around how and where I host my Rails app.

As always, any questions of feedback, don’t hesitate to drop me a note on twitter or via any other channels listed here