Home

Upgrading to Kamal 2

#all #tech #rails

Kamal

Back in February I wrote about an exciting new deployment tool called Kamal announced by DHH and 37signals. It really was a great gift and I quickly switched my personal projects over to Kamal and hosting on cheap VPS’s a Hetzner. The cool-aid was flowing and it tasted.. well, great!

It was a bit fiddly to get working and had a few limitations - such as deploying multiple apps on one host was a challenge - but for my use case it was perfect. Been using it ever since.

Kamal 2

Just in time for Rails World 2024, Donal from 37signals announced Kamal 2 which introduced some key enhancements:

  • kamal-proxy that replaces traefik
  • automatic HTTPS and provisioning with Let’s Encrypt
  • support for multiple apps on one server
  • command aliases
  • move away from .env for secrets

Upgrading to Kamal 2

I spent some time cautiously setting up a beta subdomain and deployed a second instance of my app to fresh servers just for testing the upgrade, but it was unbelievably straightforward. So much so, I was convinced it hadn’t changed anything but yep, it was that seamless!

The official upgrade guide is likely all you’ll need but I made the following brief notes to capture some nuances that tripped me up, and mainly to remind future me, but hopefully may help someone who stumbles on this page in the future.

1. Upgrade the Kamal gem

First things first, we need to update the gem! The guide tells us to make sure we have v1.9.0 or install if not, and then confirm we can do a deploy, since this is the first version that can reverse the upgrade we’re about to perform.

gem install kamal --version 1.9.0

Once we’ve deployed to confirm its all still working, we now can upgrade kamal to the latest. At the time of writing this is 2.1.1 but lets get the latest

gem install kamal

2. Deploy.yml Configuration Changes

OK! Now we’ve tested 1.9.0 and have a way back, and then installed the latest 2.x version of Kamal we’re ready to update the configurations.

Firstly we need to specify the architecture we’re building for. Remember Kamal uses Docker, and in my case its amd64. You can still do multi-architecture builds by adding another attribute eg arm64 but in my case, I dont need that.

builder:
  arch: amd64

The big change in Kamal 2 is kamal-proxy coming in to replace Traefik. So, we can remove all the previous configurations we had for traefik which in my case looked like this:

# Remove everything under `traefik:`
# traefik:
#   options:
#     publish:
#       - 443:443
#   args:
#     entryPoints.websecure.address: ":443"
#     api: true
#   labels:
#     traefik.http.routers.dashboard.rule: ...

Having removed the traefik config above, we need to configure kamal-proxy. Fortunately this is super simple. Add the following, ensuring to substitute in your-domain of course:

proxy:
  ssl: true
  host: your-domain.com

Some notes here. The docs were a bit confusing since setting ssl: true seemingly sets up Let’s Encrypt to issue an SSL cert, but only when you have one server.

In my case I’m using Cloudflare for SSL/TLS encryption, and despite reading I should set ssl: false, after much fiddling around, I found I did indeed need to set it to true and make sure Cloudflare Encryption Mode was set to “Full”.

The other gotcha for me was with Kamal 1.x traefik accepted the incoming connections on port 443 and expected the Rails server to be running on port 3000.

The default behaviour for Kamal 2.x and kamal-proxy is it to connect to port 80 on the Rails server. This is due to the addition of Thruster - more on that in a minute - but for now just know if you don’t plan to add Thruster, you need to tell the proxy to use port 3000 by adding app_port: 3000 under the proxy: key.

Lastly for the deploy.yml, you can optionally add aliases in Kamal 2.x. This is pretty handy. I was always looking up how to connect to the rails console on my app server for example. These are the defaults, but feel free to add your own too as needed

aliases:
  console: app exec --interactive --reuse "bin/rails console"
  shell: app exec --interactive --reuse "bash"
  logs: app logs -f
  dbc: app exec --interactive --reuse "bin/rails dbconsole"

So now you can just enter kamal console on the command line and boom, you’re dropped right into the console!

3. Move .env to .kamal/secrets

Next we need to update our secrets! Previously in Kamal 1.x it simply used the .env file for secrets which is really convenient but somewhat fragile as they serve other purposes too.

In Kamal 2.x a new .kamal/secrets file was introduced specifically for your secrets. The easiest way if you’re upgrading is to reference the env vars in .env that you need for your app. Create the file .kamal/secrets and add the following:

KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
RAILS_MASTER_KEY=$(cat config/master.key)
POSTGRES_PASSWORD=$POSTGRES_PASSWORD
S3_ACCESS_KEY_ID=$S3_ACCESS_KEY_ID
S3_SECRET_ACCESS_KEY=$S3_SECRET_ACCESS_KEY

One gotcha: I found I needed to prefix kamal commands with dotenv now to have the secrets available, so just fyi if you have issues with secrets, it may be this! eg. dotenv kamal deploy

4. Thruster

Thruster wraps Puma and provides some great features like Basic HTTP Caching, X-Sendfile support and compression, HTTP/2 support and Automatic TLS certificate management with Let’s Encrypt

It so simple to add, and since it default in Rails 8, lets go for it. First we need to add the gem:

bundle add thruster

Next, if you’re not using Rails 8 yet, you’ll need to add the command script, so create a new file bin/thrust and copy in the following, being sure to set the permission on that file after creating it with chmod 755 bin/thrust from your root directory:

#!/usr/bin/env ruby
require "rubygems"
require "bundler/setup"

load Gem.bin_path("thruster", "thrust")

Lastly we need to configure Docker to fire up Thruster by updating Dockerfile as below. Replace the first three lines with the second three lines:

- # Start the server by default, this can be overwritten at runtime
- EXPOSE 3000
- CMD ["./bin/rails", "server"]
+ # Start server via Thruster by default, this can be overwritten at runtime
+ EXPOSE 80
+ CMD ["./bin/thrust", "./bin/rails", "server"]

As you can see, this is where we’re setting Rails to run on port 80 as mentioned in step 2 above.

5. Run the upgrade command

OK!! Now we’re ready to rumble. WARNING run this on a test server first!

Here we go. Simply run the following to initiate the in-place upgrade process

kamal upgrade

This will remove traefik from your containers, add kamal-proxy and then reboot the current app.

Amazingly in my case, it Just Worked(TM)! I was so surprised I had to double check processes on the server to make sure, but yes, worked first time!

6. Rolling Back

If things didn’t go to plan, since we installed 1.9.0 (hopefully you did) then we can rollback.

Firstly, lets uninstall kamal 2.x. Make sure to set the version to the one you installed. In my case it was 2.1.1

gem uninstall kamal -v 2.1.1

Now lets check you have kamal 1.9.0 ready. Running the command below should return the output 1.9.0

kamal version

Next you need to revert the changes to deploy.yml and Dockerfile from steps 2 and 3 above, and then finally we can run the downgrade command that removes kamal-proxy and adds traefik back to your containers

kamal downgrade

Wrapping up

This is a really niche post. Only helpful if you a) installed Kamal 1.0 and b) for some reason don’t want to follow the official guides, but I like to document this stuff if only for myself. If it helps someone else. That’s a bonus!

As always, any questions of feedback, don’t hesitate to drop me a note on twitter or via any other channels listed here