Solid Cable in Production with Kamal

Last week I wrote about Solid Cable and how amazingly easy it was to get up and running and add real time web socket magic to a Rails app. Rails 8 and the solid trifecta of Solid Cache, Solid Queue and Solid Cable, is a massive unlock to productivity, particularly evident when adding some of these historically pretty complicated capabilities and services: caching, job queues and web sockets respectively.

When I went to move this to production, I realised that there often aren’t that many blog posts or examples - including my own previous posts - that go into details beyond a demo on localhost. Lets fix that now! It was pretty straightforward, but a couple of tiny details caught me out, so wanted to capture those for next time!

Managing Multiple Databases in Rails

In the previous post we set up the “primary” and “cable” databases both with SQLite in the development environment as that’s the default in Rails 8, and also the simplest. But in my production app - and I suspect like many others - I was already using PostgreSQL as my main “primary” database. So we need to add SQLite in production to run alongside - in my case - Postgres. We configure database connections in config/database.yml so open this up and have a look at what you currently have.

In my case I had a &default YAML anchor to set up the common configs, then a key each for development, test and production that pulls in the &default config with a merge key and adds the environment specific configs in each as needed. I had postgresql set up for all three environments as shown below. You may have something different, like sqlite3 or mysql as your primary, but the general idea is the same.

# Existing database.yml
default: &default
  adapter: postgresql
  encoding: unicode
  port: 5432
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

development:
  <<: *default
  database: myapp_development

test:
  <<: *default
  database: myapp_test

production:
  <<: *default
  host: <%= ENV["DB_HOST"] %>
  database: <%= ENV["POSTGRES_DB"] %>
  username: <%= ENV["POSTGRES_USER"] %>
  password: <%= ENV["POSTGRES_PASSWORD"] %>

Ok. So now to use Solid Cable with SQLite while keeping PostgreSQL for the main app database we need to update database.yml to support multiple databases ie. SQLite as well as our existing PostgreSQL. This capability was introduced in Rails 6 I think, and you can read all about it in the excellent Rails Guides if you want to get some more background.

Firstly, lets add a new YAML anchor for sqlite to config/database.yml as shown here. Put it just below the existing &default anchor block.

sqlite: &sqlite
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

When using multiple databases in an environment, we can specify them by adding a configuration key for each under the environment key. Here’s how my database.yml file looked after adding the &sqlite anchor and then adding primary and cable configurations for each environment:

default: &default
  adapter: postgresql
  encoding: unicode
  port: 5432
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>

sqlite: &sqlite
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  primary:
    <<: *default
    database: myapp_development
  cable:
    <<: *sqlite
    database: storage/myapp_development_cable.sqlite3
    migrations_paths: db/cable_migrate

test:
  primary:
    <<: *default
    database: myapp_test
  cable:
    <<: *sqlite
    database: storage/myapp_test_cable.sqlite3
    migrations_paths: db/cable_migrate

production:
  primary:
    <<: *default
    host: <%= ENV["DB_HOST"] %>
    database: <%= ENV["POSTGRES_DB"] %>
    username: <%= ENV["POSTGRES_USER"] %>
    password: <%= ENV["POSTGRES_PASSWORD"] %>
  cable:
    <<: *sqlite
    database: /data/myapp_production_cable.sqlite3
    migrations_paths: db/cable_migrate

We’re only really interested in the “production” settings in the context of this post but I’ve left the rest in for completeness.

So, lets go over this line by line:

primary: specifies the primary database. In our case that’s our existing “main” application database. It should always be named “primary” to keep things clear.
<<: *default is a YAML merge key that pulls in the configuration from the &default anchor
host, database, username and password obviously are the location, name and auth credentials for your database. Best practice is to pass these in from environment variables as shown
cable is the configuration key for our Solid Cable SQLite instance! It’s important that we name it “cable” so it matches with the name in config/cable.yml
<<: *sqlite is another YAML merge key that pulls in the config from the &sqlite anchor
database pay close attention here! this is the path to the SQLite database file in production. In my case I have this in a volume mounted by Kamal. More on that later
migrations_paths is the path in our codebase where we can add migrations specific to the SQL database

Solid Cable Configuration

Nothing to change here but as a reminder, Solid Cable is configured in config/cable.yml and important as mentioned above that the production database configuration key for the SQLite database we’re using for Solid Cable is named the same here. ie “cable”

production:
  adapter: solid_cable
  connects_to:
    database:
      writing: cable
  polling_interval: 0.1.seconds
  message_retention: 1.day

Kamal SQLite Location

As I’ve written about numerous times in the past like here and here, I’m all in on Kamal! It’s a fantastic tool for deploying web apps. Previously I had only used PostgreSQL and only one database per app. Now we need to add SQLite to the mix. Fortunately it’s super simple, but I had one gotcha that took some fiddling to get right and wanted to capture here.

As you saw above, in the database.yml file we need to specify the name and location of the SQLite database file. The Rails default is to set this to database: storage/production_cable.sqlite3 which makes sense as a default.

In our case with Kamal, we want to make sure that this database persists across deployments, so we need to make sure it’s mounted on the server’s file system outside the docker container but still accessible from the container each time it’s deployed with Kamal.

Fortunately this is really easy too with the volumes key in config/deploy.yml. With this setting, Kamal will automatically make a persistent storage volume at /data on our server.

## config/deploy.yml
volumes:
  - "myapp_data:/data"

Then we need to make sure we are using this volume in the database.yml config so that Rails knows where to find the file. In my case above it’s database: /data/myapp_production_cable.sqlite3

The embarrassing “gotcha” I mentioned above was I was missing / in front of data so Rails was trying to put the database in the relative path /rails/data and not the volume I had mounted at /data! Very annoying but didn’t take long to realise :)

Deploying

Finally we just need to deploy the app! Kamal always runs rails db:prepare which will create our SQLite database the first time and that should be it! Solid Cable in production

Verifying

To double check our SQLite instance is up and running and brokering our web socket messages as we expect, we can jump in to the console in production to see if we have any messages. Make sure to trigger an event in your app first and then check if the message was persisted:

kamal app exec -i 'bin/rails console'

> SolidCable::Message.last
#<SolidCable::Message:0x00007ff53fe4b8f8
 id: 12,
 channel: "item_106",
 payload: "\"\\u003cturbo-stream action=\\\"prepend\\\" target=\\\"un...",
 created_at: "2024-11-30 12:26:30.443258000 +0000",
 channel_hash: -7306872458338484105>

Lovely! Here we can see a payload was captured to prepend some content on channel “item_106”.

Wrapping Up

OK! I didn’t find any articles specifically on actually deploying Solid Cable to production with SQLite while maintaining the primary database on PostgreSQL which prompted me to write this one. Future me thanks myself at least.

Don’t hesitate to drop me a note at my new favourite place Blue Sky or via any channels listed here