(This post originally appeared on the Carbon Five blog.)
We’ve been trialing the usage of Docker and Docker Compose (previously known as fig) on a Rails project here at Carbon Five. In the past, my personal experience with Docker had been that the promise of portable containerized apps was within reach, but the tooling and development workflow were still awkward - commands were complex, configuration and linking steps were complicated, and the overall learning curve was high.
My team decided to take a peek at the current landscape of Docker tools (primarily boot2docker and Docker Compose) and see how easily we could spin up a new app and integrate it into our development workflow on Mac OS X.
In the end, I’ve found my experience with Docker tools to be surprisingly pleasant; the tooling easily integrates with existing Rails development workflows with only a minor amount of performance overhead. Docker Compose offers a seamless way to build containers and orchestrate their dependencies, and helps lower the learning curve to build Dockerized applications. Read on to find out how we built ours.
Introduction to docker-compose (née Fig).
Docker Compose acts as a wrapper around Docker - it links your containers together and provides syntactic sugar around some complex container linking commands.
We liked Docker Compose for its ability to coordinate and spin up your entire application and dependencies with one command. In the past, frameworks like Vagrant were easy ways to generate a standard image for your development team to use and get started on. Docker Compose offers similar benefits of decoupling the app from the host environment, but also provides the container vehicle for the app to run in all environments - that is, the container you develop in will often be the same container that you deploy to production with.
Docker (with the orchestration tooling provided by Compose) provides us the ability to:
- Upgrade versions of Ruby or Node (or whatever runtime your app requires) in production with far less infrastructure coordination than normally required.
- Reduce the number of moving parts in the deployment process. Instead of writing complex Puppet and Capistrano deployment scripts, our deployments will now center around moving images around and starting containers.
- Simplify developer onboarding by standardizing your team on the same machine images.
In this example, we will run two Docker containers - a Rails container and a MySQL container - and rely on Compose to build, link, and run them.
Installing boot2docker, Docker, and Docker Compose.
Docker runs in a VirtualBox VM through an image called
boot2docker. The reason we have to use
boot2docker and VirtualBox is because the Mac OSX filesystem is not compatible with the type of filesystem required to support Docker. Hence, we must run our Docker containers within yet another virtual machine.
- Download and install VirtualBox.
- Now install boot2docker and Docker Compose.
1 $ brew install boot2docker docker-compose
- Initialize and start up boot2docker
1 2 $ boot2docker init $ boot2docker start
- Configure your Docker host to point to your boot2docker image.
1 $ $(boot2docker shellinit)
You’ll need to run this for every terminal session that invokes the
docker-compose command - better export this line into your
Creating a Dockerfile
Let’s start by creating a Dockerfile for this app. This specifies the base dependencies for our Rails application. We will need:
- Ruby 2.2 - for our Rails instance
- NodeJS and NPM - for installation of Karma, jshint, and other JS dependencies.
- MySQL client - for ActiveRecord tasks
- PhantomJS - for executing JS-based tests
- vim - for inspecting and editing files within our container
Dockerfile from within your Rails app directory.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 FROM ruby:2.2.0 RUN apt-get update -qq && apt-get install -y build-essential nodejs npm nodejs-legacy mysql-client vim RUN npm install -g phantomjs RUN mkdir /myapp WORKDIR /tmp COPY Gemfile Gemfile COPY Gemfile.lock Gemfile.lock RUN bundle install ADD . /myapp WORKDIR /myapp RUN RAILS_ENV=production bundle exec rake assets:precompile --trace CMD ["rails","server","-b","0.0.0.0"]
Let’s start by breaking this up line-by-line:
1 FROM ruby:2.2.0
From here on, we are going to be executing commands that will build on this reference image.
1 2 RUN apt-get update -qq && apt-get install -y build-essential nodejs npm nodejs-legacy mysql-client vim RUN npm install -g phantomjs
RUN command builds up the image, installing specific application dependencies and setting up the environment. Here we install our app dependencies both from
An aside on how a Docker image is built
One of the core concepts in Docker is the concept of “layers”. Docker runs on operating systems that support layering filesystems such as
btrfs. Changes to the filesystem can be thought of as atomic operations that can be rolled forward or backwards.
This means that Docker can effectively store its images as snapshots of each other, much like Git commits. This also has implications as to how we can build up and cache copies of the container as we go along.
The Dockerfile can be thought of as a series of rolling incremental changes to a base image - each command builds on top of the line before. This allows Docker to quickly rebuild changes to the reference image by understanding which lines have changed - and not rebuild the image from scratch each time.
Keep these concepts in mind as we talk about speeding up your Docker build in the following section.
Fast Docker builds by caching your Gemfiles
The following steps install the required Ruby gems for Bundler, within your app container:
1 2 3 4 WORKDIR /tmp COPY Gemfile Gemfile COPY Gemfile.lock Gemfile.lock RUN bundle install
Note how we sneak the gems into
/tmp, then run the
bundle install which downloads and installs gems into Bundler’s
vendor/bundle directory. This is a cache hack - whereas in the past we would have kept the
Gemfiles in with the rest of the application directory in
Keeping Gemfiles inline with the app would have meant that the entire
bundle install command would have been re-run on each
docker-compose build – without any caching – due to the constant change in the code in the
By separating out the Gemfiles into their own directory, we logically separate the Gemfiles, which are far less likely to change, from the app code, which are far more likely to change. This reduces the number of times we have to wait for a clean
bundle install to complete.
Adding the app
Finally, we finish our Dockerfile by adding our current app code to the working directory.
1 2 3 4 ADD . /myapp WORKDIR /myapp RUN RAILS_ENV=production bundle exec rake assets:precompile --trace CMD ["rails","server","-b","0.0.0.0"]
This links the contents of the app directory on the host to the
/myapp directory within the container.
Note that we precompile all our assets before the container boots up - this ensures that the container is preloaded and ready to run and jives with Docker tenets that a container should be the same container that runs in development, test, and production environments.
Setting up Docker Compose
Now that we’ve defined a
Dockerfile for booting our Rails app, we turn to the Compose piece that orchestrates the linking phase between the Rails app and its dependencies - in this case, the DB.
docker-compose.yml file automatically configures our application ecosystem. Here, it defines our Rails container and its db container:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 web: build: . volumes: - .:/myapp ports: - "3000:3000" links: - db env_file: - '.env.web' db: image: library/mysql:5.6.22 ports: - "13306:3306" env_file: - '.env.db'
1 $ docker-compose up
will spin up both the
One of the most powerful tools of using Docker Compose is the ability to abstract away the configuration of your server, no matter whether it is running as a development container on your computer, a test container on CI, or on your production Docker host.
1 2 links: - db
will add an entry for
db into the Rails’ container’s
/etc/hosts, linking the hostname to the correct container. This allows us to write our database.yml like so:
1 2 3 # config/database.yml development: &default host: db
Another important thing to note is the
1 2 3 # docker-compose.yml volumes: - .:/myapp
This mounts the current directory
. on the host Mac to the
/myapp directory in the container. This allows us to make live code changes on the host filesystem and see code changes reflected in the container.
Also note that we make use of Compose’s
env_file directive, which allows us to specify environment variables to inject into the container at runtime:
1 2 env_file: - '.env.web'
A peek into
1 2 3 4 5 6 7 PORT=3000 PUMA_WORKERS=1 MIN_THREADS=4 MAX_THREADS=16 SECRET_KEY_BASE=<Rails secret key> AWS_REGION=us-west-2 # ...
Note that the
env_file is powerful in that it allows us to swap out environment configurations when you deploy and run your containers. Perhaps your container needs separate configurations on dev than when on CI, or when deployed to staging or on production.
Creating containers and booting them up.
Now it’s time to assemble the container. From within the Rails app, run:
1 $ docker-compose build
This downloads and builds the containers that your web app and your db will live in, linking them up. You will need to re-run the
docker-compose build command every time you change the
Running your app in containers
You can bring up your Rails server and associated containers by running:
1 $ docker-compose up
This is a combination of build, link, and start-services command for
each container. You should see output that indicates that both our
db containers, as configured in the
docker-compose.yml file, are booting up.
I was pleasantly surprised to discover that developing with Docker added very little overhead to the development process. In fact, most commands that you would run for Rails simply needed to be prepended with a
docker-compose run web.
|When you want to run:||With Docker Compose, you would run:|
Here are some nice development tricks I found useful when working with Docker:
- Add a
dockerhostentry to your
/etc/hostsfile so you can visit
dockerhostfrom your browser.
1 2 $ boot2docker ip 192.168.59.104
Then add the IP to your
1 192.168.59.104 dockerhost
Now you can pull up your app from
Debugging containers with
Sometimes you need to get inside a container to see what’s really happening. Perhaps you need to test whether a port is truly open, or verify that a process is truly running. This can be accomplished by grabbing the container ID with a
docker ps, then passing that ID into the
1 2 3 4 5 $ docker ps CONTAINER ID IMAGE 301fa6331388 myrailsapp_web:latest $ docker exec -it 301fa6331388 /bin/bash [email protected]:/myapp#
- Showing environment variables in a container with
docker-compose run web env
1 2 3 4 5 6 7 8 $ docker-compose run web env AWS_SECRET_KEY= MAX_THREADS=16 MIN_THREADS=4 AWS_REGION=us-west-2 BUNDLE_APP_CONFIG=/usr/local/bundle HOME=/root #...
Running an interactive debugger (like pry) in your Docker container
It takes a little extra work to get Docker to allow interactive terminal debugging with tools like
pry. Should you desire to start your web server with debugging capabilities, you will need to use the
--service-portsflag with the
1 $ docker-compose run --service-ports web
This works due to two internal implementations of
docker-compose runcreates a TTY session for your app to connect to, allowing interactive debugging. The default
docker-compose upcommand does not create a TTY session.
runcommand does not map ports to the Docker host by default. The
--service-portsdirective maps the container’s ports to the host’s ports, allowing you to visit the container from your web browser.
slimimages when possible on production
Oftentimes, your base image will come supplied with a
-slim variant on Docker Hub. This usually means that the image maintainer has supplied a trimmed-down version of the container for you to use with source code and build-time files stripped and removed. You can oftentimes shave a couple hundred megabytes off your resulting image – we did when we switched our
ruby image from
2.2.1-slim. This results in faster deployment times due to less network I/O from the registry to the deployment target.
Remember that your app runs in containers - so every time you do a
docker-compose run, remember that Compose is spinning up entirely new containers for your code but only if the containers are not up already, in which case they are linked to that (running) container.
This means that it’s possible that you’ve spun up multiple instances of your app without thinking about it - for example, you may have a
dbcontainer already up from a
docker-compose upcommand, and then in a separate terminal window you run a
docker-compose run web rails c. That spins up another
webcontainer to execute the command, but then links that container with the pre-launched
There is a small but noticeable performance penalty running through both the VirtualBox VM and docker. I’ve generally noticed waiting a few extra seconds when starting a Rails environment. My overall experience has been that the penalty has not been large enough to be painful.
Try it out
Give this a shot and let me know how Docker has been working for you. What have your experiences been? What are ways in which you’ve been able to get your Docker workflow smoother? Share in the comments below.
Coming up: integration with CI and deployment.
In upcoming blog posts, we will investigate how to use the power of Docker Compose to test and build your containers in a CI-powered workflow, push to Docker registries, and deploy to production. Stay tuned!