Docker for Developers Part 1

Summary

 

Recently I started working on a few projects where docker seemed like a great fit to rapidly speed up development of the project. In one case we wanted to build a prototype service that contained an API endpoint that utilized 4 micro services. The docker landscape is still green with a lot of toolsets not even existing for a year. While I feel the development side of things is great, the production deployment, auto scaling, and release management is still lacking. One of the projects I have been following closely is Rancher which seems to be on track to solve all of these things. This will be a series of posts initially focusing on development and the building of a fully featured sample application demonstrating the power of docker running locally. I will add posts documenting CI with Jenkins, through to a production deploy and management on AWS.

What will we do?

This tutorial is going to walk through the creation of a sample web application that utilizes a sample API service backed by dynamodb. Specifically we will:

  1. Layout the structure of our application and explain why things are laid out like they are.
  2. Build a sample hello world flask app that shows the initial power of docker and docker-compose.
  3. Run a local dynamodb container locally for development.
  4. Load some data into that local dynamodb install.
  5. Build a sample flask API that reads from that local dynamodb instance.
  6. Build a sample flask website that reads from the API and returns some basic data.

Setup

1. So to dive right into it, you will need two things installed on your machine to work through this tutorial. Currently everything below assumes you are running on OSX however it should work just fine under Linux as well.

  • Install Virtualbox: https://www.virtualbox.org/wiki/Downloads
  • Install the docker toolbox: https://docs.docker.com/installation/mac/

2.  Assuming that you have never used boot2docker before (and if you did, you should be prompted with instructions on how to convert to docker machine), run the following command to setup a default docker machine. This will be a virtual machine that will be where all the various containers run that you launch. More on that in a minute.

3. You can now run $ eval “$(docker-machine env default)”  to set the required environment variables. If you will be launching docker containers often, you might even elect to put this in your bashrc or zshrc file.

4. You should be able to run docker ps and see the following:

 

Helpful Commands

This area serves as a quick reference for various commands that may be helpful as a developer working with Docker. A lot of these may not make sense just yet, and thats fine. You will learn more about them below and can always come back to this one spot for reference.

 

Building our application

So now that the tooling is setup we can discuss what our project structure will look like.  You can see a completely functional and done copy of the below project on github and you can grab just the files we create here in part 1.

Initial Skeleton

  1. First create a directory called docker-movie-db-demo somewhere.
  2. Within that directory create two directories. One called movie-api and the other movie-web

It should look like this

We created two directories however, that are going to house two separate applications. The first of which, movie-api is going to be a simple flask API server that is backed by data in dynamodb. The second application called movie-www is going to have a simple web interface with a form, and allow the user to list movies from a certain year.

Our first Flask App

Within movie-api go ahead and create a few empty files and directories so that your structure matches the below. We will go through and touch these files one by one.

app.py

Open app.py up and lets toss a few lines in.

This is pretty basic, but it will initialize a basic flask app from demo/services/api.py and listen on a port that is specified as the first argument when running python app.py 5000.

requirements.txt

Open requirements.txt and add in the following

This is also pretty straightforward, but we are ensuring we install flask. Boto is installed for interfacing with dynamodb. I’ll have to write a separate article on why its important to pin versions and the headaches that can solve down the line.

demo/services/api.py

For now lets just add in the following:

We are adding a simple route for the index page of the api service that for now just returns the text “Hello World!”

Dockerfile

The dockerfile is where the magic happens. Depending on how you are used to doing development, you might create a virtual env somewhere, or possible a vagrant image. This certainly works however you often end up with a bunch of files scattered everywhere, mismatches between your virtual env and someone elses (if you arent careful) and/or multiple vagrant images floating around that slow down your machine.

Open the Dockerfile (Note that the Dockerfile should have a capital D) up and paste in the following:

When running the build command, this tells the system how to build an image. In this case it will use a python 2.7 base image, copy the CWD (movie-api) to /code in the container, set the CWD to /code, run an apt-get update, pip install our requirements and then finally run our application. If you want more details you can read the dockerfile reference here which explains whats going on in detail and whats possible.

At this point we have enough of a skeleton to actually build a container, run it if we wanted to.

We just built an image, spun up a container based off that image, queried the service and got a response, stopped the service, and deleted the container. However, every time you make a code change your going to have to rerun the build command, and then relaunch your container. If your doing quick iterative development this can get annoying quickly. There is a better way.

 

 Introducing docker-compose

The docker-compose files are how we orchestrate the the building and running of our containers in an easier fashion. We can leverage how these files work with flasks built in reload system on file change to enable rapid iterative development

docker-compose.override.yml is special. When you run the docker-compose command, it will look for docker-compose.yml and docker-compose.override.yml. If present, it will go ahead and merge them and then perform actions based on that merged data.

We leverage this behavior to build our development environment. If we wanted a test environment for example, we would add a docker-compose.test.yml file, and when running docker-compose target that environment with docker-compose -f docker-compose.yml -f docker-compose.test.yml. However, this is generally only done by build systems, and so we use the override file for development as it keeps the command for developers simpler as they don’t need to specify -f.

docker-compose.yml

Within the root of our project directory (docker-movie-db-demo) lets create a file called docker-compose.yml and make it look like so:

We have just defined a movie-api service that for now has no image, will always restart on failure, and exposes the container port 5000. You can see the full docker-compose file reference here

As mentioned above, the override file will allow us to override some things in the movie-api base compose definition to make development a little bit faster.

Create and edit a file called docker-compose.override.yml and make it look like so:

If you remember, in our dockerfile, we copy up the movie-api files into the image during its build. This is great when you want to make a container that you start shipping around to various environments such as test, staging, and production. However when you just want to do local development, building that same container every time is time consuming and annoying. With our override file, we have made it so that we will mount our code base within the running container. This allows us to use our favorite IDE locally to do development but immediately see those changes reflected. We also have exposed port 5000 on the container and mapped that to port 8080 on our docker-machine. This makes it a little bit easier to debug and test. In production you generally wouldn’t do this and i’ll detail more in a seperate article focusing around production deployment of this workflow.

Starting our movie-api app.

Now, from the root of our project directory (docker-movie-db-demo) run the following command:

 

You can tail the logs of the container by running:

So already you can see that starting up the container is simpler. However things really shine when you start editing code. Fire vim or your favorite IDE and edit movie-api/demo/services/api.py:

If you kept tailing the logs you will see that it instantly reloaded, and if you run curl 192.168.99.100:8080 again you will see that the output changes.

docker-movie-db-demo-1

Wrapping it up

This concludes part 1 of the tutorial. In summary we laid out the structure for our project, went through how to setup a machine for docker, built a sample flask application that returns a hello world message. We also walked through how to make changes to the application and test those changes in realtime without having to build an image and redeploy the container.

In the next post, i’ll focus on adding a local dynamodb instance, how to run one off data scripts to do things like load data and build a sample web interface that interacts with the API.

Part 2 Here

Leave a Reply

Your email address will not be published. Required fields are marked *