The last few days I spend by creating Docker images and improve their automated testing. I was using Jenkins as a self-hosted solution a few month ago but as it is written in Java and has an ugly UI,¹ I wasn’t really in love with it.

I killed my Jenkins instance after a few weeks. It used tons of RAM on a server which was not only for CI purposes and there were way more interesting projects to host.

So I came to TravisCI a wonderful hosted solution which provides automated software testing for free and has a perfect integration with GitHub.

Reasons for TravisCI

TravisCI is useful in various ways. First of all, it is free. That’s wonderful and makes it easy for everyone to use it.

It provides a modern UI which is important to me, too. It’s very intuitive which helps to get started quickly.

And there is useful documentation, which is one of the most important reasons to use TravisCI. The documentation containing examples and useful explanations. So even if you don’t know what to do next, check the documentation and you’ll find help. Sometimes it is useful to search it using the site:docs.travis-ci.com feature in your favourite search engine because not everything is accessible directly using the sidebar menu, but you’ll find useful entries for nearly every problem.

And last but not least it is super easy to integrate by adding a .travis.yml to your repository and hand over all tests to any fork of the repository. That results in great code quality for everyone who want to care about it without the need to setup a gigantic self-hosted test environment.

Testing Docker container

If you write software you often think about unit tests and tests deep inside your application. Those tests are super important and you should write them for a good quality of code. But in case you provide a Docker container you should test it, too. As your code in the container should run like the code in your original build you should not redo all unit tests. There is no real benefit by doing this but takes ages to finish.

Testing Docker container is mostly a basic set of tests. First of all check that your container is able to build. To do this simply use the docker build command.

After building your container you should also check that the container is able to run. Guess what the command to use for this simple test is docker run -d.

Also check that your container is running for a while without dying. For this, you use a sleep timer for at least 30 seconds (default delay for an HEALTHCHECK) and then check that your container is still listed in docker ps

A last basic check is that all needed ports are open. For this you can run nc using the following line:

nc 127.0.0.1 12345 < /dev/null || exit 1

So you have a little set of really basic tests. But you should extend them with more specific tests for your application like performing rest calls or similar.

Shorten build time by parallelizing your tests

By default TravisCI does not allow you to run different tests parallelized. But there is a very handy solution for it. The answer is write your tests as shell scripts and use the GNU tool parallel. I really love this tool. It can replace most for-loops in your shell scripts and run the tasks side by side.

In case you want to run your tests in parallel you should create a directory. Mine is called tests/. There you place your test scripts.

Make sure you place set -e at the head of your script. This makes sure the script fails if an error appears while running it.

To make your scripting more flexible use an environment variable to store the container ID of your started container:

DOCKERCONTAINER=$(docker run -d <yourcontainername>)

This is perfect for run tests against your container and makes sure that you are not matching another container running beside yours.

A basic example script for an image test can look like this:

#!/bin/sh
echo "
         ###          Example test           ##
"

# Make sure tests fails if a command ends without 0
set -e

# Generate random port for testing
CLIENT_PORT=$(cat /dev/urandom|od -N2 -An -i|awk -v f=10000 -v r=19999 '{printf "%i\n", f + r * $1 / 65536}')

# Make sure the port is not already in use. In case it is, rerun the script to get a new port.
[ $(netstat -an | grep LISTEN | grep :$CLIENT_PORT | wc -l) -eq 0 ] || { ./$0 && exit 0 || exit 1; }

# Run container in a simple way
DOCKERCONTAINER=$(docker run -d -p 127.0.0.1:${CLIENT_PORT}:12345  image:testing)
sleep 5
# Make sure port is open
nc localhost ${TLS_CLIENT_PORT} < /dev/null || exit 1

# Make sure the container is not restarting or dying
sleep 40
docker ps -f id=${DOCKERCONTAINER}

# Clean up
docker stop ${DOCKERCONTAINER} && docker rm -fv ${DOCKERCONTAINER}

For a few more tests you can checkout: https://github.com/Adam-/inspircd-docker/tree/master/tests

After you finished writing your first test you should add a way to run it using your .travis.yml. Add the following line to it:

script:
- ls tests/*.sh | parallel

ls is used to creates a list of all .sh-files in tests/. The list is piped as argument to parallel, which now runs them parallelized. But the order of the output stays the same as it would be if you run them in a for-loop. That’s one of the most powerful features of parallel.

In case some tests fail, the exit-code will show you the number of failed jobs.²

Stay up-to-date with your Docker images

Docker builds are very powerful and provide great portability for your applications. But it has the weakness that it works like a static bound binary. So you have to make sure all your dependencies stay up-to-date. In frequently updated images that’s not a problem, but in case you are not developing your application rapidly or have long release circles it should rebuild to include latest versions of libraries, patches and security updates.

You can easily do this using the TravisCI’s cron feature. Travis will rebuild your branch every day/week/month³ and run all tests.^{4 5}

To trigger automated builds on Docker Hub you can use their trigger feature. Create one and copy its URL.

Now run travis encrypt DOCKER_PUSH_URL=<place the url here> --add in your repository.^{6 7} travis creates an encrypted environment variable you can use for the next command.

With the encrypted version of the URL you can add the following statement to your .travis.yml:

after_success:
  # Run build on Docker Hub
  - '[ "$TRAVIS_EVENT_TYPE" = "cron" ] &&  [ "$TRAVIS_PULL_REQUEST" = "false" ] && [ "$TRAVIS_BRANCH" = "master" ] &&  curl --data build=true -X POST $DOCKER_PUSH_URL'

This will automatically trigger a build on Docker Hub but only if the CI job was triggered by cron, the build is not a Pull Request and it is your master branch. You can also pass some of this information to the HTTP call to build more specific branches and versions.

The more detailed examples are available, where you create the Docker Hub triggers in the “build settings” section of your Docker Hub repository.⁸

Conclusion

As you can see it’s easily possible to improve your build chain on TravisCI a lot. You can also easily stay up-to-date with your Docker images and get informed in case your setup becomes outdated or breaks.

And all this costs you $0, is super easy to setup and allows your collaborators to work with the code and create wonderful open source software. So you can focus on the quality of your code and documentation.

Is that nice? I think so!

I hope you learned something and implement it soon in your projects.

Stay in touch by following me on Mastodon and please support me to continue writing projects and blog posts. Share it with your friends and co-workers using the buttons below and don’t miss to leave a comment if you have questions or ideas for improvements.