Services in GitHub Actions

While the features of GitHub Actions have existed on various platforms - Travis CI, AppVeyor and more recently Azure DevOps - the overall lack of billing and user management friction combined with the ability to run workflows on Linux, macOS and Windows make Actions very compelling.

I’ve spent the last couple of months converting various projects to Actions from a variety of other platforms, and thought I would write a series of posts covering lesser-used features of the platform, and some of the actions I’ve needed to build along the way.

Services

It’s quite often necessary to manage external applications when running the integration tests of a project. For example, it might be necessary to talk to a database, or to Vault. GitHub Actions allows this via services.

Services are configured as part of a job, and allow running a container which is accessible via either localhost or a container network to all of the steps which are part of that job. Various aspects of the service container can be configured via the actions YAML manifest, and there is a break-glass option to directly provide flags which form part of the command line to create the container.

Example - Consul

As an example, let’s look at a simple workflow which uses Consul as a service, and “integration tests” a bash script which uses the Consul API to set a key in the key-value store, then verify the key was set correctly. The actual code does not affect the service configuration - we just need something to test that the service is up and running.

To run Consul using the official Docker container, we can add the following YAML to our actions manifest:

services:
  consul:
    image: consul:1.6.2
    env:
      CONSUL_BIND_INTERFACE: eth0
    ports:
      - 8300-8302:8300-8302/tcp
      - 8301-8302:8301-8302/udp
      - 8500:8500/tcp
      - 8600:8600/tcp
      - 8600:8600/udp

With that, we specify:

• The image name and version - we’re pinning to v1.6.2 of Consul but could use a tag such as latest, or indeed a build matrix variable to test against multiple versions of Consul,

• An interface binding of eth0 as per the documentation for the image,

• To publish the various ports that Consul uses, such that other tasks running on the virtual machine can access the Consul APIs via localhost:<port number>.

The full list of parameters is available in the GitHub Actions YAML Manifest documentation.

The Actions runner will use the Docker health check to determine when it is safe to proceed to the build steps in the job. If the image does not have a health check configured, you can use the options field in the service definition to specify a health check at the time the container is created. If no health check is specified, the steps will proceed regardless of whether the service is ready to accept requests, and may cause failures in integration tests if this is not taken into account.

Once our service is running, we can find the Consul API at http://localhost:8500/, as we’d expect. Our bash script just uses cURL with that address, as shown in this modified snippet:

CONSUL_URL="localhost:8500"
CONSUL_KEY="test-key"
VALUE="Hello World"

curl --silent \
    --show-error \
    --request PUT \
    --data "${VALUE}" \
    "http://${CONSUL_URL}/v1/kv/${CONSUL_KEY}"

If we run this in GitHub Actions, we get a new build step after the job setup - “Initialize Containers”. This creates our service, and it runs for the duration of the job:

Container Actions

The construction in the simple example above works great if the tasks using the service are running directly on the virtual machine hosting the build job. However, if build steps are running in a container, we need to use container networking instead, and access our service on the hostname formed of the service name key in the YAML manifest.

Jobs in GitHub Actions allow specification of a container inside which all actions which are not themselves based on a container should run. For example, we can add the following YAML to our integration-tests job to make our integration tests run in a container created from an Alpine image which contains the tools we need:

# This is not an image tht I'd consider using in production but works for the purposes
# of this demo since it has everything pre-installed and is very small!
container:
  image: cfmanteiga/alpine-bash-curl-jq

When we run our integration tests inside a container, we must access the service at http://consul:8500 rather than http://localhost:8500.

A good way to do this is to source the address of the service from an environment variable, default it to the localhost variant, and set the variable for task steps which run inside a container:

container:
  image: cfmanteiga/alpine-bash-curl-jq

...

steps:
  ...
  - name: Run Integration Tests
    shell: bash
    run: ./key-value-tests.sh
    # Use the container address
    env:
      CONSUL_URL: consul:8500

In our script, we use the value of CONSUL_URL if it is specified in the environment:

CONSUL_URL=${CONSUL_URL:-"localhost:8500"}

Limitations

Unfortunately, things are not quite as bright as they seem! Services only work on jobs running on Linux, because the runner which sets them up uses Docker commands which are not supported on Windows.

Furthermore, there is no documented way to pass either a command or arguments to docker create, so container images which cannot be entirely configured via environment variables are not usable.

An example of this is HashiCorp’s Vault image, which requires the -dev flag be passed on the command line to start in the development mode most useful for integration testing.

Both of these seem easily fixed, but it does not appear that the service runner is open source to make pull requests, though I’d love to wrong about that!

Code

The full code for both script and action manifest for this example is available here.