Renovate Bot at reecetech

Renovate Bot at reecetech

Renovate

What is Renovate?

Renovate is a software robot. It’s an open-source software program that is intended to keep your software dependencies up-to-date. It does this by: scanning your software repositories for files that specify dependencies; checking if any of the dependencies have updates available; and then raising pull requests to use updated dependencies if they are available.

Software dependencies are the other items of software that your software requires to build, be tested, or to run.

In software development, there is a desire for builds and testing to be repeatable - and thus “locking” or “pinning” the versions of dependencies is a way to achieve this repeatability. For example, if we develop against dependency version 3.2.0, and everything is working fine, then we will “pin” this dependency at version 3.2.0. This “pinned” version will be used for all future builds and tests. The drawback is that we are no longer keeping up-to-date with new versions of the dependency. New versions might have security flaws or bugs fixed, or a new version might have improvements in processing efficiency that we’re missing out on.

Having Renovate helps with this situation by identifying dependencies with updates available, and suggesting that we update the “pin” to the new version.

Refreshing

I work in the delivery engineering team at reecetech, and so I tried Renovate on our teams’ software projects first. After “onboarding” each software project to Renovate, we discovered there was heaps of clean up to do!

Some of our projects had last had their dependencies updated in 2019. In one project, Renovate created 73 different pull requests.

We discovered around 30 projects out of 120 projects that could be immediately retired, as they were no longer in use.

Other projects didn’t have adequate test coverage for us to be confident to merge the pull requests that Renovate created.

Working through these things has been refreshing for us, as we’ve visited some projects that we’d left alone for too. It’s a bit like a Spring clean for our software projects. We’ve cleared away the cobwebs and deployed brand new builds of our software.

We can recommend using Renovate to keep your dependencies up-to-date. It’s a perfect use-case for a bot. Keeping dependencies up-to-date on a schedule is tedious repetitive work, that’s easily put-off by humans more interested in interesting problems!

It encourages us to have valid and complete testing for each of our applications so that we can merge Renovate’s pull requests with confidence.

It’s an automatic pay-down of some of your technical debt.

Running Renovate

Open-source and self-hosted

reecetech’s source code repository is self-hosted Bitbucket. For us, it was a straight forward choice to try self-hosting the open-source version of Renovate. If you are using Github or Gitlab (in the cloud) then it would be worth investigating the SaaS offerings from White Source Software.

Running Renovate is a matter of invoking it on a schedule. At the time of writing, we invoke it daily at 12:30am. Given that each repository can define a renovate.json configuration that can set specific schedules dictating when Renovate is allowed to scan that repository, then we may have to revisit the daily schedule and run more frequently - else we could find that some repositories never get scanned if the schedule does not permit pull requests at 12:30am.

Whilst Renovate can run in Kubernetes, we have decided to run it as a scheduled build plan in our CI service. The advantages for us in doing this are: - the CI service is expected to access Bitbucket, whereas Kubernetes is not - easy to review the logs of a specific execution of Renovate - existing pattern for triggering build plans from our engineering chat-bot - the author of the blog post & implementor of Renovate was more familiar with the CI service than with Kubernetes

Custom Docker image

We extend the publicly available Renovate ‘slim’ Docker image (renovate:19-slim) to include:

Our Dockerfile looks like this:

FROM renovate/renovate:19-slim AS base

LABEL maintainer="Delivery Engineering"
ENV TZ "Australia/Melbourne"

ADD version-handler.sh /opt/renovate/
ADD wrapper.sh /opt/renovate/
ADD config.js /opt/renovate/

# Override to false in production
ENV RENOVATE_DRY_RUN "true"
ENV RENOVATE_CONFIG_FILE "/opt/renovate/config.js"

ENTRYPOINT '/opt/renovate/wrapper.sh'

The RENOVATE_DRY_RUN environment variable defaults to “true” in our Docker image, making it a bit safer to develop on. When the CI service invokes the image as a container, it overrides RENOVATE_DRY_RUN to be “false”.

The wrapper script adds some sanity checking, which should help other people start developing on our Renovate solution. It looks like this, and prevents someone from trying to run Renovate without the prerequisite environment variables, or without the Docker socket mounted and writable:

#!/bin/bash
set -eo pipefail

function check_envvar {
    if [[ -z "${!1}" ]] ; then
        echo "ERROR: ${1} is not set"
        exit 1
    fi
}

function check_dockersock {
    if [[ ! -S /var/run/docker.sock ]] ; then
        echo "ERROR: /var/run/docker.sock is not mounted (or is not a socket?!)"
        exit 2
    fi
    if [[ ! -r /var/run/docker.sock ]] ; then
        echo "ERROR: /var/run/docker.sock is not readable by user: $(id)"
        exit 3
    fi
    if [[ ! -w /var/run/docker.sock ]] ; then
        echo "ERROR: /var/run/docker.sock is not writable by user: $(id)"
        exit 4
    fi
}

check_envvar GITHUB_COM_TOKEN
check_envvar RENOVATE_USERNAME
check_envvar RENOVATE_PASSWORD
check_envvar RENOVATE_AUTODISCOVER_FILTER
check_envvar RENOVATE_BASE_DIR

check_dockersock

node /usr/src/app/dist/renovate.js

Running the Docker image

Running the image was the most tricky bit due to file-system mounts and non-root UID’s.

It doesn’t have to be that way, but during the implementation of Renovate at reecetech, we discovered the more tricky approach worked for us.

Why are we doing it “the hard way”? Because:

  1. The default UID and GID used by the public image didn’t map exactly to the UID’s and GID’s that our CI service uses
  2. Running containers as a non-root user is a good security practice
  3. Using the ‘slim’ image has advantages over the full image with Python projects
  4. Using the ‘slim’ image requires the use of ‘helper’ containers, and hence the Docker socket is required to be mounted into the main runtime container
  5. Mounting the Docker socket into the containers also required correct UID and GID mapping to have permission to use the socket

Here’s how to get Renovate ‘slim’ to run as non-root, with the ability to run ‘helper’ containers (on Linux, not Mac):

# make a new directory, owned by the current user, set as an environment variable
mkdir -p renovate_base
export RENOVATE_BASE_DIR="$(pwd)/renovate_base

# discover the current user's UID, and the docker group's GID, set as an environment variable
export RENOVATE_DOCKER_USER="$(id -u):$(getent group docker | cut -d ':' -f 3)"

# run the container
docker run --rm \
  --user "${RENOVATE_DOCKER_USER}" \  # the main container runs as this UID:GID
  --env 'RENOVATE_DOCKER_USER' \      # tells Renovate to use this UID:GID for helper containers
  --volume '/var/run/docker.sock:/var/run/docker.sock' \
  --volume "${RENOVATE_BASE_DIR}:${RENOVATE_BASE_DIR}" \
  --env 'RENOVATE_BASE_DIR' \
  ...<more environment variables as needed>...
  renovate/renovate:19-slim

Advantage of the ‘slim’ image

The Renovate ‘slim’ image defaults the configuration item ‘binarySource’ to docker. This means that Renovate can pull additional ‘helper’ containers to assist it when evaluating different software languages and versions.

For example, since April 2020, Renovate has been able to dynamically select different images corresponding to the required Python version needed to evaluate dependencies in Piplock files. This justified the extra effort in doing the UID/GID, Docker socket and filesystem mappings.

Bot, run my bot

At reecetech we have another bot in our ecosystem: Helmet, the engineering chatbot. Helmet is worth an entire blog post on its’ own. But for now, the short description is that Helmet has its’ own Slack channel, and be asked to perform engineering tasks for you, such as installing an application into our Kubernetes cluster.

After implementing Renovate, we realised that pull requests can quickly become conflicted, since they are applying to the same dependency management files (package-lock.json, Pipfile.lock, etc). Merging a series of Renovate pull requests requires multiple executions of Renovate to rebase the yet-unmerged pull requests. Remember that at the time of writing, Renovate is only scheduled to run once a day.

We needed an easy way to trigger an execution of Renovate.

That’s where Helmet comes in. We added a new command to Helmet, allowing Helmet to trigger Renovate.
Helmet also requires you to tell it the exact repository that you would like Renovate to scan. This keeps the scan time down (1 repository, instead of 1,800), providing quick feedback to the invoking user.

To invoke a specific repository scan, on-demand, with Helmet:

Helmet, please renovate DE/seadragon

Behind the scenes, Helmet triggers a build of the Renovate build plan and provides a custom variable specifying the repository.

Even this, however, is a bit too manual. The Github and Gitlab installations supported by White Source Software have a webhook listener system that uses repository events to know when to trigger a rescan of a repository. We might be able to build a similar system that takes repository events and trigger our CI service to run Renovate.

In conclusion

I really like automating tedious activities. Breaking builds if a security scan finds vulnerabilities. Using --fix with eslint to automatically fix my linting errors in Javascript. Deploying to Kubernetes immediately after a green build.

Renovate fits perfectly into this thinking. Keeping up-to-date no longer requires us to think “are we due for doing some maintenance and patching?“. Now we open up Bitbucket and find the pull requests waiting for our approval.