GitHub and Tracking a triggered Workflow run

Introduction

The worldwide growth and adoption of GitHub Actions has been undeniable. And no different to many others, we at Reecetech also merged with the masses by utilising this offering.

Aside from the commonplace with automation of application continuous integration and continuous delivery, we also facilitated manual triggering of some actions through employing the workflow dispatch event.

The workflow dispatch event is significant as it is also one of the only ways to trigger a workflow run externally of the GitHub platform.

Adoption of this is needed as we provide agility to our developers through interfacing a Slack App for repeated tasks/workflows.

The following is a diagram of a mock flow for a Slack App integration:

Little did we know that such a simple approach would come with some obstacles.

So what’s the problem?

Lets take a look at triggering the workflow dispatch event:

// Octokit.js
// https://github.com/octokit/core.js#readme
const octokit = new Octokit({
  auth: 'YOUR-TOKEN'
})

await octokit.request('POST /repos/{owner}/{repo}/actions/workflows/{workflow_file}/dispatches', {
  owner: 'OWNER',
  repo: 'REPO',
  workflow_file: 'some_fake_workflow.yml',
  ref: 'main',
  inputs: {
    name: 'person-service'
  }
})

When running such a command as above, it will:

• Trigger the workflow on GitHub ✅
• Provide 204 status code for success in triggering ✅
• Yield no content in response 🤨

So wait.

No id or information pertaining to the workflow we just triggered?

How do we track the workflow we triggered from outside of the platform?

What if we need an artifact/output generated from the Workflow?

Re-shifting our approach

Well if we can’t pull any information from triggering the workflow, can we push some detail to make the triggered workflow identifiable?

Looking into the GitHub Documentation on Creating a workflow dispatch event, the only room to provide identifiability to the workflow run are the workflow inputs.

With this approach we can make it capable to use the GitHub API’s to find the triggered workflow by providing some kind of identity for workflows called externally.

This is good, as you provide some data that will be unique and known by the actor and consumer.

From there, we shifted our implementation to the following approach:

I know it’s alot to take in, so we will break down the steps for better clarity.

Creating Workflow Dispatch ID Input

Lets provide an input to the workflow to make it identifiable:

--- a/.github/workflows/some_fake_workflow.yml
+++ b/.github/workflows/some_fake_workflow.yml
 ---
 name: Get List of App Versions 📄 📤

 on:
   workflow_dispatch:
     inputs:
+      dispatch_id:
+        description: 'A unique ID provided when dispatching this workflow'
+        required: false
+        type: string
       name:
         description: 'Name of application'
         required: true
         type: string

 jobs:
+  dispatch-id:
+    if: github.event_name == 'workflow_dispatch' && github.event.inputs.dispatch_id != ''
+    runs-on: [ ubuntu-latest ]
+    steps:
+    - id: dispatch-id
+      name: ${{ github.event.inputs.dispatch_id }}
+      run: echo "💁 The dispatch ID is ${{ github.event.inputs.dispatch_id }}"
   get-list-of-app-versions:
     name: Get App Versions 🏷

So now this extension upon the workflow will:

• Be able to consume a unique id
• Run a step named after the unique identity on a GitHub hosted runner

Awesome, so that covers the triggering of the workflow with an addition of some unique input as well.

Finding the triggered workflow

GitHub provide a neat API to List workflow runs for a repository. The API URL allows to provide a Query string with parameters. So lets look at an implementation with an appropriate degree of constraints and filtering:

// Octokit.js
// https://github.com/octokit/core.js#readme
const octokit = new Octokit({
  auth: 'YOUR-TOKEN'
})

let fiveMinsAgo = new Date();
fiveMinsAgo.setMinutes(fiveMinsAgo.getMinutes() - 5);

let response = await octokit.request(`GET /repos/{owner}/{repo}/actions/runs?created=>{time}&branch={branch}&event={trigger}`, {
    owner: 'OWNER',
    repo: 'REPO',
    time: fiveMinsAgo.toISOString(),
    branch: 'main',
    trigger: 'workflow_dispatch',
});
let runs = response['data']['workflow_runs'];

Breaking up the above implementation, it should read as:

1. Get workflow runs should be associated to the REPO repository owned by OWNER
2. The workflow run should have been created since 5 minutes ago
3. Furthermore, the run should have been triggered with the workflow_dispatch event on the main branch
4. Lastly, capture the array of workflow_runs into a variable

Awesome, so now we have a handful of workflow runs to assess 😌

Since we have an array that is low cost to iterate over, next step is to loop through the workflow runs and find our dispatch_id named step.

Looking into the workflow runs array, we have a key entry in the payload for jobs_url. The jobs_url allows you to get all workflow jobs and its details.

Here is a distilled example of what would be presented upon query of the jobs URL:

{
  "total_count": 2,
  "jobs": [
    {
      "workflow_name": "Get List of App Versions 📄 📤",
      "name": "dispatch-id",
      "steps": [
        {
          "name": "Set up job",
          "number": 1
        },
        {
          "name": "DISPATCH_ID",
          "number": 2
        },
        {
          "name": "Complete job",
          "number": 3,
        }
      ],
...

You should be able to see in the workflow run, the job named dispatch-id containing a step named by our DISPATCH_ID value

Now lets look at what an iteration over our workflow runs might look like:

let message, workflowId, workflowUrl;

if (runs) {
    // iterate over workflow runs
    for (let i in runs) {
        let workflowRun = runs[i];

        // fetch jobs for the workflow run
        let jobsUrl = new URL(workflowRun['jobs_url']);
        let response = await octokit.request(`GET ${jobsUrl.pathname}`);
        let jobs = response['data']['jobs'];

        // iterate over jobs within workflow run
        for (let j in jobs) {
            let job = jobs[j];
            let steps = job['steps'];

            if (steps.length > 1) {
                // if after setup phase is a step that contains our
                //  dispatch id then capture workflow details and break
                if (steps[1]['name'] == dispatchId) {
                    workflowId = job['run_id'];
                    workflowUrl = workflowRun['html_url'];
                    break;
                }
            }
        }

        // break iteration of workflow runs if workflow id found
        if (workflowId) {
            break;
        }
    }
}

We got it! 🙌

We retrieved what we wanted originally upon triggering the workflow.

With the workflow id available, we can utilise the GitHub API for further operations.

Await workflow completion

Without the need to show persistence, sleeps and timeout, lets look at an implementation to assess workflow completion through utilisation of Get a workflow run API:

let workflow = await octokit.request('GET /repos/{owner}/{repo}/actions/runs/{run_id}', {
    owner: 'OWNER',
    repo: 'REPO',
    run_id: workflowId
});

if (workflow.data.status === 'success') {
    // do work
}

Using our fetched workflow id, the resulting payload of this call will contain a field to easily assess completion.

Fetch Artifact (list of app versions)

The final step, in fetching data, is completely deterministic on the approach that the workflow is taking.

But for this hypothetical scenario, we can pretend that the app version data is uploaded against the workflow run as an artifact.

Artifacts uploaded against the workflow are stored in the form of a zip file.

Cool, so lets download the artifact from the workflow run using the List workflow run artifacts and Download artifact API’s:

// list workflow run artifacts
let artifacts = await octokit.request('GET /repos/{owner}/{repo}/actions/runs/{run_id}/artifacts', {
    owner: 'OWNER',
    repo: 'REPO',
    run_id: workflowId
});

// find artifact id
let artifactId;
for (let i = 0; i < artifacts.data.total_count; i++){
    if(artifacts.data.artifacts[i].name === 'some-expected-artifact-name'){
        artifactId = artifacts.data.artifacts[i].id;
    }
}

// download the artifact
let downloadedArtifact = await octokit.request('GET /repos/{owner}/{repo}/actions/artifacts/{artifact_id}/{archive_format}', {
    owner: 'OWNER',
    repo: 'REPO',
    artifact_id: artifactId,
    archive_format: 'zip'
});

// get file data
let unzippedData = UZip.parse(downloadedArtifact.data)
const textDecoder = new TextDecoder();
const decodedFile = textDecoder.decode(unzippedData['app-versions.txt']);

YAY! 🙌

So there are some things here worth taking note for the above example:

• After retrieving the list of artifacts against the workflow run, we are checking if the artifact name is consistent and known in the find artifact id phase (some-expected-artifact-name)
• After downloading the zip artifact, we use UZIP.js for parsing the zipped file
• Then the underlying filename of “App Versions” within the compressed archive is known and expected (app-versions.txt) and decoded into plaintext.

With all this set out, depending on how the slack bot is managed, all the intermediary pieces to trigger and fetch data from a GitHub workflow is completed 😌.

Some further learnings and thoughts

Even though the GitHub REST API is well documented and easily implemented, I feel there may be some missing features.

This mock example of fetching app versions similarly mimics some of the workings we had to deal with in fetching plain text outputs from a workflow.

Utilising workflow Step Summaries is an awesome feature to display information for the user on the GitHub platform. This leaves me wishing that there was a GitHub API to fetch the raw markdown (which is accessible in the GUI).

Even though it may be derivable, we stayed true to the GitHub provided way.