Moving from GitLab pipeline to GitHub Actions for CI/CD

I've been using my self-hosted GitLab instance for my git repos and pipelines for 5 or so years now, but things have changed and I've decided I no longer want to run my self-hosted instance any more. It's purely for hobby/self projects and only accessed by myself, however, one bonus of this is that I also self-host a dockerised GitLab runner that performed all of my CI/CD duties, including the deployment stage, which is the important part. As this was self-hosted and ran on a separate droplet within my VPC on DigitalOcean, connections to to my web droplet was done via an internal IP address across an SSH connection (SSH is blocked to all but my home VPN IP addresses). I didn't want to open up the SSH port to world-and-dog.

Although I'm not running a massive amount of pipeline tasks, GitLab's free tier only offers 400 minutes per month for running CI/CD pipelines. Comparing this to GitHub which offers 2,000 minutes per month, as I'm not doing this as a business, I've decided to move over to GitHub.

Migrating the repos over to GitHub was an easy task as GitLab provides repository mirroring, so it was a case of creating an access token in GitHub, providing that with the mirror target and clicking a button. I mirrored 128 repos in an evening.

The next stage was trying to figure out how I was going to use GitHub Actions for my CI/CD pipeline without granting SSH access to my server from all possible GitHub IP addresses (which wouldn't be the best idea anyway). GitHub has the functionality to send data on various events to an endpoint. All I needed to do was to code something that would provide an endpoint and perform an action.. but what if there was already something out there to do just that? That's when I discovered Adnan Hajdarević's webhook project. It's a single binary compiled from a Go project. It also provides examples in the repo for various git hosting providers to use as a starting point.

After reading the info, it didn't take long to get it up and running on my local box for testing scenarios. Once I had the tests all worked out, it was time to move this over to my production server and run it there.

The process I was looking to achieve

• Commit code changes to GitHub
• Have GitHub Action:
  • Run the test suite
  • Build the docker image for the project
  • Push the built docker image to Docker Hub
  • Send the git payload to the webhook server
• Webhook server should then:
  • Pull the latest docker image from Docker Hub
  • Stop the currentlty running container
  • Start the newly deployed container
  • Run any database migrations if required (most of my projects are Laravel-based)
  • Rebuild Laravel-based project caches (route, config, etc) as these will be cached in the Redis server that's running

With the websocket server running (I bound it to the local IP address and put it behind the nginx server so I could easily take advantage of TLS), I added the following hook for the first test site:

[
  {
    "id": "deploy-thesite",
    "execute-command": "/etc/webhook/scripts/deploy-thesite.sh",
    "command-working-directory": "/etc/webhook",
    "response-message": "Deploying TheSite",
    "trigger-rule":
    {
      "and":
      [
        {
          "match":
          {
            "type": "payload-hmac-sha256",
            "secret": "<TOKEN>",
            "parameter":
            {
              "source": "header",
              "name": "X-Hub-Signature-256"
            }
          }
        },
        {
          "match":
          {
            "type": "value",
            "value": "refs/heads/main",
            "parameter":
            {
              "source": "payload",
              "name": "ref"
            }
          }
        }
      ]
    }
  }
]

The id here is important. This is the endpoint you need to add in GitHub. The websocket URL for this site is:

https://my-deployment-server/hooks/deploy-thesite

The execute-command entry can call any script in any location you wish. It just needs to be an executable script/binary.

The response-message is returned when the endpoint is hit (though largely immaterial for this scenario).

The trigger-rule uses the and matching pattern, meaning that all rules need to be met.

Firstly, we have the header value with the token assigned. This will be set in the call from the GitHub Action. The second match is a value and checks the payload sent from GitHub (JSON data) and looks for the ref key in root of the payload. It then checks to see if it's equal to refs/heads/main. The idea here is that the deployment will only be triggered when a commit/push has been made to the main branch.

If everything here has gone well and matched, it will call the deploy-thesite.sh script, which looks something like:

#!/bin/bash

DOCKER_IMAGE="mydockerusername/thesite:latest"
CONTAINER_NAME="thesite"
CONTAINER_PORT=9999
RUN_MIGRATIONS=

## Check for run migrations in commit message
if [ -z ${1+x} ]; then
        echo "No commit message available."
else
        RUN_MIGRATIONS=$(echo "$1" | awk -v RUN_MIGRATIONS=$f '/\[RUN MIGRATIONS\]/{f=1} { print f }')
fi

## Pull latest image
docker pull $DOCKER_IMAGE

## Stop current containers
/usr/local/bin/stop-docker-container.sh $CONTAINER_NAME

## Create network
docker network create thesite

## Run new container
docker run --rm -d \
        --name $CONTAINER_NAME \
        -p $CONTAINER_PORT:80 \
        --mount source=thesite-storage,target=/application/src/storage \
        --mount source=thesite-nginx-logs,target=/var/log/nginx \
        --network thesite \
        $DOCKER_IMAGE

## Run migrations if required
if [[ $RUN_MIGRATIONS == 1 ]]; then
        echo "Running migrations..."
        docker exec -w /application/src $CONTAINER_NAME php artisan migrate --force
fi

## Rebuild caches
docker exec -w /application/src $CONTAINER_NAME php artisan route:cache
docker exec -w /application/src $CONTAINER_NAME php artisan view:cache
docker exec -w /application/src $CONTAINER_NAME php artisan config:cache

This script first checks for [RUN MIGRATIONS] in the commit message to determine whether migrations should be run or not (not every commit will require migrations to run).

Next, it pulls the latest image from Docker Hub.

Then, it calls a basic script to stop a container via name (essentially run docker stop <the-container-name>).

It will the attempt to create a new docker network (this project also includes a MeiliSearch search container and uses a docker network to communicate between the two containers).

Next up, it will run the latest image for the project.

The script will then run migrations if required.

Lastly, it will rebuild the Laravel caches.

GitHub Action step

To trigger all of this off, I added a new step to call the webhook server with the git commit payload:

- name: Workflow Webhook Action
  uses: distributhor/workflow-webhook@v2.0.4
  env:
    webhook_url: ${{ secrets.WEBHOOK_DEPLOY_URL }}
    webhook_secret: ${{ secrets.WEBHOOK_DEPLOY_SECRET }}

The WEBHOOK_DEPLOY_URL is the endpoint mentioned earlier and the WEBHOOK_DEPLOY_SECRET is the token that needs to be matched in the hooks.json configuration for this project.