Pros and Cons of runners on GitHub Actions

In the previous article I skimmed the surface on Github actions, and wrote briefly about the limitations the macOS runner. Now I want to dive deeper into the macOS runner, and the pros and cons of using the in-built image provided by Github.

Positives to Github runners

Some of these are fairly simple and easy to understand and don't concern macOS runners. What I think you'll come to realise the more you delve into using actions is that there is a large cross-section of use cases in other OS runners.

What you should be thinking about when setting these up is, what can I abstract away into a Linux runner, and what specifically needs to run on a Windows or macOS runner?

Compatibility

One of the main positives of using the macOS runner is the compatibility it has with other languages and frameworks like I mentioned in the last article.

This means you can run different scripts or actions against your code base within the same ecosystem. But what it also means is everything you normally would do on your own macOS system, you can run in the runner.

Specific apps

There are some tools that are Mac only - particularly Xcode. Since it is locked to the macOs system, the only way to run it - and as a result compile your code - is to use a macOS OS.

If you want to use Github actions to build your iOS or macOS apps, then push it to Test Flight you are 100% locked into using a macOS runner. You might think this is a trivial issue, but later on I hope to explain the drawbacks of this.

Global checks

If you have linters or checklists you need to run before merging code, using the macOS runner is a great way to maintain consistency.

In Github, if you are the owner of the repo, you can set up rules for certain actions needing to be passed before merging.

This means if you want to enforce a SwiftLint rule - you can. Or if you want to ensure that there is localisation. Or if you want to run a spellchecker over all the Strings - it is possible.

D.R.Y. - don't repeat yourself

One of the fundamental "rules" you learn when writing code is to not repeat yourself (or DRY). Though this rule can cause issues for bloating functions, or leading to over-engineering your code, I think applying it to your side-scripts is important.

Every time I build my static sites, I run them through a series of checks and balances:

• markdown linter
• spellchecker
• broken link checker
• malicious link checker
• style sheet compressor

I wrote myself a bash script to help me with all this to make sure that I don't have to run each command individually. It's something similar to a package.json script section, where you can chain commands into a single operation.

If I were to run these individually I'm sure I'd forget one or more of the steps. Even before using Github actions, I'd sometimes forget to run the bash script, meaning I'd have to re-push and hope that my previous commit didn't break the live site.

Chaining

On the topic of forgetting to run a script and affecting the live site as well as many checkers for the code - chaining actions.

One of the greatest and powerful features of Github actions is being able to chain actions together.

For example, you can have your first action lint all your Swift code. If that fails, the actions don't continue. But if it passes then it can go on to the next step.

This is great for ensuring that the entire workflow you want is achieved correctly. No more bypassing steps by contributors and having to clean the mess.

Better PR comments

One of the annoying parts of having open source software and repos is you are now having to manage all the contributions - be it the good, the bad, and the ugly.

Github does allow templating for your PRs and even issues, which is great since you can guide users who might not do that kind of contribution often. But it's not really enforceable if they selected all the text, cleared it, and wrote whatever they wanted.

One of the better Github actions I've seen is a PR checker.

It's not a perfect implementation, and can be easily duped - but if the user is going that far do you really need them contributing?

You can use the action to check the PR notes, ensure the headings exist, and if there are any checkboxes that they are checked. You can also go as far as checking if data has been filled in, or validate it against your own set of supported operating systems.

Or you can go as simple as checking if the PR has a label or the title is in a specific format.

Negatives to the Github macOS runner

The positives mentioned above as cross-OS items. Most of the above can be run on the default Linux runner, but with any tool, the macOS runner has its limitations.

Usage of other runners

One of the biggest annoyances is using other runners (like the ubuntu-latest) when everything could be run on the macos-latest runner.

Having to learn apt-get or debug Linux operations now makes your workflow harder to build or another thing on the development maintenance checklist.

You can of course run all your operations within the macOS runner but then you come into the cost issue.

Cost

For whatever reason, macOS runners incur a 10x multiplier on their workflows. This means, even if you could run all your operations on a macOS system, you probably shouldn't.

For example, if I have the following action.yml workflow:

name: Generate Swift Publish website
on:
  push:
    branches:
      - main
jobs:
  publish:
    runs-on: macos-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Install Homebrew
        uses: |
          /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
      - name: Install npm and node
        run: brew install node
      - name: Install markdownlint
        run: npm install -g markdownlint-cli

By the time you get to the installation of the markdown linter, you've probably used 2 minutes realtime, which is 20 minutes (10x multiplier) Github actions time. This isn't even the complete script. There is still URL checking, validation, and building - and once all that is completed deploying to the domain.

If you wanted to run this on a CRON everyday (so you could schedule articles) you could be well over the 2000 minutes on the free tier.

Of course, extracting some of the code into Linux runners would help, but you would still be wasting time somewhere in the chain.

Systems versions

Another of the bigger issues with macOS runners are they are outdated. macos-latest implies the latest major OS, but realistically you're an entire version behind.

This isn't a new issue either. For whatever reason behind the scenes, Github has always been running one version behind in the runner version.

With this means you cannot use the latest Xcode or Swift, resulting in missing out on new SwiftUI or UIKit functionalities, and essentially making the entire action build pointless.

The mismatch in versions ends up creating bigger headaches than what the process is actually worth.

Under resourced

The last limitation I want to touch on is the provided resources. Compared to running these steps locally I've always found that running them on the Github runner has been slower.

Yes there is the setup, and checking out - but it seems there is a limiter or under powered machines used. Combine this with running a somewhat resource-intensive project and you're essentially fighting an uphill battle.

Best practices on using macOS runners

Despite its limitations, the macOS runner can be a powerful tool when used correctly. Here are a few best practices to keep in mind when using the macOS runner:

• Consider using alternative runners, such as the Ubuntu runner, for tasks that are not compatible with macOS. This can help to reduce the cost of running your workflows and simplify your development process
• Be mindful of the cost of running your workflows on the macOS runner. Try to minimise the use of the macOS runner by running certain tasks on alternative runners or using caching and parallelism to improve performance
• Keep an eye on the versions of systems and software that are being used by the macOS runner. Try to use the latest version of Xcode or Swift to take advantage of new features and functionalities
• Consider using more powerful machines or alternative hosting options to run your workflows if you find that the resources provided by the GitHub macOS runner are insufficient for your needs
• Consider splitting your workflows in smaller chunks and running them in parallel, this can help to reduce the time taken to run the workflows and also make it more efficient
• Consider using other Github Actions features like using matrix to test your code on multiple versions of a language or framework
• Look into using caching and artefacts to store the results of your build and test steps and make them available to other jobs and workflows
• Look into using self-hosted runners, which are hosted on your own infrastructure, this allows you to have more control over the environment, and also allows you to run workflows on non-macOS environments

By keeping these recommendations in mind, you can improve your workflow and make the most of the GitHub macOS runner while minimising its limitations.

Conclusion

As discussed in the article, it is important to weigh the pros and cons of using the macOS runner, specifically, the compatibility with various languages and frameworks, limitations and drawbacks, and best practices for using the macOS runner.

Additionally, it's important to consider alternative options and to abstract away certain tasks into Linux runners, as well as being mindful of the cost of running your workflows on the macOS runner.

It's important to chain actions together, and make use of the global checks and best practices, as well as using version control and keep track of the changes in your workflows.

Overall, by keeping these considerations in mind, you can make an informed decision and improve your workflow when using the GitHub macOS runner.