The main superpower of a programmer is her ability to automate almost everything. That's where GitHub Actions shines... if you can write with confidence a GitHub Workflow in YAML to solve your particular problem. That's a big if. But if you can't, there is still hope.
Let me introduce you to an open-source project started by Piotr Krzemiński and to which I contributed a lot in the last months 👨🏻💻
Authoring GitHub Actions workflows in Kotlin. You won't go back to YAML!
github-workflows-kt is a tool for creating
GitHub Actions workflows in a type-safe script, helping you to
build robust workflows for your GitHub projects without mistakes, with pleasure, in
Kotlin.
You won't go back to YAML!
💡 Idea
We're often surrounded by YAML configuration. It's a powerful format that provides simple syntax for defining
hierarchical data, but it is sometimes used (abused?) to configure complicated scenarios which leads to complicated
files that are difficult to write and maintain.
Who among us hasn't accidentally used the wrong indentation, missed a possibility to extract a reusable piece of code
or been confused by ambiguous types? The power of a generic-purpose would come in handy in these cases.
We're developing github-workflows-kt to solve these and other problems, so you can create GitHub Workflows with
confidence.
✨ Benefits
no indentation confusion - Kotlin's syntax doesn't rely on it
github-actions-kotlin-dsl(*) is a library that allow you to generate a GitHub Workflow for Kotlin Actions in a type-safe way.
(*) I know that's quite a mouthful. I personally call it github-actions.kts
1) Edit the script in IntelliJ IDEA Community Edition which is free and awesome
2) Put the script in the folder .github/workflows, make it executable
3) Install Kotlin
4) github-actions-kotlin-dsl is the library you use to build a GitHub Workflow object
5) At the end, you call workflow.writeToFile()
6) which generates .github/workflows/$NAME$.yaml which you can then run on GitHub Actions.
You can see underlined in blue that the main components of a YAML workflow have a striaghtforward Kotlin equivalent: workflow() Push() job() uses() run() GithubActionNameVxxx()
The main superpower of a programmer is her ability to automate almost everything.
Since that's precisely what GitHub Actions is for, all good right?
Unfortunately most people struggle to write a GitHub workflow solving their particular problem: And that's because YAML is a bad programming language - see below.
So instead we resort to copy/paste programming, trying to find someone that has already spent the time writing a workflow to solve a problem hopefully similar to ours.
Don't get me wrong, copy/paste programming is better than nothing, and I have written myself a guide to get a CI up and running in no time for Java/Kotlin developers
But we are missing out if we can't automate our particular problems.
So what's the issue?
Github's YAML is a Bad Programming Language
Average YAML developer trying to get things done, loosing ten minutes or more between each iteration.
YAML in general is just a different way to write JSON.
With a different set of issues...
But GitHub has extended YAML to something we should recognize as a programming language, according to the principle of Duck Typing.
*🐥 GitHub YAML as a programming language: 🐥 *
it allows to write arbitrary scripts
it runs on abritrary VM on Azure
...with an arbitrary number of jobs and commands
...with a shitload of plugins written in a arbitrary language, and which can be configured in all sort of ways
...with variables and secrets
...with for loops - a.k.a. strategy matrix
...with boolean logic - run this job only if the previous was successfull, run this action only on error
...with powerful GitHib expressions - a.k.a an eval function
...
If it walks like a duck and it quacks like a duck, then it must be a duck.
The issue here is that YAML was never designed for this
Therefore support in your IDE is quite poor. They do try to help but it's nowhere as good as with an actual programming language. Also the reason that YAML looks fancier than JSON is that its rules are less simple, but which means you are more likely to screw up.
Most importantly:
GitHub Action's edit-compile-run feedback loop is super slow
What you can't see in the picture above is that the GitHub Actions YAML developer might be wasting 10 minutes or more for each iteration of her script!
So is there a way out of the YAML wasteland?
Type-Safe Workflows
In YAML, an incorrect syntax looks very much like a correct one.
valworkflow=workflow(name="Hello: World",// 1. runtime error: semicolon not allowedon=listOf(Schedule(listOf(// 2. named arguments 07:42 instead of 42:07 in the YAML// 3. runtime error: Field 'dayWeek' outside of range 0..6Cron(minute="42",hour="7",dayWeek="7"))),PR(),// 4. compile error: should be PullRequest() ),sourceFile=__FILE__.toPath(),){job(id="git checkout",// 5. runtime error: space not allowed in a job idrunsOn=ubuntu_latest// 6. compile error: should be UbuntuLatest){// 7. runtime error: semicolon not allowed hereuses(name="First: Check out",action=CheckoutV3())}}
By just compiling and running the script we can prevent plenty of small errors that would have wasted each 10 minutes of our time on the CI.
This is pretty cool, and I wished I had that when I got started.
But now I have YAML workflow that work,
and I don't want to restart from scratch.
I hear you.
I had the same concern.
And then a diabolic idea was born in my mind:
Right so we can have a Kotlin script that generates its YAML version.
But could we take the existing YAML and generate the Kotlin script that would generate itself? A bit like a child who would give birth to its parent?
As it turns out it's possible, and after some intense hacking, the script-generator was born. It allows you to automate your migration to Kotlin.
You run something like:
$ ./gradlew :script-generator:run --args /path/to/.github/workflows/build.yaml
Kotlin script written to build.main.kts
Run it with: ./build.main.kts
The resulting YAML file with be available at build.yaml
You can then make sure the new YAML is equivalent to the old one by doing a semantic diff with https://yamldiff.com/
This really hits home for me: what a superb asset GitHub Actions can be if you can feel confident that you can write a workflow to solve your particular problem.
FAQ
What if I Don't Know Kotlin?
I don't think the choice of programming language matters, as long that it's not Bash. I mean that any modern programming language with good IDE support would be up to the task.
We are doing only basic stuff here. We build a Workflow object by calling workflow() then job then run("my command") or uses(MyActionWithParameters(....))
The YAML and the Kotlin will get out of sync if you don't run the Kotlin script before committing it, if you forget to commit the YAML file, or if you edit the YAML file directly. We have a consistency check to fail early when that's the case. Just use workflow.writeToFile(addConsistencyCheck = true). The trade-off is that the CI will be something like 15s slower.
GitHub's YAML has a huge number of features so you may stumble upon one not supported by the library yet. We have a type-unsafe alternative to compensante those gaps. You can also look at open issues or create your own.
IntelliJ does not support Kotlin Scripts as well it does normal Kotlin programming. Hopefully this gets better with time.
Compiling and running a Kotlin script is slower than you are used to with an interpreted language like Python or JavaScript. But it's still two orders of magnitude faster than having to commit, push and wait 10 minutes on the CI.
If you use it in a team, you have to make sure that your colleagues undertand that the YAML file was generated by the Kotlin script. Mind you the first YAML line tells it: # This file was generated using Kotlin DSL (.github/workflows/hello_world_workflow.main.kts).
You are welcome to send them a link to my article :)
The title for this article was stolen from Michel's Pardo talk Kotlin: a new Hope in Java 6 Wasteland which made want to switch to Kotlin back in the days.
The End 🔚
That's all I have friends. Thanks a lot of you made is this far.
I'm especially happy to be back writing articles, something I didn't manage to do since almost one year.
I would love to hear from your feedback if you give it a try.