Tips and Tricks of Pipelines

How did this chapter come to be? I was basically just pasting here stuff during work. There are many subtleties to pipelines, I recommend to all of you to create an own Jenkins as a playground of study, and experiment with it.

Handling build dependencies

Now before we start, let's keep in mind that pipeline scripts can be formulated in either Declarative or Scripted syntax. The first example code in this section is formulated in Declarative Pipeline syntax. If you don't know what the difference is, read up the first chapter. Let's see the following problems:

1. You have two jobs following each other:

#!/usr/bin/env groovy

// HEADS UP: This is Declarative Pipeline syntax

pipeline {
agent any
    stages {
        stage("build") {
            steps {
                build('job-1')
                build('job-2')
            }
        }
    }
}

In this case, the first build step will be executed first, but the whole job will stop and fail if the first job fails. But let's say that you don't want this. Instead, you want the following:

• You want both of them to run, even if the first fails, and only fail if the second job fails
• You want the jobs to run after each other but not parallel.

Here's your solution: Use the propagate attribute, and set it to false. If disabled, then this step succeeds even if the downstream build is unstable, failed.

build(job: 'job-1', propagate: false)
// This step will run even if job-1 failed
build(job: 'job-2')

This case the job will start the second build step even if the first one failed. But there is a catch. If the first one failed, you will not know this from the pipeline's result. The pipeline job's result will evaluate to be a SUCCESS even if the first job fails. This is not necessarily a problem, but the third example will offer a solution for this issue.

2. You want to run parallel builds:

#!/usr/bin/env groovy

pipeline {
    agent any
    stages {
        stage("test") {
            steps {
                parallel (
                    "Unit Test" : {
                        build("unit-test-job")
                    },
                    "Component Test" : {
                        build("component-test-job")
                    },
                    "Build" : {
                        build("build-job")
                    }
                )
            }
        }
    }
}

Yes but is there a catch? Yes there is:
For one, you can't have anything else, but that parallel block in that stage. If you would add a separate build step, like this:

//    !!!!!!!!!!!!!!!!!!!!!!!!!!
//    DONT USE! BAD CODE

            steps {
                parallel (
                    "Unit Test" : {
                        build("unit-test-job")
                    },
                    "Component Test" : {
                        build("component-test-job")
                    },
                    "Build" : {
                        build("build-job")
                    }
                )
                // extra build step together with parallel block
                build("extra-job")
            }
//    !!!!!!!!!!!!!!!!!!!!!!!!!!

...you would get the following ERROR:
WorkflowScript: 6: Invalid step "parallel" used - not allowed in this context - The parallel step can only be used as the only top-level step in a stages step block.

A quick solution is to reformulate your script in Scripted pipeline syntax:

node {
// HEADS UP: this is Scripted Pipeline syntax
    stage("Build") {
        parallel (
            "First Build" : {
                build("first-build-job")
            },
            "Second Build" : {
                build("second-build-job")
            }
        )
        build("test-job")
        parallel (
            "Third Build" : {
                build("third -build-job")
            },
            "Last Build" : {
                build("last-build-job")
            }
        )
    }
}

3. You want to examine the build

The build step has a hidden attribute: wait (just like propagate, which you have seen before in this chapter).

build(job: 'example-job', wait: true)

WARNING: I've seen online that many people confuse wait with propagate, when they want to ignore the failure of a job in the pipeline. Let's see the differences:

build(job: 'example-job', wait: false): This means that the pipeline will not wait for the result of this build step, just starts it and jumps to the next step. If you put blocks like this after each other, they will be started one after another, without waiting for each other to finish (Actually almost like a parallel block).
build(job: 'example-job', propagate: false): This means that the pipeline will first execute this step, and continue to the next step even if this step failed.

wait is true as default, so normally you don't have to add it to the code. In case you don't turn it false, the return value of the build step will be an object on which you can obtain read-only properties: so you can inspect its .result and so on.

node {
    stage("example") {
        echo build("example-job").result
    }   

With this, you can actually achieve to set the build's result even if you ignored failing builds with setting propagate: false: Basically what you have to do is just to set the jobs result by hand after examining given conditions with an if block.
This way the pipeline will keep running even if the first job fails, but you will see from the result of the pipeline if something was wrong.

node {
    stage("example") {
        b = build(job: "example-job", propagate: false).result
        if(b == 'FAILURE') {
            echo "First job failed"
            currentBuild.result = 'UNSTABLE' // of FAILURE
        }
    }
    stage("test") {
        build("test-job")
    }
}