Ampelofilosofies

homeaboutrss
The mind is like a parachute. If it doesn't open, you're meat.

Handling pull requests with Jenkins

09 Sep 2017

My requirements for handling pull requests in CI are:

  • I want to build the merge reference of a branch for which there is an open pull request and I want to do it every time something is commited on either branch participating in the pull request.
  • I want to be able to structure my build jobs in separate, parametrizable “component” jobs which I can then compose in highly prallelizable pipelines.
  • I want all of the above “as code”, meaning no GUI interaction necessary to setup.

This in the current state of my build stack seems nigh on impossible.

Lets take it from the beginning:

Git hosting and PR support is provided by BitBucket Server, build management by Jenkins 2.x.

Add BitBucket Branch Source Plugin to get the option of triggering builds that have open PRs on a multi branch pipeline.

The as-code part means we have a job DSL spec, the pull request trigger and “componetized” jobs means a multi-branch pipeline job.

Components and Composites

We have a big system with lots of moving parts. I tend to structure my code in building blocks and now that I can code my CI jobs, I tend to do the same.

So we have “component” jobs, that do one thing (build, test) and are implemented as parametrized pipeline jobs. These are the jobs that require a workspace, check out the repository and do the heavy duty compiling and testing.

I tend to implement them as reusable as possible, so one of the parameters is the branch to build.

Then we have “composite” jobs, which take care of scaling our CI. They put together batches of the component jobs, adjust parameters according to scale criteria (e.g “fast”, “thorough” etc.), launch jobs in parallel and then aggregate results and report.

Most of the time we end up with a single composite pipeline which then becomes Main, Nightly or Release with the adjustment of a couple of parameters. Having a branch parameter allows us to do the same thing on any branch.

Tweaking this pipeline to use in pull requests was the exercise leading to this post.

You can’t have what you want

Right from the start it becomes obvious that I cannot have builds that build the merge ref.

Mostly because it means figuring out the two SHAs (easy when reading the checkout step’s return values) and then adjusting the component jobs to perform the merge every time - which kinda defeats the purpose of the component jobs, since we do this merge only for pull requests and not for the rest of the mainline builds. We end up with a second set of component jobs built specifically for the pull request which is not worth the effort.

I can live with building just the PR source branch and letting BitBucket detect merge conflicts given that we rebase frequently and aim for short lived branches. Not building the merge ref increases the probability of integration test breakage on the master branch after merging, so it is now up to the devs to ensure master does not break by an obscure integration problem.

Que requiring a rebase before merge.

The other problem is that we can only build the tip of the source branch and not the specific SHA that triggered the build.

The reason for that is that ‘git’ in the Pipeline DSL does not understand SHAs. It is possible to do by falling back to the generic checkout directive but omg! is that unreadable.

Again, not the end of the world, just less than exact verification than we wish and the possibility for building the same ref twice. Still BitBucket keeps track of the build status for each ref and merging requires a green build for that last commit.

Btw. this problem exists also for the mainline builds so I consider it a major bug of git integration in Jenkins.

Some way of passing the scm context to jobs started via build would be nice. Might also be possible to hack with current means.

Regroup.

The compromise we come to is:

  • Build the tip of the source branch for every open PR.
  • Re-use the main pipeline script.

To get everything as code we had to revert to manipulating XML. I did not manage to get the configure block correctly set using the overloaded syntax.

I kept getting missing method errors for String.div() and could not set class for the XML elements. The sollution was to go back to proper XML.

The following code defines two XML snippets in the beginning. The first configures the BitBucket Branch Source Plugin for the multibranch pipeline so that only PRs trigger.

The second snippet changes the location and filename that is used as Jenkinsfile.

Thus we get a multibranch pipeline job that polls BitBucket every minute, triggering the “Jenkinsfile” pipeline when it detects changes in open pull requests.

Addendum 2017/09/13

Things break when faced with the real world

The snippet configuring the BitBucket Branch Source Plugin originally defined two traits. The first one was a branch discovery trait and it resulted in not only building a pull requests but also the source branch for the pull request as well. Chaos ensued and resources were wasted.

The code is now also formated for humans :).

def xmlSources='''<sources class="jenkins.branch.MultiBranchProject$BranchSourceList">
<data><jenkins.branch.BranchSource>
<source class="com.cloudbees.jenkins.plugins.bitbucket.BitbucketSCMSource">
<serverUrl>https://bitbucket.zuehlke.com</serverUrl>
<credentialsId>f2361e6b-b8e0-dead-beef-5eed5bc1a59f</credentialsId>
<repoOwner>FOO</repoOwner>
<repository>bar</repository>
<traits>
<com.cloudbees.jenkins.plugins.bitbucket.OriginPullRequestDiscoveryTrait>
<strategyId>2</strategyId>
</com.cloudbees.jenkins.plugins.bitbucket.OriginPullRequestDiscoveryTrait>
</traits>
</source>
<strategy class="jenkins.branch.DefaultBranchPropertyStrategy">
<properties class="empty-list"/>
</strategy></jenkins.branch.BranchSource>
</data>
<owner class="org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject" reference="../.."/></sources>'''

def factorySource='''
<factory class="org.jenkinsci.plugins.workflow.multibranch.WorkflowBranchProjectFactory">
<owner class="org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject" reference="../.."/>
<scriptPath>Tools/jenkins/pipelines/PullRequest.groovy</scriptPath></factory>'''
def sourcesNode= new XmlParser().parseText(xmlSources)
def factoryNode= new XmlParser().parseText(factorySource)
multibranchPipelineJob('PullRequests') {
  description "Pull request builds"
  displayName "PullRequests"
  triggers{
    periodic(1)
  }
  orphanedItemStrategy {
    discardOldItems {
        numToKeep(20)
    }
  }

  configure { project -> 
    project<<sourcesNode
  }
  configure { project -> 
    project << factoryNode
  }

Within the pipeline, the source branch of the pull request is then in the environment variable CHANGE_BRANCH

blog comments powered by Disqus