My name is Alexandru Somai. I’m following a major in Software Engineering at the Babes-Bolyai University of Cluj-Napoca, Romania. I have more than two years hands-on experience working in Software Development.

I enjoy writing code in Java, Groovy and JavaScript. The technologies and frameworks that I’m most familiar with are: Spring Framework, Spring Security, Hibernate, JMS, Web Services, JUnit, TestNG, Mockito. As build tools and continuous integration, I’m using Maven and Jenkins. I’m a passionate software developer who is always learning, always looking for new challenges. I want to start contributing to the open source community and Google Summer of Code is a starting point for me.

Currently, Jenkins’ build workspace may become very large in size due to the fact that some compilers generate very large volumes of data. The existing plugins that share the workspace across builds are able to do this by copying the files from one workspace to another, process which is inefficient. A solution is to have a Jenkins plugin that is able to manage and reuse the same workspace between multiple builds.

As part of the Google Summer of Code 2016 I will be working on the External Workspace Manager plugin. My mentors for this project are Oleg Nenashev and Martin d’Anjou. This plugin aims to provide an external workspace management system. It should facilitate workspace share and reuse across multiple Jenkins jobs. It should eliminate the need to copy, archive or move files. The plugin will be written for Pipeline jobs.

You may find the complete project proposal, along with the design details, features, more examples and use cases, implementation ideas and milestones in the design document. The plugin repository will be available on GitHub.

A prototype version of the plugin should be available in late June and the releasable version in late August. I will be holding plugin functionality demos within the community.

I do appreciate any feedback. You may add comments in the design document. If you are interested to have a verbal conversation, feel free to join our regular meetings on Mondays at12:00 PM UTC on the Jenkins hangout. I will be posting updates from time to time about the plugin status on theJenkins developers mailing list.

Recently, I started working on a project to automate some performance tests for my company’s products. We use the awesome Gatling load testing tool for these tests, but we’ve largely been handling the testing very manually to date, due to a lack of bandwidth to get them automated in a clean, maintainable, extensible way. We have a years-old Jenkins server where we use the gatling jenkins plugin to track the history of certain tests over time, but the setup of the Jenkins instance was very delicate and not easy to reproduce, so it had fallen into a state of disrepair.

Over the last few days I’ve been putting some effort into getting things more automated and repeatable so that we can really maximize the value that we’re getting out of the performance tests. With some encouragement from the fine folks in the #jenkinsIRC channel, I ended up exploring the JobDSL plugin and the new Pipeline jobs. Combining those two things with some Puppet code to provision a Jenkins server via thejenkins puppet module gave me a really nice way to completely automate my Jenkins setup and get a seed job in place that would create my perf testing jobs. And the Pipeline job format is just an awesome fit for what I wanted to do in terms of being able to easily monitor the stages of my performance tests, and to make the job definitions modular so that it would be really easy to create new performance testing jobs with slight variations.

So everything’s going GREAT up to this point. I’m really happy with how it’s all shaping up. But then…​ (you knew there was a "but" coming, right?) I started trying to figure out how to add the Gatling Jenkins plugin to the Pipeline jobs, and kind of ran into a wall.

As best as I could tell from my Googling, the plugin was probably going to require some modifications in order to be able to be used with Pipeline jobs. However, I wasn’t able to find any really cohesive documentation that definitively confirmed that or explained how everything fits together.

Eventually, I got it all sorted out. So, in hopes of saving the next person a little time, and encouraging plugin authors to invest the time to get their plugins working with Pipeline, here are some notes about what I learned.

Spoiler: if you’re just interested in looking at the individual git commits that I made on may way to getting the plugin working with Pipeline, have a look at this github branch.

The main task that the Gatling plugin performs is to archive Gatling reports after a run. I figured that the end game for this exercise was that I was going to end up with a Pipeline "step" that I could include in my Pipeline scripts, to trigger the archiving of the reports. So my first thought was to look for an existing plugin / Pipeline "step" that was doing something roughly similar, so that I could use it as a model. The Pipeline "Snippet Generator" feature (create a pipeline job, scroll down to the "Definition" section of its configuration, and check the "Snippet Generator" checkbox) is really helpful for figuring out stuff like this; it is automatically populated with all of the steps that are valid on your server (based on which plugins you have installed), so you can use it to verify whether or not your custom "step" is recognized, and also to look at examples of existing steps.

Looking through the list of existing steps, I figured that the archive step was pretty likely to be similar to what I needed for the gatling plugin:

So, I started poking around to see what magic it was that made that archive step show up there. There are some mentions of this in thepipeline-plugin DEVGUIDE.md and theworkflow-step-api-plugin README.md, but the real breakthrough for me was finding the definition of thearchive step in the workflow-basic-steps-plugin source code.

With that as an example, I was able to start poking at getting agatlingArchive step to show up in the Snippet Generator. The first thing that I needed to do was to update the gatling-plugin project’s pom.xml to depend on a recent enough version of Jenkins, as well as specify dependencies on the appropriate pipeline plugins

Once that was out of the way, I noticed that the archive step had some tests written for it, using what looks to be a pretty awesome test API for pipeline jobs and plugins. Based on those archive tests, I addeda skeleton for a test for the gatlingArchive step that I was about to write.

Then, I moved on toactually creating the step. The meat of the code was this:

Note that in that commit I also added a config.jelly file. This is how you define the UI for your step, which will show up in the Snippet Generator. In the case of this Gatling step there’s really not much to configure, so my config.jelly is basically empty.

With that (and the rest of the code from that commit) in place, I was able to fire up the development Jenkins server (via mvn hpi:run, and note that you need to go into the "Manage Plugins" screen on your development server and install the Pipeline plugin once before any of this will work) and visit the Snippet Generator to see if my step showed up in the dropdown:

GREAT SUCCESS!

This step doesn’t actually do anything yet, but it’s recognized by Jenkins and can be included in your pipeline scripts at that point, so, we’re on our way!

The step that we created above is a first-class DSL addition that can be used in Pipeline scripts. There’s another way to make your plugin work usable from a Pipeline job, without making it a first-class build step. This is by use of the step "metastep", mentioned in the pipeline-plugin DEVGUIDE. When using this approach, you simply refactor your Builder or Publisher to extend SimpleBuildStep, and then you can reference the build step from the Pipeline DSL using the step method.

In the Jenkins GUI, go to the config screen for a Pipeline job and click on the Snippet Generator checkbox. Select step: General Build Step from the dropdown, and then have a look at the options that appear in the Build Step dropdown. To compare with our previous work, let’s see what "Archive the artifacts" looks like:

From the snippet generator we can see that it’s possible to trigger an Archive action with syntax like:

This is the "metastep". It’s a way to trigger any build action that implements SimpleBuildStep, without having to actually implement a real "step" that extends the Pipeline DSL like we did above. In many cases, it might only make sense to do one or the other in your plugin; you probably don’t really need both.

For the purposes of this tutorial, we’re going to do both. For a couple of reasons:

GatlingPublisher is the main build action that we’re interested in using in Pipeline jobs. So, with all of that in mind, here’s our next goal: get step([$class: 'GatlingPublisher', ...) showing up in the Snippet Generator.

The javadocs for the SimpleBuildStep class have some notes on what you need to do when porting an existing Builder orPublisher over to implement the SimpleBuildStep interface. In all likelihood, most of what you’re going to end up doing is to replace occurrences of AbstractBuild with references to the Run class, and replace occurrences of AbstractProject with references to the Job class. The APIs are pretty similar, so it’s not too hard to do once you understand that that’s the game. There is some discussion of this in the pipeline-plugin DEVGUIDE.

For the Gatling plugin, myinitial efforts to port the GatlingPublisher over to implement SimpleBuildStep only required the AbstractBuild→ Run refactor.

After making these changes, I fired up the development Jenkins server, and, voila!

So, now, we can add a line like this to a Pipeline build script:

And it’ll effectively be the same as if we’d added the Gatling "Post-Build Action" to an old-school Freestyle project.

Well…​ mostly.

At this point our modified Gatling plugin should work the same way as it always did in a Freestyle build, but in a Pipeline build, it only partially works. Specifically, the Gatling plugin implements two different "Actions" to surface things in the Jenkins GUI: a "Build" action, which adds the Gatling icon to the left sidebar in the GUI when you’re viewing an individual build in the build history of a job, and a "Project" action, which adds that same icon to the left sidebar of the GUI of the main page for a job. The "Project" action also adds a "floating panel" on the main job page, which shows a graph of the historical data for the Gatling runs.

In a Pipeline job, though, assuming we’ve added a call to the metastep, we’re only seeing the "Build" actions. Part of this is because, in the last round of changes that I linked, we only modified the "Build" action, and not the "Project" action. Running the metastep in a Pipeline job has no visible effect at all on the project/job page at this point. So that’s what we’ll tackle next.

The key thing to know about getting "Project" actions working in a Pipeline job is that, with a Pipeline job, there is no way for Jenkins to know up front what steps or actions are going to be involved in a job. It’s only after the job runs once that Jenkins has a chance to introspect what all the steps were. As such, there’s no list of Builders or Publishers that it knows about up front to call getProjectAction on, like it would with a Freestyle job.

This is whereSimpleBuildStep.LastBuildAction comes into play. This is an interface that you can add to your Build actions, which give them their own getProjectActions method that Jenkins recognizes and will call when rendering the project page after the job has been run at least once.

So, effectively, what we need to do is toget rid of the getProjectAction method on our Publisher class, modify the Build action to implement SimpleBuildStep.LastBuildAction, and encapsulate our Project action instances in the Build action.

The build action class now constructs an instance of the Project action and makes it accessible via getProjectActions (which comes from theLastBuildAction interface):

After making these changes, if we run the development Jenkins server, we can see that after the first successful run of the Pipeline job that calls theGatlingPublisher metastep, the Gatling icon indeed shows up in the sidebar on the main project page, and the floating box with the graph shows up as well:

So at this point we’ve got the metastep syntax working from end-to-end, and we’ve got a valid Pipeline DSL step (gatlingArchive()) that we can use in our Pipeline scripts without breaking anything…​ but our custom step doesn’t actually do anything. Here’s the part where we tie it all together…​ and it’s pretty easy! All we need to do is to make our step "Execution" class instantiate a Publisher and call perform on it.

As per thenotes in the pipeline-plugin DEVGUIDE, we can use the @StepContextParameter annotation to inject in the objects that we need to pass to the Publisher’s perform method:

After these changes, we can fire up the development Jenkins server, and hack up our Pipeline script to call gatlingArchive() instead of the metastepstep([$class: 'GatlingPublisher', enabled: true]) syntax. One of these is nicer to type and read than the other, but I’ll leave that as an exercise for the reader.

With that, our plugin now works just as well in the brave new Pipeline world as it did in the olden days of Freestyle builds. I hope these notes save someone else a little bit of time and googling on your way to writing (or porting) an awesome plugin for Jenkins Pipeline jobs!

The UI aims to improve clarity, reduce clutter and navigational depth to make the user experience very concise. A modern visual design gives developers much needed relief throughout their daily usage and screens respond instantly to changes on the server making manual page refreshes a thing of the past.

Pipelines are visualised on screen along with the steps and logs to allow simplified comprehension of the continuous delivery pipeline – from the simple to the most sophisticated scenarios.

Scrolling through 10,000 line log files is a thing of the past. Blue Ocean breaks down your log per step and calls out where your build failed.

Modern pipelines make use of multiple Git branches, and Blue Ocean is designed with this in mind. Drop a Jenkinsfile into your Git repository that defines your pipeline and Jenkins will automatically discover and start automating any  Branches and validating Pull Requests.

Jenkins will report the status of your pipeline right inside Github or Bitbucket on all your commits, branches or pull requests.

Favourite any pipelines, branches or pull requests and see them appear on your personalised dashboard. Intelligence is being built into the dashboard. Jobs that need your attention, say a Pipeline waiting for approval or a failing job that you have recently changed, appear on the top of the dashboard.

You can read more about Blue Ocean and its goals on theproject page and developers should watch theDevelopers list for more information.

For Jenkins developers and plugin authors:

The Jenkins Design Language (JDL) is a set of standardised React components and a style guide that help developers create plugins that retain the look and feel of Blue Ocean in an effortless way. We will be publishing more on the JDL, including the style guide and developer documentation, over the next few weeks.

The Jenkins plugin tool chain has been extended so that developers can useES6,React, NPM in their plugins without endless yak-shaving. Jenkinsjs-modules are already in use in Jenkins today, and this builds on this, using the same tooling.

Client Side plugins use Jenkins plugin infrastructure. The Blue Ocean libraries built on ES6 and React.js provide an extensible client side component model that looks familiar to developers who have built Jenkins plugins before. Client side extension points can help isolate failure, so one bad plugin doesn’t take a whole page down.

Server Sent Events (SSE) allow plugin developers to tap into changes of state on the server and make their UI update in real time (watch this for a demo).

To make Blue Ocean a success, we’re asking for help and support from Jenkins developers and plugin authors. Please join in our Blue Ocean discussions on the Jenkins Developer mailing list and the #jenkins-ux IRC channel on Freenode!

My name is Samat Davletshin and I am from HSE University from Moscow, Russia. I interned at Intel and Yandex, and cofounded a startup project where I personally developed front-end and back-end of the website.

I am excited to participate in GSoC with Jenkins this summer as a chanсe to make a positive change for thousands of users as well as to learn from great mentors.

Although powerful, Jenkins new job creation and configuration process may be non obvious and time consuming. This can be improved by making UI more intuitive, concise, and functional. I plan to achieve this by creating a simpler new job creation, configuration process focused on essential elements, and embedding new functionality.

My mentors are Kirill Merkushev andMichael Neale

By May 25th I learned about the structure and tools of Jenkins and started working on the first project:

View the current pop-up progress

I used Remodal library for popup and put thereexisting New Job container. Surprisingly, it was fully functional right away. On the GIF you can see that popup receives all job types and then successfully submits the post form creating a new job. I think that could be a good first step. Further I can start changing the window itself.

I am Cynthia Anyango from Nairobi, Kenya. I am a second year student at Maseno University. I am currently specializing on Ruby on Rails and trying to learn Python. I recently started contributing to Open source projects.My major contribution was at Mozilla, where I worked with the QA for Cloud services. I did manual and automated tests for various cloud services. I wrote documentation too. Above that, I am competent and I am always passionate about what I get my hands on.

Currently Jenkins plugin documentation is being stored in Confluence. Sometimes the documentation is scattered and outdated. In order to improve the situation we would like to follow the documentation-as-code approach and to put docs to plugin repositories and then publish them on the project website using the awestruct engine. The project aims an implementation of a documentation continuous deployment flow powered by Jenkins and Pipeline Plugin.

The idea is to automatically pull in the README and other docs from GitHub, show changelogs with versions and releases dates. I will be designing file templates that will contain most of the docs information that will be required from plugin developers. Initially the files will be written inAsciiDoc. Plugin developers will get a chance to review the templates. The templates will be prototyped by various plugin developers.

The docs that will be automatically pulled from github and will be published onJenkins.io under the Documentation section.

My mentors are R.Tyler andBaptiste Mathus

I hope to achieve this by 25th June when we will be having our mid-term evaluations.

I will update more on the progress.

To get started, login to Amazon EC2 console, and click on Spot Requests in the left hand navigation pane. Alternatively, you can directly login toAmazon EC2 Spot Requests console. Then click on theRequest Spot Instances button at the top of the dashboard.

In the Spot instance launch wizard, select the Request & Maintain option to request a Spot fleet that automatically provisions the most cost-effective EC2 Spot instances, and replenishes them if interrupted. Enter an initial target capacity, choose an AMI, and select multiple instance types to automatically provision the lowest priced instances available.

On the next page, ensure that you have selected a key pair, complete the launch wizard, and note the Spot fleet request ID.

Amazon EC2 Spot fleet automates finding the lowest priced instances for you, and enables your Jenkins cluster to maintain the required capacity; so, you don’t need any bidding algorithms to provision the optimal Spot instances over time.

After you have finished the previous step, you can view the EC2 Fleet Status in the left hand navigation pane on the Jenkins dashboard. Now, as you submit more jobs, Jenkins will automatically scale your Spot fleet to add more nodes. You can view these new nodes executing jobs under the Build Executor Status. After the jobs are done, if the nodes remain free for the specified idle time (configured in the previous step), then Jenkins releases the nodes, automatically scaling down your Spot fleet nodes.

If you have a story to share about your team or product, or have a question to ask, do leave a comment for us; we’d love to connect with you!

Hello, my name is Payal Priyadarshini. I am pursing my major in Computer Science & Engineering at the Indian Institute of Technology Kharagpur, India. I am very proficient in writing code in Python, C++, Java and currently getting familiar and hopefully good in Groovy too.

I have internship experiences in renowned institutions like Google and VMware where I worked with some exciting technologies for example Knowledge Graphs, BigTable, SPARQL, RDF in Google. I am a passionate computer science student who is always interested in learning and looking for new challenges and technologies.That’s how I came across to Google Summer of Code where I am working on some exciting data mining problems which you are going to encounter below in this blog.

Jenkins has collected anonymous usage information of more than 100,000 installations which includes set of plugins and their versions etc and also release history information of the upgrades. This data collection can be used for various data mining experiments. The main goal of this project is to perform various analysis and studies over the available dataset to discover trends in data usage. This project will help us to learn more about the Jenkins usage by solving various problems, such as:

As a part of the Google Summer of Code 2016, I will be working on the above mentioned problems. My mentors for the project are Kohsuke Kawaguchi and Daniel Beck. Some analyses has already been done over this data but those are outdated as charts can be more clearer and interactive. This project aims to improvise existing statistics and generating new ones discussed above.

This project covers wide-range of the use-cases that has been derived from the problems mentioned above.

You may find the complete project proposal along with the detailed design of the use-cases with their implementation details here in thedesign document.

A complete version of the use-case 1: Upgrade & Downgrade Analysis should be available in late June and basic version of plugin recommendation system will be available in late July.

I do appreciate any kind of feedback and suggestions. You may add comments in thedesign doc. I will be posting updates about the statistics generation status on thejenkins-dev mailing list and Jenkins-infra@lists.jenkins-ci.org">jenkins-infra mailing list.

I am Minudika Malshan, an undergraduate student in Computer Science and Engineering from University of Moratuwa, Sri Lanka.

As a person who is passionate in open source software development and seeking for new knowledge and experience, I am willing to give my contribution for this project.

LinkedIn | Twitter

The Support-Core Plugin provides the basic infrastructure for generating "bundles" of support information with Jenkins. There are two kinds of bundles.

However current support-core plugin is not much user friendly. The object of this project is to make it more user friendly by adding some features which make a sophisticated environment for the user who use support plugin.

In this project scope, there are three features and improvements we are going to consider.

Arnaud Héritier and Steven Christou are guiding me through the project as my mentors.

Additionally, Jenkins World offers great opportunities for hands-on learning, exploring and networking:

Due to its popularity in previous years, we are bringing back the plugin development workshop. This workshop will introduce developers to the Jenkins plugin ecosystem and terminology. The goal is to provide a cursory overview of the resources available to Jenkins plugin developers. Armed with this information, Jenkins developers can learn how to navigate the project and codebase to find answers to their questions.

BoFs, as they are usually known, will be a new addition to Jenkins World this year. Sessions will be curated on various technical topics from DevOps to how enterprises are integrating Jenkins in their environment. Discussions will be lead by the industry’s brightest minds who have an influence in shaping the future of Jenkins.

Got a Jenkins question that’s been keeping you up at night? Need to bounce ideas off somebody? Or you just need someone to fix your Jenkins issue? This is your chance to get connected with the Jenkins Experts. Experts will be on hand to help with all your Jenkins needs on Sept 14th & 15th.

The objective of this session is to help you assess your level of readiness for the certification exam - either the Certified Jenkins Engineer (CJE/open source) certification or the Certified CloudBees Jenkins Platform Engineer (CCJPE/CloudBees-specific) certification. After an overview about the certification program, a Jenkins expert from CloudBees will walk you through the various sections of the exam, highlighting the important things to master ahead of time, not only from a pure knowledge perspective but also in terms of practical experience. This will be an interactive session.

Don’t miss out on Super Early Bird Rate $399. Price goes up after July 1.

At the beginning of the journey there were several projects using roughly the same technologies. Those projects had several branches, for maintenance of releases, for new features.

In turn, each of those branches had to be carefully built, deployed on different platforms and versions, promoted so they could be tested for functionalities, performances and security, and then promoted again for actual delivery.

Additionally, we had to offer the test teams the means to deploy any version of their choice on any supported platform in order to carry out some manual tests.

This represented, for each branch, around 20 jobs. Multiply this by the number of branches and projects, and there you are: more than two years after the start of the story, we had more than 3500 jobs.

3500 jobs. Half a dozen people to manage them all…​

How did we deal with this load?

We were lucky enough to have several assets:

We also knew that, in order to scale, we’d have to provide a solution with the following characteristics:

Creating pipelines using the Job DSL and embedding the scripts in the code was simple enough. But what about branching? We needed a mechanism to allow the creation of pipelines per branch, by downloading the associated DSL and to run it in a dedicated folder.

But then, all those projects, all those branches, they were mostly using the same pipelines, give or take a few configurable items. Going this way would have lead to a terrible duplication of code, transforming a job maintenance nightmare into a code maintenance nightmare.

Our trick was to transform this vision of "pipeline as code" into a "pipeline as configuration":

Once this was done, the only remaining trick was to automate the creation, update, start and deletion of the pipelines using SCM events. By enabling SCM hooks (in GitHub, BitBucket or even in Subversion), we could:

Once a project wants to go in our ecosystem, the Jenkins team "seeds" the project into Jenkins, by running a job and giving a few parameters.

It will create a folder for the project and grant proper authorisations, using Active Directory group names based on the project name.

The hook for the project must be registered into the SCM and you’re up and running.

Mixing the use of strong pipeline libraries configured by properties and the direct use of the Jenkins Job DSL is still possible. The Seed plugin supports all kinds of combinations:

Usually, we tried to have a maximum reuse, through only pipeline libraries, for most of our projects, but in other circumstances, we were less strict and allowed some teams to develop their own pipeline script.

In the end, what did we achieve?

Self service✔︎

Security✔︎

Simplicity✔︎

Extensibility✔︎

Now, what about the Pipeline plugin? Both this plugin and the Seed plugin have common functionalities:

What we have found in our journey is that having a "pipeline as configuration" was the easiest and most secure way to get a lot of projects on board, with developers not knowing Jenkins and even less the DSL.

The outcome of the two plugins is different:

If time allows, it would be probably a good idea to find a way to integrate the functionalities of the Seed plugin into the pipeline framework, and to keep what makes the strength of the Seed plugin:

You can find additional information about the Seed plugin and its usage at the following links:

Every time I’ve moved a team to continuous integration and delivery, one problem we always encounter is how to run all the tests needed to ensure high-quality changes while still keeping pipeline times reasonable and changes flowing smoothly. More tests mean greater confidence, but also longer wait times. Build systems may or may not support running tests in parallel but only on one machine even while other lab machines sit idle. In these cases, parallelizing test execution across multiple machines is a great way to speed up pipelines. The Parallel Test Executor plugin lets us leverage Jenkins do just that with no disruption to the rest of the build system.

For this post, I’ll be running a pipeline based on the Jenkins Git Plugin. I’ve modified the Jenkinsfile from that project to allow us to compare execution times to our later changes, and I’ve truncated the "mvn" utility method since it remains unchanged. You can find the original file here.

It’s a Maven project, so the Jenkinsfile is pretty simple. As noted above, I’ve split the Maven build into separate “Build” and “Test” stages. Maven doesn’t support this split very well, it wants to run its all the steps of the lifecycle in order every time. So, I have to call Maven twice: first using the “skipTests” property to do only build steps in the first call, and then a second time with out that property to run tests.

On my quad-core machine, executing this pipeline takes about 13 minutes and 30 seconds. Of that time, it takes 13 minutes to run about 2.7 thousand tests in serial.

This looks like an ideal project for parallel test execution. A short build followed by a large number of serially executed tests that consume the most of the pipeline time. There are a number of things I could try to optimize this. For example, I could modify test harness to look for ways to parallelize the test execution on this single machine. Or I could try speed up the tests themselves. Both of those can be time-consuming and both risk destablizing the tests. I’d need to know more about the project to do it well.

I’ll avoid that risk by using Jenkins and the Parallel Test Executor Plugin to parallelize the tests across multiple nodes instead. This will isolate the tests from each other, while still giving us speed gains from parallel execution.

The plugin will read the list of tests from the results archived in the previous execution of this job and splits that list into the specified number of sublists. I can then use those sublists to execute the tests in parallel, passing a different sublist to each node.

Let’s look at how this changes the pipeline:

That’s it! The change is significant but it is all encapsulated in this one method in the Jenkinsfile.

Here’s the results for the new pipeline with parallel test execution:

The tests ran almost twice as fast, without changes outside pipeline. Great!

However, I used 4 test executors, so why am I not seeing a 4x? improvement. A quick review of the logs shows the problem: A small number of tests are taking up to 5 minutes each to complete! This is actually good news. It means that I should be able to see further improvement in pipeline throughput just by refactoring those few long running tests into smaller parts.

While I would like to have seen closer to a 4x improvement to match to number of executors, 2x is still perfectly respectable. If I were working on a group of projects with similar pipelines, I’d be completely comfortable reusing these same changes on my other project and I’d expect to similar improvement without any disruption to other tools or processes.

Meeting link

Both meetings will be conducted and recorded via Hangouts on Air. The recorded sessions will be made public after the meetup. The agenda may change a bit.

Tool Auto-Installers are exposed in Pipeline through the tool step and onci.jenkins.io we already had JDK8 and Groovy available. This meant that the Jenkinsfile would invoke tool and Pipeline would automatically install the desired tool on the agent executing the current Pipeline steps.

The tool step does not modify the PATH environment variable, so it’s usually used in conjunction with the withEnv step, for example:

Satisfying the MongoDB dependency would still be tricky. If I caved in and installed MongoDB on a single unicorn agent in the cluster, what could I say the next time somebody asked for a special, one-off, piece of software installed on our Jenkins build agents?

After doing my usual complaining and whining, I discovered that the CloudBees Docker Pipeline plugin provides the ability to run containers inside of aJenkinsfile. To make things even better, there areofficial MongoDB docker images readily available on DockerHub!

This feature requires that the machine has a running Docker daemon which is accessible to the user running the Jenkins agent. After that, running a container in the background is easy, for example:

The beauty, to me, of this example is that you can pass aclosure to withRun which will execute while the container is running. When the closure is finished executin, just the sh step in this case, the container is destroyed.

With that system requirement satisfied, the rest of the stages of the Pipeline fell into place. We now have a single source of truth, theJenkinsfile, for the sequence of dependent tasks which need to be executed, accounting for variations in systems requirements, and it actually generatesthose pretty blue charts!

Of course, a nice added bonus is the beautiful visualization of ournew Pipeline!

I’ve found a Ruby project, hermann, I’d like to build using Jenkins Pipeline. I’d also like to have the code coverage results published with each build job. I could write a plugin to publish this data, but I’m in a bit of hurry and the build already creates an HTML report file using SimpleCov when the unit tests run.

I’m going to use the HTML Publisher Plugin to add the HTML-formatted code coverage report to my builds. Here’s a simple pipeline for building the hermann project.

Simple enough, it builds, runs tests, and archives the package.

Now I just need to add the step to publish the code coverage report. I know that rake spec creates an index.html file in the coverage directory. I’ve already installed the HTML Publisher Plugin. How do I add the HTML publishing step to the pipeline? The plugin page doesn’t say anything about it.

Documentation is hard to maintain and easy to miss, even more so in a system like Jenkins with hundreds of plugins the each potential have one or more groovy fixtures to add to the Pipeline. The Pipeline SyntaxSnippet Generator helps users navigate this jungle by providing a way to generate a code snippet for any step using provided inputs.

It offers a dynamically generated list of steps, based on the installed plugins. From that list I select the publishHTML step:

Then it shows me a UI similar to the one used in job configuration. I fill in the fields, click "generate", and it shows me snippet of groovy generated from that input.

I can use that snippet directly or as a template for further customization. In this case, I’ll just reformat and copy it in at the end of my pipeline. (I ran into a minor bug in the snippet generated for this plugin step. Typing error string in my search bar immediately found the bug and a workaround.)

When I run this new pipeline I am rewarded with an RCov Report link on left side, which I can follow to show the HTML report.

I even added the keepAll setting to let I can also go back an look at reports on old jobs as more come in. As I said to to begin with, this is not as slick as what I could do with a custom plugin, but it is much easier and works with any static HTML.