Quantcast
Channel: Jenkins Blog
Viewing all 1088 articles
Browse latest View live
↧

Jenkins World 2016: Call For Papers Is Open!

January 29, 2016, 10:16 pm
$
0
0
This is a guest post by Alyssa Tong. Alyssa works for CloudBees, helping to organize Jenkins community events around the world.

Jenkins World 2016

Planning is underway for Jenkins World, a major Jenkins event for developers, release engineers and others interested in automation. The conference will be held from September 13th to 15th in Santa Clara, California and is being organized and sponsored in part by CloudBees. Just like the "Jenkins User Conferences" before it, this year’s event will feature many experts from the Jenkins community that help make Jenkins the most popular open source automation server on the planet. We’ve found that we outgrew the popular multi-city one-day Jenkins User Conferences, so unlike previous years Jenkins World will be a three-day event in one place with an incredible amount of great content.

The goal of the event is to bring Jenkins contributors and users of all levels together, from around the world, to discuss, share and learn from one another. Starting today we’re opening the call for proposals. As a global event, users from all over the world are encouraged to submit a talk between now and May 1st, 2016 (11:59pm PST).

We look forward to receiving your amazing submission, and seeing you in Santa Clara this fall.

Submit a proposal today!

↧
Search

SCaLE 14x Conference Report

February 13, 2016, 6:58 pm
$
0
0

Historically January has always been a very big month for the Jenkins community. Between FOSDEMSouthern California Linux Expo (also known as SCaLE) we seem to hand out more stickers during the last week in January than any other week of the year.

This year’s SCaLE 14X conference finally outgrew the LAX Hilton in Los Angeles, where it had been hosted in years past, and moved over to the Pasadena Convention Center in Pasadena California. While the organizers of the conference expanded their scope, so did the Jenkins project!

In addition to our normal Jenkins stickers, we also had some special edition stickers with special logos to give away this year, namely:

  • Angry Jenkins

  • General Jenkins

  • Superhero Jenkins

  • "Cute" Jenkins

  • Ninja Jenkins

To accompany the stickers we also had both blue Jenkins and red "Angry Jenkins" pins. Savvy Jenkins users might recognize "Angry Jenkins" from the Jenkins server’s internal 500 page; fortunately however very few people that came by the booth to say hello were familiar with Angry Jenkins.

Talking Points

Aside from talking about the cool stickers and pins, we spent the vast majority of time talking about Jenkins to two groups of people:

  1. those who never had actually used Jenkins, even if they had heard of it

  2. users who knew plenty about Jenkins but hadn’t actually heard about some of the Jenkins 2.0 Proposals.

Anecdotally, it seemed like most of the people that I talked to about "Jenkins 2.0" were pretty excited about the Jenkinsfile idea and starting to define their build processes and delivery pipelines as code in their source repositories.

Perhaps more importantly though, we spoke with many users about where Jenkins is causing them pain or frustration. Speaking directly with users at events like SCaLE or Jenkins Area Meetups is always fun, having a high-bandwidth conversation about what we can do better and/or offering solutions/workarounds to hopefully relieve some pain-points.

In one such case, a contributor approached me and complained that he had emailed the developers' mailing list and frustratingly never actually received a response. Comically enough, neither of us were able to find the email he had sent the mailing list (whoops!) but because of the dynamic nature of booth-duty at SCaLE, we got him squared away with a repository to contribute aJenkins Charm for the Juju configuration management tool.

Jammin'

Among the booth-duty highlights was meeting a few folks who were interested on starting a southern California Jenkins meetup. Over the days following the conference, and a brief discussion on thejenkinsci-jam@ mailing list, and theLos Angeles Jenkins Area Meetup was born!

I’m looking forward to the meetup growing over the next couple months and helping build a stronger local Jenkins community in Southern Califonia for the other 51 weeks a year that SCaLE isn’t happening.

SCaLE is one of my more favorite open source conferences, the positive community in attendance, a kid-friendly atmosphere ("Game Night" was a blast) and the broad spectrum of sessions available make it a great way to spent the weekend in southern California.

We hope to see you there again next year!

↧
↧

January 2016 San Francisco JAM Report

February 21, 2016, 8:48 pm
$
0
0

On January 20, the first San Francisco JAM(Jenkins Area Meetup) of the new year was held at Mesosphere’s offices. We had two speakers - myself, and Roger Ignazio, an infrastructure automation engineer at Mesosphere. Around forty people attended and enjoyed the food and drinks Mesosphere provided for us.

Links to the talks are below:

More JAMs will be happening in the coming months - for example, the firstLos Angeles JAM is tentatively planned for early March, 2016! You can always find out the latest on JAMs around the world at the Jenkins Area Meetup page.

↧

Jenkins security updates

February 24, 2016, 6:04 pm
$
0
0

We released Jenkins updates today that include important security fixes: 1.650 and 1.642.2. For detailed information about the security content of these updates, see the security advisory.

Subscribe to the jenkinsci-advisories mailing list to receive important notifications related to Jenkins security.

↧

Jenkins 2.0 alphas released

March 3, 2016, 9:23 am
$
0
0

After firstannouncing the Jenkins 2.0 effort last fall, we are pleased to announce the availability of the first Jenkins 2.0 alpha builds. For months we have had builds available from the jenkins_2.0 branch of development, but the "alpha" builds mark Jenkins 2.0 being officially made available for testing and feedback.

Download 2.0-alpha now

Getting Started with Jenkins 2.0

Jenkins 2.0 Highlights

Pipeline as Code

The new Pipeline functionality in Jenkins allows you to define configuration as code, which can be checked in and version controlled along with the rest of your project’s source code.

A simple build/test pipeline

Defining your pipeline’s configuration as code makes it easier to create a simple "build and test" pipeline, while enabling more advanced and complex pipelines through the expressive Groovy-based domain specific language.

Out of the box experience

For new users, Jenkins 2.0 starts off with set of recommended plugins, seen in the image above, to help get you started with the right set of tools to get up and running with Jenkins quickly.

For the more adventurous users, the Jenkins 2.0 initial setup process also allows you to pick and choose exactly the plugins you want to meet your specific needs.

Totally backwards compatible

Jenkins 2.0 is a drop-in replacement of the Jenkins 1.x series of releases and fully backward compatible. There will be practically no reason not to upgrade once 2.0 is released in the next couple of months.

Tell us what you think!

We’re very interested in your feedback on what you think of the Jenkins 2.0 preview releases.

If you use Twitter, you can leave us some feedbackon Twitter

Our jenkinsci-users mailing list is also available for feedback inthis thread

And of course, since this is a preview release, if you find any issues please report them to ourissue tracker to the JENKINS project.

↧
↧

Jenkins Hackergarten : mercredi 9 mars 2016 à Toulouse

March 3, 2016, 9:23 am
$
0
0

Vous êtes développeuse ou développeur, vous avez envie de découvrir le projet Jenkins de l’intérieur, en bricolant sur un sujet qui vous intéresse, lisez la suite !

Mercredi 9 mars, le Toulouse Jenkins Area Meetup organise à Toulouse un Hackergarten Jenkins, occasion idéale pour faire ses premiers pas dans la communauté assisté(e) d’un contributeur au projet.

Hackergart quoi ?

Hackergarten est un mot qui provient de la contraction des mots Hacker etKindergarten, ce dernier étant le mot allemand qui désigne en gros l’école maternelle.

Comment ça va se passer ?

En partenariat avec le Toulouse Java User Group (c’est sur ce lien qu’il faut s’inscrire), nous nous donnons rendez-vous à partir de 18h30 dans les locaux de l’Epitech Toulouse, chacun avec son ordinateur (non fourni), et on commence à jouer.

Et c’est bien sûr gratuit et ouvert à tous.

Un tableau trello a été initialisé pour tenter de s’organiser un peu. La liste des choses à faire n’est pas du tout figée, et les idées sont les bienvenues.

Les commentaires sont ouverts à tous, et l’accès sera donné à quiconque en fait la demande (façon communauté Jenkins :-)).

Goodies !

Grâce à l’aide de CloudBees, on a pas mal de goodies à offrir : stickers, badges, t-shirts et bobble-heads !

Les Bobble Heads Jenkins

9725573715 fa056b6652 n Nous en avons 2 ! Et ceux-ci seront offerts au deux premiers participants à voir leur pull-request envoyée pendant la soirée mergée.

Que faire pour (se) préparer ?

Si vous n’en avez pas, créez-vous un compte pour les services suivants :

Au niveau machine, idéalement, vous avez :

  • Git et Maven bien installés

  • Docker installé (natif sous Linux), ou via Docker Toolbox pour les autres OS

Des informations plus précises seront normalement données très bientôt aux inscrits via meetup quant à la préparation des machines.

Quel(s) langage(s) faut-il connaître ?

Idéalement, puisque Jenkins est écrit en Java, il serait souhaitable que vous connaissiez au moins les bases.

Toutefois, même si par exemple vos compétences sont plutôt côté Web, il y aura aussi des choses à faire, que ce soit jouer avec le nouveau site en préparation, ou ajouter une page web câblée sur certains fichiers json des statistiques.

Connaître au moins les bases de Git sera un gain de temps, mais ce n’est pas indispensable.

Récapitulatif

↧

Jenkins joins the Google Summer of Code 2016

March 3, 2016, 9:23 am
$
0
0

We are happy to announce that Jenkins project application has been accepted toGoogle Summer of Code 2016 (GSoC). Thanks to everybody who helped prepare the application and submitted project ideas!

We would like to invite students to join the Jenkins community and work together on the ongoing Jenkins 2.0 activities and other medium-term projects.

The student projects we are primarily interested in would improve the overall Jenkins user experience in a number of different aspects. This includes user interface changes and stability improvements but also major new features such as Pipeline as code.

The projects we've suggested revolve around all parts of the Jenkins project: core, plugins, website and our internal automation infrastructure. More details on what has been suggested can be found on the wiki which include:

  • Jenkins web interface improvements
  • "Update Center 2.0"
  • New generation of the fingerprinting engine
  • External workspace manager
  • Integration of Docker plugins with Jenkins 2.0 features
  • Plugins for Electronic Design Automation and Embedded tools integration
  • Improvements of the Support plugin
  • Improvements to Jenkins project infrastructure: core infra, website, plugin documentation and more

If you are a student:

  1. Check out the project ideas here.
  2. Select an interesting project idea or draft your own proposal.
  3. If you are not familiar with Jenkins, we highly recommend trying it out with one of your previous projects. You can also try available Jenkins features from the project ideas.
  4. Introduce yourself the community and start your project proposal discussion (see the guidelines here).
  5. Join us at GSoC office hours. We plan to have two meetings starting on March 7th.

If you want to be a mentor:

  • Feel free to team up with other mentors
  • We accept extra project proposals from mentors until March 9th.
↧

Jenkins 2.0-alpha-3 Preview Build has been released!

March 18, 2016, 12:23 pm
$
0
0

We just published the new Jenkins 2.0-alpha-3 preview build.

What’s new?

  • Jenkins is now secure out of the box: Administrators previously had to set up authentication and authorization while Jenkins was accessible to anyone on the same network. Now, Jenkins is protected out of the box, so that it is always safe from unauthorized access.

Unlock Jenkins

  • Plugin selection for setup: We refined the plugin selection on the setup dialog. You’ve always wondered why Jenkins does not install the Git Plugin by default? Now it does, along with a number of other plugins popular in the Jenkins community. We’re also including more plugins complementing the Pipeline plugin: ThePipeline Stage View plugin lets you quickly see what’s going on in your CD pipeline, and the GitHub Organization Folder will automatically scan your GitHub organization for repositories with Pipeline definitions (e.g. Jenkinsfile), and set up jobs for those.

Install suggested plugins

  • Redesigned job configuration forms: The job configuration form has been redesigned so its structure is visually clear when showing complex configuration forms. Additionally, the tabs on the top of the page show where you are, and can be used to quickly navigate between the different sections of the configuration form.

Job configuration

Download now!

Get Jenkins 2.0 alpha 3 now, and tell us what you think:

If you use Twitter, you can leave us some feedbackon Twitter.

Ourjenkinsci-users@ mailing list is also available for feedback inthis thread..

And of course, since this is a preview release, if you find any issues please report them on ourIssue Tracker to the JENKINS project.

We have a list ofknown issues on our wiki but if you’re not sure whether you’re experiencing a known issue or not, don’t hesitate to ask!

↧
Search

Introducing Jenkins Certification

March 18, 2016, 12:23 pm
$
0
0
This is a guest post by Francois Dechery, he works at CloudBees managing Customer Engagement/Support, Consulting and Training. He is also leading the Jenkins Certification program at CloudBees which has been discussed in some of our previous (1,2,3)governance meetings.

In the IT world, namely in software, "certification" is used in many different ways and for many different purposes. From very simple and light certifications to very heavy and complex ones. In the "light" category you can usually be certified on the basis of a short quiz at the end of an online training. At the other end of the spectrum, certifications are based on a proctored multiple-choice questionnaire-based exam and/or hands-on labs. In some industries, certifications are even more demanding. For instance, to become a Certified Public Accountant in the US, you have to pass a standard examination and, on top of this, each state/jurisdiction has its own set of education and experience requirements that individuals must meet.

Creating the Jenkins certification

When we started our internal discussions at CloudBees regarding a certification program for Jenkins, we were aware of this broad set of certification definitions. Therefore, our first goal was to define what type of certification we wanted to develop and for what purpose. We quickly agreed on the fact that it should be a professional-grade certification, whose purpose would be to provide a professional standard for the Jenkins ecosystem, benefiting both individuals and organizations, thanks to a common, respected and well-known body of knowledge and practice. "Professional" means that you have the expected level of skills and experience in order to leverage them in a professional environment, for example in enterprise projects or as a consultant.

Many members of the CloudBees team have firsthand experience with certification programs developed in other IT ecosystems such as telecoms (Cisco), infrastructure (Microsoft, Red Hat) or business applications (SAP), to name a few. This was definitely the type of professional certification we wanted to bring to Jenkins. We knew it would represent a substantial investment but we also knew that the whole Jenkins ecosystem would benefit. Whether at the overall community or individual level, as well as IT organizations, system integrators or recruiting firms looking for qualified Jenkins personnel.

I have had the privilege to supervise the creation and implementation of theJenkins Certification Program at CloudBees. The program is comprised of two certifications: "Certified Jenkins Engineer" (CJE) for Jenkins certification, and Certified CloudBees Jenkins Platform Engineer (CCJPE) for certification on the CloudBees Jenkins Platform.

We started by creating a Certification Advisory Board whose members are:Kohsuke Kawaguchi, Jenkins creator and CTO at CloudBees; Harpreet Singh, VP Products at CloudBees; Oliver Gondža, initially representing the Jenkins community; Jason Shawn, senior director DevOps at Ellucian, representing the CloudBees customer constituency; and Jose Alvarez, managing director at Zivra, representing the CloudBees partner ecosystem.

This dedicated group helped us first to create the certification blueprint which defines the main sections of the exam and their relative importance in the overall scoring. This blueprint also provides the high-level table of contents of the certificationstudy guides. They also helped to define the Jenkins Engineer profile that the certification assesses.

With this blueprint in hand, we put together a team of 40 Jenkins subject-matter experts (SMEs), mostly from CloudBees with a few from partners. Together they worked for several months on the creation of hundreds of exam questions, doing iterative peer reviews, filtering out any irrelevant or ambiguous questions and narrowing down the pool of questions to the best questions for each section. All this, plus a thorough analysis and balancing exercise to make sure the level of difficulty was evenly distributed across each section of the exam.

The big lesson from the exam creation experience is that creating a professional-grade exam is hard! And it requires very specific experience. In short, being a subject-matter expert is definitely not enough and we’re glad to have collaborated with Prometric's certification specialists who guided us through this process. The result is definitely worth the effort. Either of the two certifications offered within the Jenkins Certification Program are truly what we would consider "professional-grade certifications."

What does certification get you?

Getting certified means being recognized for your skills and experience as a Jenkins professional. However, like any exam-based recognition, its actual value depends on three criteria: the level of difficulty of the exam, its quality and its integrity.

As far as difficulty is concerned, it is clear that not everyone will pass and that is expected from a professional-grade certification, as mentioned earlier. We have definitely created an exam that is demanding. It does not only measure your theoretical knowledge of Jenkins but also your hands-on practical experience. To ensure its quality, we have applied best-industry practices regarding the exam’s creation and review process, working with certification specialists. It includes the weighing of questions, the distribution of easy, medium and difficult ones, the removal of any ambiguous wording, as well as alpha and beta final test procedures, in order to only keep the most appropriate questions. We are also putting in place a formal maintenance process to capture any "bug" in the exam and adapt the questions to Jenkins evolutions over time. Last but not least, we ensure the exam’s integrity by working with Prometric for the administration of exams. Tests are taken in fully secured and proctored test rooms, without any access to any human or electronic resource and without any doubt about who takes the test. Thanks to Prometric’s hundreds of test centers around the world, this integrity is ensured in any location.

Beyond this external recognition, getting certified is also a process that lets you take a step back from your day-to-day practice of Jenkins and assess your skills and knowledge. You start this reassessment process by reading the Study Guides for the certifications. Then, by taking the test itself, you can identify your strengths and weaknesses in a very practical way. In short, a certification gives you a measurable goal to achieve.

Click here for more information on the Jenkins Certification Program by CloudBees.

↧
↧

Jenkins 2.0 community test fest!

March 21, 2016, 1:49 pm
$
0
0

The beta release of Jenkins 2.0 is rapidly approaching! The development team is working hard to find and squash as many bugs as possible, but do you know what would make that effort even more successful? You! A big part of Jenkins’s power lies in its extensive flexibility, but that flexibility poses challenges to testing. In short, it’s difficult for the core team to test in all the myriad environments and with all the different workflows that Jenkins users have. To give just one example, users of Jenkins on Windows often uncover Windows-specific issues that are missed during development.

That’s where you come in! The Jenkins team has organized a Test Fest to take place all day on Monday, March 21st, wherever you are. We encourage you to download the lastest alpha release of Jenkins 2.0, start it up, and configure it in the way you would your production Jenkins installation. Try out your usual workflows, install those plugins you just can’t live without, and let us know about any issues you encounter. You can file them in the Jenkins JIRA, with the label 2.0, or you can discuss them with us in the #jenkins-testfest IRC channel on the Freenode IRC network (connect to irc.freenode.org). We encourage you to hang out with us on IRC regardless; it’ll be an all-day party!

Thanks in advance for joining us, and for supporting Jenkins!

↧

vJAM: Virtual Jenkins Area Meetup

March 21, 2016, 1:49 pm
$
0
0

Over the past few months, I’m happy to say, the number of Jenkins Area Meetups (JAMs) has grown tremendously! The excitement around JAMs has gotten us thinking about something larger, something more globally focused. That led us to createvJAM, an online Jenkins Area Meetup, where we can share what we’re learning together. The effort will be spear-headed by long time Jenkins contributor, R. Tyler Croy.

The key goals for the Virtual Jenkins Area Meetup are:

  1. Connect the global Jenkins user and developer community.

  2. Help spread the latest and greatest best practices.

  3. Support other JAMs by offering another, broader, audience for speakers and organizers

vJAM, originally inspired by Virtual JUG, will supplement local JAMs but nothing virtual can replace the value of talking with other Jenkins users over pizza and drinks.

We’re currently working on the agenda for the first vJAM, which will be posted to this Meetup group, so be sure to sign up if you’re interested in participating!

If you’re interested in creating your own local Jenkins Area Meetup, read this page for more details.

↧

Jenkins 2.0 beta released

March 24, 2016, 12:24 pm
$
0
0

We released the Jenkins 2.0 beta earlier today. Download it here and try it!

Besides a number of bug fixes and minor improvements, the following changes are new since the last alpha preview release:

Redesigned "New Item" page

We redesigned the "New Item" page. Item types now have icons to be more visually distinctive.

Additionally, item types can now define a category they belong to (such as "Project" or "Folder"). Once the complexity of the "New Item" page reaches a certain threshold, the item types will be grouped into categories to be easier to find. However, for now, it's unlikely that you will see these categories, as support for this mechanism will need to be added in plugins. This is a new API in core, and we invite plugin developers to support it to make Jenkins easier to use for users with a large number of item types. It doesn't even require raising the minimum supported Jenkins version.

Separate configuration page for tools

The length and complexity of the Configure Jenkins page once a few dozen plugins are installed made it unnecessarily difficult to use. To improve that we're moving the tools configuration (Git, Maven, Gradle, Ant, etc.) out of that page, into the new Global Tools Configuration.

Upgrade notice and plugin installer

The Pipeline plugin suite is a big part of Jenkins 2. Over the past few weeks, open-source plugins adding support for visualization (Pipeline Stage View), automatic GitHub project creation (GitHub Branch Source Plugin) and Bitbucket project creation (Bitbucket Branch Source Plugin) have been released. However, when upgrading from Jenkins 1.x, users weren't even given any information on these features.

To address this, users upgrading from Jenkins 1.x will now be shown a banner when they first log into Jenkins as administrator, offering them to install the suite of Pipeline plugins.

↧

Important notice regarding usage statistics

March 31, 2016, 3:00 pm
$
0
0

A bug was introduced in Jenkins versions 1.645 and 1.642.2 which caused Jenkins to sendanonymous usage statistics, even if the administrator opted-out of reporting usage data in the Jenkins web UI.

If you are running one of the affected versions, the best/easiest solution is to upgrade. The bug does not affect Jenkins 1.653 or newer, or Jenkins LTS 1.642.4 or newer.

If you cannot upgrade, it is possible to immediately disable submission of usage statistics by running the following script in "Manage Jenkins » Script Console":

    hudson.model.UsageStatistics.DISABLED = true

This will immediately disable usage data submission until you restart Jenkins. To make this permanent, change your Jenkins startup script so it passes a system property to the java process:

    java -Dhudson.model.UsageStatistics.disabled=true -jar …/jenkins.war

For information how to do this when using one of the installers/packages, see the installer/package documentation here.

To verify that usage stats submission is disabled, run the following script in "Manage Jenkins » Script Console" and confirm the result is true:

    println hudson.model.UsageStatistics.DISABLED

We have much more information about the issue and our usage statistics processin our wiki.

While we do not consider this a security advisory, if you are a Jenkins administrator we highly recommend subscribing to ourjenkinsci-advisories@ mailing list.
↧
↧

March 2016 St. Petersburg Jenkins Meetup Report

April 2, 2016, 4:34 pm
$
0
0

On March 10th we have conducted the second Jenkins meetup in Saint Petersburg, Russia. The meetup topic was "Jenkins and Continuous Delivery". We had 3 talks addressing various aspects of Jenkins usage in this area.

stpetersburg butler 0

Talks

  1. Introduction slides [ru]

  2. Jenkins 2.0 and Pipeline-as-Code

  3. Continuous Delivery for Documentation

  4. Continuous Delivery with Jenkins at ZeroTurnaround

We also had a long Jenkins afterparty. Starting from the next meetup we hope to make this part more official.

Acknowledgments

The event has been organized with the help fromYandex andCloudBees.

More Jenkins meetups

If you want to organize a Jenkins meetup in St. Petersburg or to be a speaker there, please contact us via theMeetup discussions page

Regarding other areas, check out whereJenkins Area Meetups (JAMs) are located in the world.

Don’t see a JAM in your area? Why not start your own, find out how.

↧

Jenkins 2.0 Release Candidate available!

April 9, 2016, 8:19 am
$
0
0

Those who fervently watch thejenkinsci-dev@ list, like I do, may have caught Daniel Beck's email today which quietly referenced a significant milestone on the road to 2.0 which has been reached: the first 2.0 release candidate is here!

The release candidate process, in short, is the final stabilization and testing period before the final release of Jenkins 2.0. If you have the cycles to help test, please download the release candidate and give us your feedback as soon as possible!

The release candidate process also means that changes targeting release after 2.0 can start landing in the master branch, laying the groundwork 2.1 and beyond.

I pushed the merge to master. So anything targeting 2.1+ can be now proposed in pull requests to that branch.

Anything happening on 2.0 branch will be limited to critical fixes for the 2.0 release specifically.

— Daniel Beck

Compared to the2.0 beta release, the first release candidate has a number of fixes for issues discovered in the alpha and beta process. Most notable perhaps is the stabilization of a system property which configuration management tools, like Puppet/Chef/Ansible/etc, can use to suppress the user-friendly Getting Started wizard. Since users of those tools have alternative means of ensuring security and correctness of their Jenkins installations, the out-of-the-box experience can be skipped.

Based on ourrough timeline this gives us a couple weeks to test the release candidates and get ready for a big exciting release of 2.0 at the end of April!

↧
Search

Security fixes in Script Security Plugin and Extra Columns Plugin

April 11, 2016, 7:54 pm
$
0
0

The Script Security Plugin and the Extra Columns Plugin were updated today to fix medium-severity security vulnerabilities. For detailed information about the security content of these updates, see the security advisory.

Subscribe to the jenkinsci-advisories mailing list to receive important notifications related to Jenkins security.

↧

Run Your API Tests Continuously with Jenkins and DHC

April 11, 2016, 7:54 pm
$
0
0
This is a guest post by Guillaume Laforge. Well known for his contribution to the Apache Groovy project, Guillaume is also the "Product Ninja and Advocate" of Restlet, a company focusing on Web APIs: with DHC (an API testing client),Restlet Studio (an API designer),APISpark (an API platform in the cloud), and the Restlet Framework open source project for developing APIs.

Modern mobile apps, single-page web sites and applications, are more and more relying on Web APIs, as the nexus of the interaction between the frontend and the backend services. Web APIs are also central to third-party integration, when you want to share your services with others, or when you need to consume existing APIs to build your own solution on top of their shoulders.

With APIs being a key element of your architecture and big picture, it’s obviously important to assess that this API is functioning the way it should, thanks to proper testing. Your framework of choice, regardless of the technology stack or programming language used, will hopefully offer some facilities for testing your code, whether in the form of unit tests, or ideally with integration tests.

Coding Web API tests

From a code perspective, as I said, most languages and frameworks provide approaches to testing APIs built with them. There’s one I wanted to highlight in particular, which is one developed with a DSL approach (Domain-Specific Language), using the Apache Groovy programming language, it’s AccuREST.

To get started, you can have a look at the introduction, and the usage guide. If you use the contract DSL, you’ll be able to write highly readable examples of requests you want to issue against your API, and the assertions that you expect to be true when getting the response from that call. Here’s a concrete example from the documentation:

GroovyDsl.make {
    request {
        method 'POST'
        urlPath('/users') {
            queryParameters {
                parameter 'limit': 100
                parameter 'offset': containing("1")
                parameter 'filter': "email"
            }
        }
        headers {
            header 'Content-Type': 'application/json'
        }
        body '''{ "login" : "john", "name": "John The Contract" }'''
    }
    response {
        status 200
        headers {
            header 'Location': '/users/john'
        }
    }
}

Notice that the response is expected to return a status code 200 OK, and a Location header pointing at /users/john. Indeed, a very readable way to express the requests and responses!

Tooling to test your APIs

From a tooling perspective, there are some interesting tools that can be used to test Web APIs, like Paw (on Macs),Advanced REST client,Postman orInsomnia.

But in this article, I’ll offer a quick look at DHC, a handy visual tool, that you can use both manually to craft your tests and assertions, and whose test scenarios you can export and integrate in your build and continuous integration pipeline, thanks to Maven and Jenkins.

At the end of this post, you should be able to see the following reporting in your Jenkins dashboard, when visualising the resulting API test execution:

Export a scenario

Introducing DHC

DHC is a Chrome extension, that you caninstall from the Chrome Web Store, in your Chrome browser. There’s also an online service available, with some limitations. For the purpose of this article, we’ll use the Chrome extension.

DHC interface

In the main area, you can create your request, define the URL to call, specify the various request headers or params, chose the method you want to use, and then, you can click the send button to issue the request.

In the left pane, that’s where you’ll be able to see your request history, create and save your project in the cloud, or also set context variables.

The latter is important when testing your Web API, as you’ll be able to insert variables like for example {localhost} for testing locally on your machine or {staging} and {prod} to run your tests in different environments.

In the bottom pane, you have access to actual raw HTTP exchange, as well as the assertions pane.

Again, a very important pane to look at! With assertions, you’ll be able to ensure that your Web API works as expected. For instance, you can check the status code of the call, check the payload contains a certain element, by using JSON Path or XPath to go through the JSON or XML payload respectively.

DHC interface

Beyond assertions, what’s also interesting is that you can chain requests together. A call request can depend on the outcome of a previous request! For example, in a new request, you could pass a query parameter whose value would be the value of some element of the JSON payload of a previously executed request. And by combining assertions, linked requests and context variables together, you can create full-blown test scenarios, that you can then save in the cloud, but also export as a JSON file.

Running a test scenario

To export that test scenario, you can click on the little export icon in the bottom left hand corner, and you’ll be able to select exactly what you want to export:

Export a scenario

Running your Web API tests with Maven

Now things become even more interesting, as we’ll proceed to using Maven and Jenkins! As the saying goes, there’s a Maven plugin for that! For running those Web API tests in your build! Even if your Web API is developed in another technology than Java, you can still create a small Maven build just for your Web API tests. And the icing on the cake, when you configure Jenkins to run this build, as the plugin outputs JUnit-friendly test reports, you’ll be able to see the details of your successful and failed tests, just like you would see JUnit’s!

Let’s sketch your Maven POM:

<projectxmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0http://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><groupId>com.example</groupId><artifactId>my-first-api-test</artifactId><version>1.2.3</version><build><plugins><plugin><groupId>com.restlet.dhc</groupId><artifactId>dhc-maven-plugin</artifactId><version>1.1</version><executions><execution><phase>test</phase><goals><goal>test</goal></goals><configuration><file>companies-scenario.json</file></configuration></execution></executions></plugin></plugins></build><pluginRepositories><pluginRepository><id>restlet-maven</id><name>Restlet public Maven repository Release Repository</name><url>http://maven.restlet.com</url></pluginRepository></pluginRepositories></project>

Visualizing Web API test executions in Jenkins

Once you’ve configured your Jenkins server to launch the test goal of this Maven project, you’ll be able to see nice test reports for your Web API scenarios, like in the screenshot in introduction of this article!

Next, you can easily run your Web API tests when developers commit changes to the API, or schedule regular builds with Jenkins to monitor an online Web API.

For more information, be sure to read the tutorial ontesting Web APIs with DHC. There are also some more resources like ascreencast, as well as theuser guide, if you want to learn more. And above all, happy testing!

↧
↧

Jenkins Community Survey Results

April 12, 2016, 12:38 pm
$
0
0
This is a guest post by Brian Dawson at CloudBees, where he works as a DevOps Evangelist responsible for developing and sharing continuous delivery and DevOps best practices. He also serves as the CloudBees Product Marketing Manager for Jenkins.

logo 128

Last fall CloudBees asked attendees at the Jenkins User Conference – US West (JUC), and other in the Jenkins community to take a survey. Almost 250 people did – and thanks to their input, we have results which provided interesting insights into how Jenkins is being used.

Back in 2012, at the time of the last community survey, 83% of respondents felt that Jenkins was mission-critical. By 2015, the percentage saying that Jenkins was mission-critical was 92%. Additionally, echoing the importance of Jenkins, 89% of respondents said their use of Jenkins had increased over the last year, while 11% said it had stayed the same. 0% said that it had decreased.

The trend in the industry over the last couple of years has been to adopt continuous delivery (CD), thus pushing automation further down the pipeline – from development all the way into production. Jenkins being an automation engine applicable to any phase of the software delivery lifecycle, is readily supporting this trend. Jenkins' extensible architecture and unparalleled plugin ecosystem enables integration with and orchestration of practically any tool in any phase of software delivery.

The trend towards adoption of CD is clearly reflected amongst the community: 59% of respondents are using Jenkins for continuous integration (CI), but an additional 30% have extended CI into CD and are manually deploying code to production. Finally, 11% are practicing continuous deployment – they have extended CI to CD and are deploying code automatically into production.

Another trend tied to the adoption of CD and DevOps is the frequent deployment of incremental releases to production. 26% of those respondents using continuous delivery practices are deploying code at least once per day. Another 37% are deploying code at least once per week.

In keeping with the move to CD, 30% of survey takers are already using the relatively new Pipeline plugin to automate their software delivery pipelines. Of those not using the Pipeline plugin, 79% plan to adopt it in the next 12 months.

Survey respondents are also using Jenkins for many different activities. 97% of survey takers use it for "build" – no surprise, since that is where Jenkins got its start - but 58% now also use it for their deployment.

2016 survey blog strongbutler

When the 2012 community survey was conducted, container technology was not as well understood as it is today, and many didn’t know what a “Docker” was. A short four years later, 96% of survey respondents who use Linux containers are using Docker. Container technology has seen impressive adoption and arguably is revolutionizing the way application infrastructure is delivered. When coupled with Jenkins as an automation engine, containers help accelerate software delivery by providing rapid access to lightweight environments. The Jenkins community has recognized and embraced the power of containers by providing plugins for Docker and Kubernetes.

The Jenkins improvements which survey respondents desired the most were quality/timely bug fixes, a better UI and more documentation/examples. Interestingly, Jenkins 2.0 - which is just about to officially launch, provides UI improvements and the new Jenkins.io website provides improved, centralized documentation.

2016 survey blog bb8

Finally, the respondents favorite Star Wars character was R2-D2, followed by Obi-Wan and Darth Vader. Yoda and Han Solo also got a fair amount of votes. The votes for Jar-Jar Binks and Jabba the Hutt left us puzzled. Notably, BB-8 had a write-in vote despite the fact the new Star Wars movie hadn’t been released yet.

As to where the community is headed, our prediction is that by the next Jenkins Community Survey:

  • More Jenkins users will have transitioned from just continuous integration to continuous delivery with some evening practicing continuous deployment

  • Pipeline plugin adoption and improvements will continue, leading to pipeline-as-code becoming an essential solution for automating the software (and infrastructure) delivery process

  • There will be a significant increase in use of the Docker plugin to support elastic Jenkins infrastructure and continuous delivery of containers using software development best practices

  • BB-8 will be the next favorite Star Wars character! <3

See you at Jenkins World, September 13-15, in Santa Clara, California!Register now for the largest Jenkins event on the planet in 2016 – and get the Early Bird discount. The Call for Papers is still open – so submit a talk and share your knowledge with the community about Jenkins.

↧

Google Summer of Code. Call for Mentors

April 12, 2016, 12:38 pm
$
0
0

As you probably know, Jenkins project has been accepted toGoogle Summer of Code 2016.

During last month we were working with students in order to discuss their project ideas and to review their application drafts. Thanks again to all students and mentors for your hard work during about ten office hours and dozens of other calls/chats!

Current status

  • We have successfully handled the student application period

  • We have received a bunch of good project proposals (mentors cannot disclose the number)

  • We have done the preliminary filtering of applications

  • GSoC mentors and organization admins have prepared the project slot application draft

Currently we are looking for mentors. We have a minimal required number for the current project slot application plan, but additional expertise would allow us to share the load and to provide more expertise to students.

If you want to be a mentor:

  1. Check out mentor requirements here.

  2. Check out the project ideashere.

    • Student application period is finished, so it is too late to propose project ideas for this year

    • You can join the mentor team for one of the mentioned projects

    • Hot areas: UI improvements, Fingerprints, External Workspace Manager

  3. Contact Google GSoC admins via jenkinsci-gsoc-org@googlegroups.com

↧

Automating test runs on hardware with Pipeline as Code

April 13, 2016, 9:46 am
$
0
0

In addition to Jenkins development, during last 8 years I’ve been involved into continuous integration for hardware and embedded projects. At JUC2015/London I have conducted a talk about common automation challenges in the area.

In this blog post I would like to concentrate on Pipeline (formerly known as Workflow), which is a new ecosystem in Jenkins that allows implementing jobs in a domain specific language. It is in the suggested plugins list in the upcoming Jenkins 2.0 release.

The first time I tried Pipeline two and half years ago, it unfortunately did not work for my use-cases at all. I was very disappointed but tried it again a year later. This time, the plugin had become much more stable and useful. It had also attracted more contributors and started evolving more rapidly with the development of plugins extending the Pipeline ecosystem.

Currently, Pipeline a powerful tool available for Jenkins users to implement a variety of software delivery pipelines in code. I would like to highlight several Pipeline features which may be interesting to Jenkins users working specifically with embedded and hardware projects.

Introduction

In Embedded projects it’s frequently required to run tests on specific hardware peripherals: development boards, prototypes, etc. It may be required for both software and hardware areas, and especially for products involving both worlds. CI and CD methodologies require continuous integration and system testing, and Jenkins comes to help here. Jenkins is an automation framework, which can be adjusted to reliably work with hardware attached to its nodes.

Area challenges

Generally, any peripheral hardware device can be attached to a Jenkins node. Since Jenkins nodes require Java only, almost every development machine can be attached. Below you can find a common connection scheme:

Connecting the external device

After the connection, Jenkins jobs could invoke common EDA tools via command-line interfaces. It can be easily done by a Execute shell build steps in free-style projects. Such testing scheme is commonly affected by the following issues:

  • Nodes with peripherals are being shared across several projects. Jenkins must ensure the correctness of access (e.g. by throttling the access).

    • In a single Freestyle project builds utilize the node for a long period. If you synthesize the item before the run, much of the peripheral utilization file may be wasted.

    • The issue can be solved by one of concurrency management plugins:Throttle Concurrent Builds, Lockable Resources orExclusions.

  • Test parallelization on multiple nodes requires using of multiple projects orMatrix configurations, so it causes job chaining again.

  • Hardware infrastructure is usually flaky. If it fails during the build due to any reason, it’s hard to diagnose the issue and re-run the project if the issue comes from hardware.

    • Build Failure Analyzer allows to identify the root cause of a build failure (e.g. by build log parsing).

    • Conditional Build Step andFlexible Publish plugins allow altering the build flow according to the analysis results.

    • Combination of the plugins above is possible, but it makes job configurations extremely large.

  • Tests on hardware peripherals may take much time. If an infrastructure fails, we may have to restart the run from scratch. So the builds should be robust against infrastructure issues including network failures and Jenkins master restarts.

  • Tests on hardware should be reproducible, so the environment and input parameters should be controlled well.

    • Jenkins supportscleaning workspaces, so it can get rid of temporary files generated by previous runs.

    • Jenkins provides support of slaves connected via containers (e.g.Docker) or VMs, which allow creating clean environments for every new run. It’s important for 3rd-party tools, which may modify files outside the workspace: user home directory, temporary files, etc.

    • These environments still need to be connected to hardware peripherals, which may be a serious obstacle for Jenkins admins

The classic automation approaches in Jenkins are based on Free-style and Multi-configuration project types. Links to various articles on this topic are collected on theHW/Embedded Solution pageEmbedded on the Jenkins website. Tests automation on hardware peripherals has been covered in several publications by Robert Martin, Steve Harris, JL Gray, Gordon McGregor, Martin d’Anjou, and Sarah Woodall. There is also a top-level overview of classic approaches made by me at JUC2015/London (a bit outdated now).

On the other hand, there is no previous publications, which would address Pipeline usage for the Embedded area. In this post I want to address this use-case.

Pipeline as Code for test runs on hardware

Pipeline as Code is an approach for describing complex automation flows in software lifecycles: build, delivery, deployment, etc. It is being advertised in Continuous Delivery and DevOps methodologies.

In Jenkins there are two most popular plugins:Pipeline and Job DSL. JobDSL Plugin internally generates common freestyle jobs according to the script, so it’s functionality is similar to the classic approaches. Pipeline is fundamentally different, because it provides a new engine controlling flows independently from particular nodes and workspaces. So it provides a higher job description level, which was not available in Jenkins before.

Below you can find an example of Pipeline scripts, which runs tests on FPGA board. The id of this board comes from build parameters (fpgaId). In this script we also presume that all nodes have pre-installed tools (Xilinx ISE in this case).

// Run on node having my_fpga label
node("linux && ml509") {
  git url:"http://github.com/oleg-nenashev/pipeline_hw_samples"
  sh "make all"
}

But such scenario could be also implemented in a Free-style project. What would we get from Pipeline plugin?

Getting added-value from Pipeline as code

Pipeline provides much added-value features for hardware-based tests. I would like to highlight the following advantages:

  • Robustness against restarts of Jenkins master.

  • Robustness against network disconnects. sh() steps are based on theDurable Task plugin, so Jenkins can safely continue the execution flow once the node reconnects to the master.

  • It’s possible to run tasks on multiple nodes without creating complex flows based on job triggers and copy artifact steps, etc. It can be achieved via combination of parallel() and node() steps.

  • Ability to store the shared logic in standalone Pipeline libraries

  • etc.

First two advantages allow to improve the robustness of Jenkins nodes against infrastructure failures. It is critical for long-running tests on hardware.

Last two advantages address the flexibility of Pipeline flows. There are also plugins for freestyle projects, but they are not flexible enough.

Utilizing Pipeline features

The sample Pipeline script above is very simple. We would like to get some added value from Jenkins.

General improvements

Let’s enhance the script by using several features being provided by pipeline in order to get visualization of stages, report publishing and build notifications.

We also want to minimize the time being spent on the node with the attached FPGA board. So we will split the bitfile generation and further runs to two different nodes in this case: a general purpose linux node, and the node with the hardware attached.

You can find the resulting Pipeline script below:

// Synthesize on any nodedef imageId=""
node("linux") {
  stage "Prepare environment"
  git url:"http://github.com/oleg-nenashev/pipeline_hw_samples"// Construct the bitfile image ID from commit ID
  sh 'git rev-parse HEAD > GIT_COMMIT'
  imageId= "myprj-${fpgaId}-" + readFile('GIT_COMMIT').take(6)

  stage "Synthesize project"
  sh "make FPGA_TYPE=$fpgaId synthesize_for_fpga"/* We archive the bitfile before running the test, so it won't be lost it if something happens with the FPGA run stage. */
  archive "target/image_${fpgaId}.bit"
  stash includes: "target/image_${fpgaId}.bit", name: 'bitfile'
}/* Run on a node with 'my_fpga' label.
In this example it means that the Jenkins node contains the attacked FPGA of such type.*/
node ("linux && $fpgaId") {
  stage "Blast bitfile"
  git url:"http://github.com/oleg-nenashev/pipeline_hw_samples"def artifact='target/image_'+fpgaId+'.bit'
  echo "Using ${artifact}"
  unstash 'bitfile'
  sh "make FPGA_TYPE=$fpgaId impact"/* We run automatic tests.
  Then we report test results from the generated JUnit report. */
  stage "Auto Tests"
  sh "make FPGA_TYPE=$fpgaId tests"
  sh "perl scripts/convertToJunit.pl --from=target/test-results/* --to=target/report_${fpgaId}.xml --classPrefix=\"myprj-${fpgaId}.\""
  step([$class:"JUnitResultArchiver", testResults:"target/report_${fpgaId}.xml"])

  stage "Finalization"
  sh "make FPGA_TYPE=$fpgaId flush_fpga"
  hipchatSend("${imageId} testing has been completed")
}

As you may see, the pipeline script mostly consists of various calls of command-line tools via the sh() command. All EDA tools provide great CLIs, so we do not need special plugins in order to invoke common operations from Jenkins.

Makefile above is a sample stuff for demo purposes. It implements a set of unrelated routines merged into a single file without dependency declarations. Never write such makefiles.

It is possible to continue expanding the pipeline in such way.Pipeline Examples contain examples for common cases: build parallelization, code sharing between pipelines, error handling, etc.

Lessons learned

During last 2 years I’ve been using Pipeline for Hardware test automation several times. The first attempts were not very successful, but the ecosystem has been evolving rapidly. I feel Pipeline has become a really powerful tool, but there are several missing features. I would like to mention the following ones:

  1. Shared resource management across different pipelines.

  2. Better support of CLI tools.

    • EDA tools frequently need a complex environment, which should be deployed on nodes somehow.

    • Integration withCustom Tools Plugin seems to be the best option, especially in the case of multiple tool versions (JENKINS-30680).

  3. Pipeline package manager (JENKINS-34186)

    • Since there is almost no plugins for EDA tools in Jenkins, developers need to implement similar tasks at multiple jobs.

    • A common approach is to keep the shared "functions" in libraries.

    • Pipeline Global Library andPipeline Remote Loader can be used, but they do not provide features like dependency management.

  4. Pipeline debugger (JENKINS-34185)

    • Hardware test runs are very slow, so it is difficult to troubleshoot and fix issues in the Pipeline code if you have to run every build from scratch.

    • There are several features in Pipeline, which simplify the development, but we still need an IDE-alike implementation for complex scripts.

Conclusions

Jenkins is a powerful automation framework, which can be used in many areas. Even though Jenkins has no dedicated plugins for test runs on hardware, it provides many general-purpose "building blocks", which allow implementing almost any flow. That’s why Jenkins is so popular in the hardware and embedded areas.

Pipeline as code can greatly simplify the implementation of complex flows in Jenkins. It continues to evolve and extend support of use-cases. if you’re developing embedded projects, consider Pipeline as a durable, extensible and versatile means of implementing your automation.

What’s next?

Jenkins automation server dominates in the HW/Embedded area, but unfortunately there is not so much experience sharing for these use-cases. So Jenkins community encourages everybody to share the experience in this area by writing docs and articles for Jenkins website and other resources.

This is just a a first blog post on this topic. I am planning to provide more examples of Pipeline usage for Embedded and Hardware tests in the future posts. The next post will be about concurrency and shared resource management in Pipelines.

I am also going to talk about running tests on hardware at theupcoming Automotive event in Stuttgart on April 26th. This event is being held byCloudBees, but there will be several talks addressing Jenkins open-source as well.

If you want to share your experience about Jenkins usage in Hardware/Embedded areas, consider submitting a talk for theJenkins World conference or join/organize aJenkins Area Meetup in your city. There is also aJenkins Online Meetup.

Related articles and events:

Pipeline:

↧
Viewing all 1088 articles
Browse latest View live
Search