Quantcast
Channel: Jenkins Blog
Viewing all 1088 articles
Browse latest View live
↧

Jira upgrade for the Jenkins project

December 18, 2020, 2:13 pm
$
0
0
jira logo

The Jenkins project has used Jira to track issues for many years. Jenkins core, Jenkins modules, Jenkins infrastructure, and many Jenkins plugins manage their issue reports with our Jira server.

Jira helps the Jenkins project manage issues and tasks related to over 250 000 Jenkins installations. It tracks bugs, enhancement requests, tasks, and security issues. It is used regularly by users around the world.

We’re grateful for the long-standing contribution that Atlassian provides by donating the Jira license to the Jenkins project. We’re grateful to the Oregon State University Open Source Lab for their donation of equipment and bandwidth to host the server.

Upgrade Timeline

We were running Jira 7.13 and had been managing that installation for a few years. Atlassian announced that Jira 7.13 would end its support life on November 28, 2020. We needed to upgrade from Jira 7.13 to a more recent version of Jira. As part of our membership in the Continuous Delivery Foundation, a Linux Foundation initiative, we could use their project services team to manage our Jira server. We decided to move from hosting our own Jira server to having the Jira experts at the Linux Foundation host it.

The upgrade timeline looked like this:

  • November 2019 - Infrastructure team begins discussions about the November 2020 end of support for Jira 7.13

  • August 2020 - First conversations with Linux Foundation to host Jira for the Jenkins project. Draft of the upgrade plan assembled and shared with the community

  • September 2020 - Schedule for testing week and final transition week proposed. Authentication options evaluated and selected

  • October 2020 - Test upgrade performed and tested

  • November 2020 - Final upgrade completed and verified

Confronting the Complications

Initial discussions between the Jenkins infrastructure team and the Linux Foundation identified complications related to authentication and SSL certificates. We planned, negotiated, and tested our assumptions throughout the project.

Authentication

Jira servers at the Linux Foundation typically use Linux Foundation accounts for user access. Unfortunately, the Jenkins LDAP database includes over 100,000 users and for many of them, Linux Foundation username doesn’t correspond to Jenkins account username. It was not feasible to transition 100,000 user accounts from the Jenkins LDAP database to the Linux Foundation accounts system and still complete the Jira upgrade before the November 28, 2020 deadline.

The Linux Foundation Project Services team evaluated authentication alternatives and confirmed that they could use the Jenkins LDAP server. Using the Jenkins LDAP server spared us from two transitions, LDAP and Jira, and kept the project timeline feasible.

SSL Certificates

Jira servers at the Linux Foundation use Let’s Encrypt to generate SSL certificates for HTTPS. The Linux Foundation uses the DNS method to obtain SSL certificates. Unfortunately, the Jenkins project uses the HTTP method to obtain SSL certificates.

Thankfully, Olivier Vernin of the Jenkins project and Anton Baranov of the Linux Foundation found a solution. They created an ACME record in the Jenkins DNS server and pointed the issues.jenkins.io DNS record at the new Linux Foundation Jira server.

Building the Prototype

Anton Baranov created a prototype Jira server, restored an older Jenkins Jira backup, and upgraded it to Jira 8.13. That first restore detected that we had not provided the Jira attachments or the Jira avatars. That attachments and avatars added multiple gigabytes to the initial backup data and were vital to complete the update.

Testing the Upgrade

A group of volunteers including Jenkins users, security team members, and infrastructure team members tested the upgrade during the week of October 26, 2020. The tests confirmed that authentication worked as expected and that the Jira prototype was functioning as expected.

We thank the test team, including:

  • Daniel Beck

  • Tim Jacomb

  • Olivier Vernin

  • Mark Waite

The tests included:

  • Creating and routing issues

  • Commenting on issues

  • Viewing dashboards with the expected content

  • LDAP settings

  • Email notification

The tests detected minor issues that Anton was able to correct in preparation for the final upgrade. The testing team agreed that the tests were successful.

Deploying the Upgrade

Olivier Vernin announced the final upgrade by email to the Jenkins infra list with details of the changes happening during the upgrade. Monday, November 9, 2020, the final backup of the existing Jira server was copied into the new Linux Foundation server.

The final upgrade encountered issues that we had not seen during the initial tests. The "bumps and bruises" from the unexpected issues were resolved by Anton Baranov as he used a multi-step upgrade process. The steps included:

  • Restore the earlier backup to Jira 7.13

  • Restore the most recent backup

  • Upgrade to Jira 8.13

  • Install avatars, attachments, and other images

  • Update DNS entries to point to the new Jenkins Jira server

Lessons from the Upgrade

Lessons were related to timing, estimation, and communication.

Scheduling the Upgrade

The test upgrade started the week of October 19, 2020. It took several days longer than originally expected. Thankfully, we had allowed an extra week between the test upgrade and the production upgrade.

The originally announced schedule for the final upgrade was intentionally placed in a week that would not include a long term support release. That reduced the risk of disruption if the upgrade took longer than required or failed and we had to roll back.

Estimating the Work

Discussions with the Jenkins project Jira administrators and the Linux Foundation Jira experts provided very reasonable estimates of time to complete the work. We intentionally allowed additional time between first test and final upgrade. We needed that additional time and used it well as the testing week.

Communicating the Plan

The distributed nature of the Jenkins project makes communication challenging for major changes. We communicated plans at various stages but still found occasions where the communication was insufficient. In this case, the adage held true that it is, "impossible to communicate too much".

Thanks for your patience during the upgrade and thanks to the Linux Foundation for administering the Jenkins Jira server.

↧
Search

Google Summer of Code 2020 summary

December 22, 2020, 7:13 pm
$
0
0

With the mentor summit and the project retrospectives finished in October, now we can call Google Summer of Code 2020 officially over in the Jenkins community. On behalf of the Jenkins org team, we would like to thank all participants: students, mentors, applicants, and dozens of other contributors who participated in the project this year. Google Summer of Code would not be successful without the active participation of the Jenkins community.

Jenkins GSoC

If you follow the Jenkins blog, you may have already seen many GSoC 2020 articles created by the project teams. Here I would like to focus on the key highlights from the project.

Projects

In 2020 we had seven students working in the Jenkins mentoring organization. We had 6 projects focused on Jenkins and one project focused on Jenkins X. As usual, in GSoC we focused on problems important to the Jenkins users and community members. The projects delivered highly anticipated new features and key architecture changes needed for long-term evolution of Jenkins.

Here are the projects we had this year:

Please refer to the project pages for more information, links to the blog posts, and project demos. Let’s focus on the results instead. This is the first ever time in Jenkins when all GSoC students have reached the final evaluation and successfully passed it. It was an incredible effort by all the project members and, most importantly, by the students. Thanks a lot to them!

GSoC 2020 students.

Thanks a lot to all mentors who were guiding students during their projects:Zhao Xiaojie,Kristin Whetstone,Parichay Barpanda,Martin d’Anjou,Oleg Nenashev,Andrey Falko,Mike Cirioli,Mark Waite,Francisco Fernandez,Justin Harringa,Omkar Deshpande,Ulli Hafner,Tim Jacomb,Kara de la Marck,James Strachan,Neha Gupta,Oscar Medina,Nikhil Da Rocha,Sahil Kalra,Bruno P. Kinoshita,Ioannis Moutsatsos,Marky Jackson,Shivay Lamba, andNext Turn.

GSoC 2020 Org admin team:Kara de la Marck,Martin d’Anjou,Marky Jackson,Oleg Nenashev.

Events

Thanks to the GSoC organization stipend from Google and other donations, the Jenkins project usually provides travel grants to successful students so that they can visit a major community event, meet their mentors and community members in person, and present their work there. Here are some notes about the GSoC 2019 travel. Unfortunately this year it was not possible, and GSoC went completely virtual this year.

Online meetups

In August we organized Jenkins Online Meetups where students have presented their projects (part 1, part 2). You can find recordings of these presentations in this playlist on the Jenkins YouTube channel.

DevOps World

This year CloudBees, one of the Jenkins corporate sponsors, invited all students to participate in the DevOps World virtual conference on September 23-25. GSoC students did lighting talks about their projects, attended other conference talks, and joined the Continuous Delivery Foundation booth which represented the project at the conference. You can find recordings of the talks and all materials here. Although the conference was in September, the talks were pre-recorded in early August. Please refer to the Jenkins online meetup recordings for the recent versions.

GSoC Mentor summit

This is a regular gathering for Google Summer of Code mentors and org admins where they share their experiences about GSoC, outreach programs, community management, and tools. Usually it is organized as a multi-day unconference after the end of GSoC, with 2-3 representatives from each project. It has been a great learning experience to participate in it. This year it was a single-day virtual event, and all mentors were able to attend.Shivay Lamba, one of the GSoC 2020 mentors, also did a lightning talk about the GSoC projects he was working on in Jenkins and CNCF (slides).

Swag

All Google Summer of Code students and mentors get swag from Google. This year, Contrinuos Delivery Foundation (CDF) has sponsored swag for 50 most active GSoC participants: all students, mentors, and many other contributors who participated and helped the projects to succeed. This is the third year when the Jenkins organization sends extra GSoC swag, In the previous years the swag logistics was one of the most challenging tasks for org admins during the entire project. and we highly appreciate help from CDF with this part. As DevOps World presenters, the students have also received special edition speaker swag from CloudBees.

GSoC swag by the Jenkins project

Thanks a lot to Google, Contrinuos Delivery Foundation (CDF) and CloudBees!

Retrospective

After completion of the coding phases, org admins have reached out to all GSoC 2020 participants to gather their feedback and suggestions. We also recommended that project teams hold their own retrospective meetings. Such information is instrumental to continuously improving GSoC in the Jenkins community. We thank all contributors who shared their feedback!

The organization-wide retrospective was organized as a survey and a series of retrospective meetings. You can find aggregated results in this Google Doc. Overall, we received very positive feedback from students and mentors. The GSoC framework in Jenkins has matured significantly during the previous years. The effort we invested to create guidelines and recommendations for all parties helped a lot because all the expectations were known in advance. As usual, there is much to improve, especially with regards to the community bonding phases and cross-project communications. We are processing the feedback, and we will expand our documentation and the contributor onboarding plans next year.

Some personal notes

I have been involved in leading and coordinating Google Summer of Code in open source projects since 2016. This year I visited the GSoC stand at FOSDEM and met a few organizers and former students. A few days after, I proposed participating in GSoC at the Jenkins contributor summit in Brussels, and several contributors supported this idea. We spent several hours to create the first Jenkins GSoC page and brainstorm on project ideas. We submitted our application and were accepted. Thanks a lot to the Google team that gave us a chance!

It is great to work with the students and see how they explore the open source community and grow professionally as engineers. It is also awesome to see how some of them stay in the project and keep contributing, including becoming plugin maintainers and GSoC mentors. But, for me, Google Summer of Code is not just about mentoring. It also helps a lot with the community bonding…​ for the existing community like Jenkins which has a lot of isolated sub-communities in plugins. Many maintainers work alone, and it can be quite lonely working to maintain a plugin without feedback, developer ideas, and user interactions. When plugin contributors become project mentors, they join the wider community effort and work in teams. In many cases they start contributing to the organization-wide activities and goals, and it grows the "backbone" of the Jenkins community. Like other community-driven projects, we need such backbone to scale the community and onboard more contributors to the countless Jenkins components. So far it works really well and GSoC excels among outreach programs in this regard.

I would like to thank the Google Open Source team, students and all Jenkins community members for the great Google Summer of Code this year. We also thank the Continuous Delivery Foundation for their help to recognize contributors and allow organization administrators to focus on projects. Last but not least, I would like to thank the Jenkins org admin team:Martin d’Anjou,Marky Jackson, andKara de la Marck. This was a crazy year for everyone. Regardless of that, the org admins stepped up and took responsibility for students and mentors involved in the project, with a serious time commitment. Not all work by the organizers is publicly visible (applications, project selection, resolving conflicts), but this work is essential to the project’s success. Thanks a lot to org admins and mentors who helped with the administrative tasks this year!

What’s about GSoC 2021?

Yes, we plan to participate in Google Summer of Code 2021. The application period for organizations will start in a few months, but we have already started preparing for the next GSoC session. We are looking for mentors, org admins and project ideas. Please contact us if you are interested!

We invite potential students to start exploring the project and the available project ideas. Original ideas are always welcome in the project, and starting early is a great opportunity to get introduced to the Jenkins community, collect more information about the problem areas, and to create a good proposal. "Start early" is the most popular recommendation from GSoC 2020 participants to future GSoC students, and we encourage you to follow this advice!

↧
↧

2020 A Year Like No Other

January 13, 2021, 1:13 am
$
0
0

The Jenkins community congratulates all users and contributors with the New Year! Let’s take a look at some changes this year. We would like to thank all awesome Jenkins users and contributors who have been with us during this year.

NewYear

Highlights

Some of the key highlights:

  • Hundreds of first-timer contributors joined the community

  • Major UI/UX improvements in the Jenkins core, including the landing page, plugin manager, dark theme, and read-only configurations support

  • Outreach programs like Google Summer of Code (7 projects), Google Season of Docs, UI/UX hackfest, etc.

  • Public roadmap for the project

  • Terminology changes in the project, new Code of Conduct

  • Technical debt cleanup: XStream unforking, Acegi Security replacement, etc.

  • Continued evolution of the plugin ecosystem, especially in the area of Cloud Native solutions and tool integrations

  • Continued documentation cleanup, great progress with plugin documentation migration

  • Graduation in the Continuous Delivery Foundation

Jenkins User Interface and User Experience

This year there were many activities around Jenkins user experience and long-anticipated user interface changes. This is a coordinated effort being led by the User Experience SIG, and by many contributors to the project. Key project highlights:

  • Look & Feel updates of the Jenkins Web UI, including styling rework, new typography and layouts

  • Major rework of plugin management UI/UX

  • Dark theme for Jenkins

  • Accessibility improvements

  • Support for read-only configuration pages

In May we also organized a Jenkins UI/UX hackfest where we worked on some key stories improving user experience.

Jenkins security

In 2020 the Jenkins security team has released 19 advisories for the Jenkins core, plugins and other components. In total 198 vulnerabilities were fixed, and 72 plugin vulnerabilities were announced without a fix at the time of advisory publishing. As a project, we are receiving a continuous flow of new reports and continue to provide corrections. Cross-site scripting (XSS) vulnerabilities were the most popular type this year, followed by unprotected credentials.

There have also been developer tooling improvements, including GitHub CodeQL evaluation for targeted security issues search and Find-Sec-Bugs adoption for static analysis of the plugin and Jenkins core code. Along with wider adoption of Dependabot and automated dependency scanning on GitHub.

Documentation

Jenkins Documentation SIG is working on creating more documentation for Jenkins on different platforms, including cloud platforms. Jenkins on Kubernetes was one of the key stories this year for the SIG, along with documentation migration to jenkins.io and wider adoption of Documentation-as-code in plugins. 95% of the 200 most installed Jenkins plugins have moved to "documentation as code" or have a pending pull request with updates. In total, almost 600 plugins have already been migrated. There were major updates in Jenkins documentation on jenkins.io, with a lot of content being moved from the old Jenkins Wiki.

2020 was the first year when Jenkins participated in Google Season of Docs (GSoD). This program brings together open-source and technical writers communities for the benefit of both. This year’s student, Zainab Abubakar, did an amazing job documenting Jenkins on Kubernetes. Now Jenkins users can find official documentation about deploying and scaling Jenkins in Kubernetes. See the project report by Zainab here.

Jenkins Release Automation

The Jenkins project has delivered weekly and long term support releases since it was formed in 2011. Those releases were delivered by Kohsuke Kawaguchi from his release infrastructure.

Beginning in April 2020, those releases are delivered by the new release automation setup. It is hosted within the Jenkins’ Kubernetes cluster, with fully automated management and continuous delivery of services within the setup. We transitioned to new build processes, new code signing certificates, and new release automation jobs. Thanks to Olivier Vernin and all Infrastructure sub-project contributors for the successful completion of the release automation project!

Moreover, there is ongoing work on continuous delivery of Jenkins plugins (JEP-229) and on re-designing other Jenkins instances within the project (infra-ci, trusted-ci, and ci.jenkins.io for plugins). In the next few months these stories should provide Jenkins contributors with a modern environment for CI and CD of all Jenkins components.

Terminology updates

Since July, we have officially replaced the old "master" terminology with the "controller" term. It is a follow-up to the "agent" terminology introduced in 2016. We have also deprecated usages of the “blacklist/whitelist” terminology in all components. Currently the community is working on the cleanup of the remaining occurrences in the codebase and documentation, and we invite everyone to contribute.

As a part of the terminology cleanup, last spring we announced the renaming of the official Docker images for Jenkins agents. As a reminder, it does not have any immediate impact on Jenkins users, but they are expected to gradually upgrade their instances.

See more information about terminology updates here and here.

“Jenkins is the Way” program

This year the Advocacy and Outreach SIG started the “Jenkins is the Way” initiative which focuses on promoting user success stories. Over the year, the team published 54 user stories and six case studies on https://jenkinsistheway.io/ as well as a significant amount of community marketing. We also published a number of testimonial videos advertising user stories, including this Introduction to "Jenkins is the Way" video.

See all the stories HERE

Events

Google Summer of Code

In 2020 we had seven students working in the Jenkins mentoring organization. We had 6 projects focused on Jenkins and one project focused on Jenkins X. As usual, in GSoC we focused on problems important to the Jenkins users and community members. The projects delivered highly anticipated new features and key architecture changes needed for the long-term evolution of Jenkins.

This is the first-ever time in Jenkins when all GSoC students have reached the final evaluation and successfully passed it. It was an incredible effort by all the project members and, most importantly, by the students. Thanks a lot to them!

Read more: https://www.jenkins.io/blog/2020/12/22/gsoc-report/

Jenkins in Hacktoberfest 2020

In October we participated in Hacktoberfest. Our featured projects included the Jenkins core, jenkins.io website and plugins.jenkins.io, Helm charts, and multiple plugins. We also encouraged contributors to participate in the Documentation as Code and terminology cleanup across the entire Jenkins ecosystem.

See the details in the Hacktoberfest page.

In total we received 226 pull requests from Hacktoberfest participants. Some stats per Jenkins GitHub organization:

  • 'jenkinsci', PRs: 189, Hacktoberfest contributors: 61

  • 'jenkins-infra', PRs: 100, Hacktoberfest contributors: 40

  • 'jenkins-zh', PRs: 37, Hacktoberfest contributors: 2

Jenkins at DevOps World

The annual DevOps World, formerly known as DevOps World | Jenkins World held on Sept 22-24, with workshops on Sept 25. Just like other events in 2020, DevOps World pivoted to a virtual event but that didn’t mean there was a shortage of sessions or networking opportunities. There were over 50 Jenkins/open-source. And a special congratulations is in order to this year’s Jenkins Contributor Award winners:

  • James Holderness - Jenkins security MVP

  • Marky Jackson - Most valuable Jenkins advocate

  • Tim Jacomb - Most valuable Jenkins contributor

Below are just a few sessions, the full agenda can be found HERE:

Graduation at Continuous Delivery Foundation

Jenkins is the first project to graduate in the CD Foundation. In August the project announced that the Jenkins project has achieved the graduated status in the Continuous Delivery Foundation (CDF). Thanks to all contributors who made our graduation possible! Below you can find a few key changes we have applied during the graduation process:

  • We introduced a new public roadmap for the Jenkins project. This roadmap aggregates key initiatives in all community areas: features, infrastructure, documentation, community, etc. It makes the project more transparent to all Jenkins users and adopters, and at the same time helps potential contributors find the hot areas and opportunities for contribution. The roadmap is driven by the Jenkins community and it has a fully public process documented in JEP-14.

  • A new list of Jenkins adopters was introduced on jenkins.io. This list highlights Jenkins users and references their case studies and success stories, including ones submitted through the Jenkins Is The Way portal. Please do not hesitate to add your company there!

  • We passed the Core Infrastructure Initiative (CII) certification. This certification helps us to verify compliance with open source best practices and to make adjustments in the project (see the bullets below). It also provides Jenkins users and adopters with a public summary about compliance with each best practice. Details are on the Jenkins core page.

  • Jenkins Code of Conduct was updated to the new version of Contributor Covenant. In particular, it sets best practices of behavior in the community, and expands definitions of unacceptable behavior.

More information can be found HERE, and HERE.

Public Roadmap

The Jenkins project now has a public, community-driven project roadmap. Roadmap items are major initiatives and are considered as official plans. The roadmap aggregates key initiatives in all areas of the project.

Many of the 2020 released roadmap items are mentioned elsewhere in this document, including release automation, Core Infrastructure Initiative (CII) certification, user interface improvements, read-only configuration pages, and Google Summer of Code projects like the GitHub Checks API or External Fingerprint Storage.

Other roadmap items include mirror infrastructure improvements, a new Windows installer, and preview releases of pluggable storage for external fingerprints, build logs, and unit test results.

Jenkins 2020 Elections

In October-December the Jenkins community held the regular elections. This year we were electing for 2 governance board members and for all five officer positions, namely: Security, Events, Release, Infrastructure, and Documentation. These roles are an essential part of Jenkins' community governance and well-being. We thank all candidates and voters who participated this year.

Key results:

Full election results: https://www.jenkins.io/blog/2020/12/03/election-results/

And even more

This blog post does not provide a full overview of what changed in the project, it is just a slice of the key highlights mentioned by the contributors. The Jenkins project consists of more than 2000 plugins and components which are developed by thousands of contributors. Thanks to them, a lot of changes happen in the project every day. We are cordially grateful to everybody who participates in the project, regardless of contribution size. Everything matters: new features, bug fixes, documentation, blog posts, well reported issues, Stackoverflow responses, etc. THANKS A LOT TO ALL CONTRIBUTORS!

So, keep updating Jenkins and exploring new features. And stay tuned, there is much more to come next year!

What’s next?

Technical changes. 2021 will be another busy year for the Jenkins community. There are many long-overdue changes in the project, which need to happen if we want Jenkins to succeed. There are many areas on the roadmap: UX revamp, cloud native Jenkins, pluggable storage, etc. There will also be a continued cleanup of old dependencies and technical debt. Several key changes are expected to land in the March LTS baseline: update to Spring Security, XStream unforking, JQuery update, etc.(announcement). In addition to that, we will keep working on expanding platform support in Jenkins, including provisioning support for new Java versions and official images for more architectures like Arm.

Documentation. Documentation efforts will continue in the next year, with a focus on documenting Jenkins usage on modern platforms and and automation use-cases. Wide adoption of documentation-as-code will also continue for plugins By this time almost 600 plugins have been migrated, but there are hundreds more plugins to go.

Security. Another important area is Jenkins security. Automation tools like Jenkins are a key part of the software delivery process in organizations, and their security is essential for the security of products. Misconfigured or outdated systems are a common attack vector, but there are also areas for improvement on the project’s side. Be sure there will be security advisories and vulnerability fixes in 2021. We plan to keep adopting best security development and software delivery practices, and to improve dependency management and developer tools in the project. These areas will be in the spotlight for the project next year.

Events. Next month we will participate in FOSDEM, and there will be a virtual Jenkins stand there. There will also be a CI/CD devroom. If you are interested to meet Jenkins contributors, it is a great opportunity. We also plan to continue all outreach programs and on onboarding more contributors. At the moment we are looking for Google Summer of Code 2020 mentors and project ideas (announcement). We are also ready to consider other non-coding project ideas as a part of CommunityBridge. If you are interested, please contact the Advocacy and Outreach SIG.

Join us in 2021!

We are always looking for more contributors, regardless of the profile and experience. Jenkins is a vast ecosystem which includes many modern technologies.

We invite Jenkins users and contributors to participate in the community and to move these initiatives forward! Join us in the mailing lists and special interest groups,

↧

New eBook: Build, deploy, and automate great stuff with Jenkins

January 26, 2021, 4:43 pm
$
0
0

Jenkins Is The Way

In April of last year, we launched a new Jenkins community website called JenkinsIsTheWay. The Jenkins Is The Way site has collected the experiences of Jenkins users around the world as they develop software and create solutions. They are charting new paths, discovering new opportunities, and overcoming challenges.

That’s what makes Jenkins Is The Way tick. Engineers in the Netherlands might have already met challenges faced by developers in India. Solutions uncovered by DevOps teams in Spain may benefit those just starting in the USA. Interns in Bogota may develop pipeline solutions that can be integrated into workflows used across the globe in Tokyo. Or vice versa!

No matter where you are, regardless of whether your solution is regional or global in nature, Jenkins Is The Way.

Telling your stories in a new eBook

To date, we have nearly 60 user stories, a handful of case studies, and some new user testimonial videos from the Jenkins community. You shared how using Jenkins has helped make your builds faster, your pipelines more secure, and your developers and software engineers happier. In essence, Jenkins Is The Way showcases how Jenkins has made it a whole lot easier to do the work you do every day.

We’ve now gathered some of those stories in our first Jenkins Is The Way ebook, covering various challenges and solutions from six different industries: Aerospace, Education, Finance, Insurance, Retail, and Travel.

ebook cover

You’ll read about innovations from KP Labs in Poland and Preply in Ukraine. You’ll discover how automation helped Avoris out of Spain & Portugal and Tymit in the United Kingdom. And you’ll be inspired by what China’s JD.com and Topdanmark in Denmark were able to achieve after tapping into the Jenkins community.

This curated collection of stories illustrates how Jenkins community members build next-generation DevOps and CI/CD platforms as the backbone for software innovation across companies of all sizes. They highlight the innovation, ingenuity, and keen ability to adapt Jenkins plugins to handle everyday business issues, everywhere.

The technology landscape is full of solutions perfected to tackle all aspects of software development. But time and time again, developers keep coming back to Jenkins. Your engineering time and resources are too valuable to be spent re-inventing the wheel or handling manual testing and deployment cycles.

Jenkins' vast array of plugins and pre-built solutions will have you automating more than you ever imagined. So, get inspired to build great stuff with Jenkins. And, please share this ebook with your network.

Share your story!

And when you’re ready to tell your story, we’re prepared to help you share it. Fill out the short questionnaire, and we’ll send you our Jenkins Is The Way T-shirt as a thank you!

Special thanks to CloudBees, Inc. for sponsoring this program.

↧

Docker image updates

February 16, 2021, 12:08 pm
$
0
0

Beginning with Jenkins 2.279 and Jenkins 2.263.4, the Jenkins project is upgrading the base operating system and Java version used in the jenkins/jenkins:latest and jenkins/jenkins:lts images. The update replaces OpenJDK 8u242 with AdoptOpenJDK 8u282 and replaces Debian 9 ("Stretch") with Debian 10 ("Buster").

Jenkins and Docker

Why?

We’re changing the base image so that we have a better supported operating system and a more current Java release for Jenkins controllers.

Better supported operating system

The Docker images provided by the Jenkins project rely on the operating system security processes of the operating system provider.

Our Docker images have used Debian 9 ("Stretch") for multiple years. Debian 9 security updates have been discontinued as of July 6, 2020. Debian 9 Long Term Support security updates will be discontinued at the end of June 2022. The upgrade to Debian 10 keeps us on an operating system maintained by the operating system security team.

More current Java release

The Debian 9 Docker images were based on the openjdk:8-jdk-stretch Docker image. The last update to that image was one year ago with the release of JDK 8u242. We need a maintained Docker base image that keeps pace with JDK releases and operating system updates so that the controller is running the most recent Java updates and most recent operating system updates.

Other Jenkins controller images have already switched from using openjdk base images to instead use base images provided by Eclipse Adoptium.Eclipse Adoptium is the Eclipse project formed when AdoptOpenJDK joined the Eclipse Foundation. This change adapts the jenkins/jenkins:latest and jenkins/jenkins:lts images to use the Adoptium JDK images in the same pattern as is already used for the Jenkins JDK 11 Docker images like jenkins/jenkins:lts-jdk11. The Jenkins Platform SIG has enjoyed very good results in our interactions with the Eclipse Adoptium project. We look forward to continuing our collaboration with them.

Thanks a lot to Alex Earl and Jim Crowley for the image build restructuring groundwork that made the image upgrade possible! Also thanks to Oleg Nenashev and other contributors for their reviews and testing.

Packaging changes

The Jenkins Docker image based on Debian 10 ("Buster") includes some different packages than Debian 9 ("Stretch"). Some packages have been removed because they are no longer supported by their communities. Some packages have been removed due to infrequent and decreasing use. Users of the Jenkins Docker images may need to extend their definition of their Docker image to include packages that are no longer included in the base image.

SCM packages removed

The following source control management packages are no longer included in the Jenkins controller images for jenkins/jenkins:latest or for jenkins/jenkins:lts:

  • bzr

  • mercurial

  • subversion

Other packages removed

Additional packages that are no longer included in the Jenkins controller images include:

  • bzip2

  • mime-support

  • python (the Python project stopped supporting Python 2 January 1, 2020)

  • xz-utils

A detailed list of the exact package changes is available in the pull request.

Upgrade and compatibility notes

The Jenkins controller images are designed to be extended to meet user needs. Custom Jenkins controller images can be created from the base images and are designed to allow additional Jenkins plugins and additional operating system packages.

For example, the Installing Docker instructions illustrate a technique to install the Blue Ocean plugins and some operating system packages in a custom Docker image.

Docker image with Subversion

The following Docker image definition installs the most recent Jenkins Long Term Support release with the subversion plugin and the operating system subversion command:

FROM jenkins/jenkins:lts
USER root
RUN apt-get update && \
    apt-get install -y --no-install-recommends subversion
USER jenkins
RUN jenkins-plugin-cli --plugins subversion:2.14.0

Build a new docker image from this Dockerfile and assign the image a meaningful name, e.g. "myjenkins-subversion:1.1":

docker build -t myjenkins-subversion:1.1 .

Docker image with Mercurial

The following Docker image definition installs the most recent Jenkins Weekly release with the mercurial plugin and the operating system hg command:

FROM jenkins/jenkins:latest
USER root
RUN apt-get update && \
    apt-get install -y --no-install-recommends mercurial
USER jenkins
RUN jenkins-plugin-cli --plugins mercurial:2.12

Build a new docker image from this Dockerfile and assign the image a meaningful name, e.g. "myjenkins-mercurial:1.1":

docker build -t myjenkins-mercurial:1.1 .

What’s next?

We will continue Docker image updates as new Java versions are released.

If you are interested in new features in Jenkins Docker packaging, stay tuned for future announcements! There are multiple ongoing initiatives which you can find on the public Jenkins roadmap. Some stories:

  • Switching to AdoptOpenJDK.

  • General availability of Windows images.

  • Support for more platforms (AArch64, IBM s390x, PowerPC).

  • Introducing multi-platform Docker images.

If you are interested in any of these projects and would like to contribute, please reach out to the Platform Special Interest Group which coordinates initiatives related to Jenkins in Docker.

↧
↧

Jenkins Contributor Summit Online Feb 23-25

February 16, 2021, 12:08 pm
$
0
0

The Jenkins Contributor Summit brings together current and future contributors to the Jenkins project. We’re hosting an online summit this year to encourage contributors from around the world to meet, discuss, and plan for the future.

The Contributor Summit will be Tuesday, February 23rd 2021 through Thursday, February 25, 2021. The summit brings together community members to learn, meet, and help shape the future of Jenkins. In the Jenkins commmunity we value all types and sizes of contributions and love to welcome new participants.

register button

Format

The online format allows greater flexibility for meeting times and topics. Contributors will meet to discuss specific topics in smaller groups at times that are convenient for those in the meeting.

Opening session

The opening session will start Tuesday, February 23, 2021 at 15:00 UTC. After an initial welcome and overview, we’ll hear from leaders in the Jenkins project as they share the results from the previous 12 months and outline ideas for the next 12 months.

2021 02 16 contributor summit

  • Security - Daniel Beck

  • Infrastructure - Olivier Vernin

  • Release - Tim Jacomb

  • User experience - Félix Queiruga

  • Chinese localization - 赵晓杰(Rick)

  • Configuration as code - Tim Jacomb

  • Google Summer of Code - Kara de la Marck

  • Documentation - Mark Waite

  • Events and Advocacy - Alyssa Tong

  • Cloud Native - Kara de la Marck

  • Platforms - Mark Waite

After those presentations, we’ll create "breakout rooms" in the online meeting that will allow those interested in specific tracks to meet, identify preferred times for their tracks, and prepare draft agendas for their tracks.

Tracks

Smaller sessions ("tracks") will be run in the 48 hour period between the opening session and the closing session. These smaller sessions will be focused on specific topics. A track leader will organize the track meeting, facilitate discussions in the track meeting, and present a summary of the track results in the closing session.

Closing session

The opening session will start Thursday, February 25, 2021 at 15:00 UTC. After an initial welcome and overview, we’ll hear from track leaders as they share the results from their track meetings.

We’ll identify items that should be added to the Jenkins roadmap, items that should be further investigated by special interest groups, and items that need further discussion elsewhere.

↧

cdCon 2021 - Call for Jenkins Proposals

February 17, 2021, 12:42 pm
$
0
0

Hear ye! Hear ye! Jenkins Community,

cdCon 2021 (the Continuous Delivery Foundation’s annual flagship event) is happening June 23-24 and its call for papers is open!

This is your chance to share what you’ve been doing with Jenkins. Are you building something cool? Using it to solve real-world problems? Are you making things fast? Secure? Or maybe you’re a contributor and want to share what’s new. In all cases, we want to hear from you!

Submit your talk for cdCon 2021 to be part of the conversation driving the future of software delivery for technology teams, enterprise leadership, and open-source communities.

Submission Deadlines

Early-Bird Deadline

Friday, February 19 by 11:59 PM PST

Final Deadline

Friday, March 5 at 11:59 PM PST

Topics

Here are the suggested tracks:

Continuous Delivery Ecosystem

This track spans the entire Continuous Delivery ecosystem, from workflow orchestration, configuration management, testing, security, release automation, deployment strategies, developer experience, and more.

Advanced Delivery Techniques

For talks on the very cutting edge of continuous delivery and emerging technology, for example, progressive delivery, observability, and MLOps.

GitOps & Cloud-Native CD

Submit to this track for talks related to continuous delivery involving containers, Kubernetes, and cloud*native technologies. This includes GitOps, cloud-native CD pipelines, chatops, best practices, etc.

Continuous Delivery in Action

This track is for showcasing real-world continuous delivery addressing challenges in specific domains e.g. fintech, embedded, healthcare, retail, etc. Talks may cover topics such as governance, compliance, security, etc.

Leadership Track

Talks for leaders and decision-makers on topics such as measuring DevOps, build vs buy, scaling, culture, security, FinOps, and developer productivity.

Community Track

There is more to open source than code contributions. This track covers topics such as growing open source project communities, diversity & inclusion, measuring community health, project roadmaps, and any other topic around sustaining open source and open source communities.

Singular project focus and/or interoperability between:

  • Jenkins

  • Jenkins X

  • Ortelius

  • Spinnaker

  • Screwdriver

  • Tekton

  • Other – e.g. Keptn, Flagger, Argo, Flux

View all tracks and read CFP details.

We look forward to reading your proposal!

Submit it here

cdcon2021jenkinscfp
↧

Update-Center certificate rotation

March 15, 2021, 4:12 pm
$
0
0

Jenkins update center certificate rotation

On the 29th of March 2021, we’ll rotate the Jenkins update center certificate. The existing certificate expires in April 2021. When the new certificate is installed on March 29, 2021, Jenkins versions older than 2.178 (April 2018), won’t be able to communicate with the default and experimental update centers. Instances using alternative update centers (self-hosted or vendor-provided) will not be affected by this change. Regarding plugins update, the update-center usually supports up to one-year-old Jenkins core versions with 2.204 being the oldest version supported.

If you don’t update regularly, please review the Jenkins security advisories and use this change as your motivation to update to a more recent Jenkins version.

Who

Jenkins users running Jenkins versions older than 2.178 will not see any further updates after the update center certificate change March 29, 2021.

Jenkins developers will not see plugin updates when they use mvn hpi:run to test their plugin if the Jenkins version is older than 2.178. Plugin developers can update their minimum Jenkins version to a newer Jenkins version. Refer to the guidelines in "Choosing a Jenkins version" when selecting the new minimum Jenkins version. Plugin developers may also be able to test with a newer Jenkins version using arguments like mvn -Djenkins.version=2.249.1 hpi:run.

Jenkins users running versions 2.178 or newer are not affected by this change.

What

Jenkins uses the update center to identify updates to core and to plugins. The service signs its metadata with a certificate that is cross-signed by a root certificate. Jenkins is bundled with the root certificate so it can confirm the authenticity of update center data. When updates are available, an alert is shown to Jenkins users that reminds them to update.

Why

The root certificate bundled in Jenkins was created in April 2011 and will expire in April 2021. We prepared for this rotation in April 2018 when we bundled the new root certificate with Jenkins core releases. It’s now time to use the new root certificate with a new update center certificate. The new root certificate will expire in April 2028.

You can follow the work for this certificate rotation in this ticket INFRA-2902

So again, keep your instance updated and everything should be fine.

See you in 2028,

Various Links:

↧
Search

Jenkins accepted in Google Summer Of Code 2021!

March 18, 2021, 5:42 am
$
0
0

Jenkins GSoC

On behalf of the Jenkins GSoC org team, I am happy to announce that this year, for the first time, the Jenkins projects will be participating inGoogle Summer of Code 2021 as part of theContinuous Delivery Foundation (CDF) GSoC organization.

We’re very excited to have the Jenkins project participate in GSoC as part the CDF mentoring organisation along with fellow CDF projects such as Ortelius, Screwdriver, Spinnaker, and Tekton. We believe that being part of the CDF GSoC org will create an environment for students with even more mentoring channels, potential cross-fertilization of ideas, and an even greater community of DevOps practitioners to join!

CDF GSoC

What’s next? GSoC is officially announced, and please expect more students to contact projects in ourGitter channels and mailing lists. Many communications will also happen in SIG and sub-project channels. Also, please join the gsoc channel on the CDF Slack. We will be working hard in order to help students to find interesting projects, to explore the area, and to prepare their project proposals before the deadline on April 13th. Then we will process the applications, select projects and assign mentor teams.

All information about the Jenkins GSoC is available on its sub-project page.

I am a student. How do I apply?

See the Information for students page for full application guidelines.

We encourage interested students to reach out to the Jenkins community early and to start exploring project ideas. All project ideas have chats and mailing lists referenced on their pages. We will be also organizing office hours for students, and you can use these meetings to meet org admins and mentors and to ask questions. Also, join our Gitter channel and themailing list to receive information about such incoming events in the project.

The application period starts on March 29th, but you can prepare now! Use the time before the application period to discuss and improve your project proposals. We also recommend that you become familiar with Jenkins and start exploring your proposal areas. Project ideas include quick-start guidelines and reference newbie-friendly issues which may help with initial study. If you do not see anything interesting, you can propose your own project idea or check out ideas proposed by other CDF organizations participating in GSoC.

I want to be a mentor. Is it too late?

It’s not! We are looking for more project ideas and for Jenkins contributors/users who are passionate about Jenkins and want to mentor students. No hardcore experience required, mentors can study the project internals together with students and technical advisors. We are especially interested in ideas beyond the Java stack, and in ideas focusing new technologies and areas (e.g. Kubernetes, IoT, Python, Go, whatever).

You can either propose a new project idea or join an existing one. See the Call for Mentors post and Information for mentors for details. If you want to propose a new project, please do so as soon as possible so that students have time to explore them and to prepare their proposals.

This year mentorship does NOT require strong expertise in Jenkins development. The objective is to guide students and to get involved into the Jenkins community. GSoC org admins will help to find advisers if special expertise is required.

Important dates

  • Apr 13 - deadline for student applications

  • May 17 - accepted projects announced, teams start community bonding and coding

  • Aug 16 - coding period ends

  • Aug 31 - Results announced

See the GSoC Timeline for more info.

↧
↧

Jenkins accepted in SheCodeAfrica Contributhon

March 21, 2021, 1:34 am
$
0
0

SheCodeAfrica logo

SheCodeAfrica is a non-profit organization focused on celebrating and empowering young girls and women in technology across Africa. They provide resources and events that help women grow and achieve their personal and career goals. Their mentoring programs provide help and guidance as participants learn and grow in their careers.

This year, SheCodeAfrica is organizing the SheCodeAfrica Contributhon. Contributhon is a boot camp where African women are paid to work with open source organizations on selected projects with dedicated mentors. This program aims to create a more diverse, inclusive, and innovative culture within the African open source ecosystem by matching African women in technology with sponsor and mentor open source organizations.

Jenkins in SheCodeAfrica Contributhon

Jenkins

The Jenkins project has been accepted as a Contributhon mentoring organization. Our project idea will introduce participants to Jenkins and plugin development as they create Pipeline examples and create Pipeline help for Jenkins Pipeline plugins. Participants will learn more about Jenkins Pipeline and will submit plugin pull requests with examples and online help.

The Jenkins Pipeline Steps Reference and Pipeline online help often receive feedback that more examples are needed, that step return values need to be described, and that arguments need more description of their purpose, allowed values, and expected results. Most plugin maintainers do not provide detailed documentation of the pipeline steps, or the arguments to those pipeline steps. This project will improve the documentation of pipeline steps and their arguments while introducing Jenkins Pipeline, Jenkins plugin development, Jenkins documentation as code, and the concepts of GitHub forks and pull requests.

We’ve identified development tasks that up to three Contributhon participants will complete during April. The tasks will introduce the participants to Jenkins plugin development. They will experiment with plugin changes in Jenkins and submit pull requests to provide Pipeline examples and help. They will meet twice a week with Jenkins mentors, Kristin Whetstone, Mark Waite, and Meg McRoberts to review their progress, provide coaching, and help with issues.

Data driven choices

We’ve been collecting Jenkins documentation feedback since 2017. Now we’re using that feedback to prioritize the plugins to improve as part of this project. The top 10 Pipeline plugins that have received the most feedback are:

What about my plugin?

If you are a plugin maintainer and would like help to add examples and online help for the Pipeline steps in your plugin, send email to the Jenkins Documentation mailing list. We’ll consider including additional plugins as we better understand the development pace for SheCodeAfrica participants.

↧

Welcome Ewelina Wilkosz - new Jenkins Governance Board member

March 22, 2021, 1:12 pm
$
0
0

We would like to announce changes in the Jenkins Governance Board. As it was announced earlier this month, Marky Jackson has decided to step down from his elected roles. On behalf of the Jenkins community, we would like to thank Marky for all contributions and for the continued participation in the Jenkins community. As an active Jenkins contributor and community leader, Marky helped a lot of initiatives to happen: Jenkins and Kubernetes ecosystem, terminology changes, GSoC and GSoD, pipeline authoring SIG and many more activities. Thank you Marky!

The governance board has followed the interim procedure to nominate the new governance board member until the end of the term. The board decided to respect the results of the 2020 elections and to nominate Ewelina Wilkosz who received the most of the votes after the elected candidates. She has accepted the nomination, and on March 10th the Governance Meeting has confirmed it. We are happy to welcome Ewelina Wilkosz as a new Governance Board Member!

Ewelina will join other governance Board Members:Kohsuke Kawaguchi,Ullrich Hafner,Oleg Nenashev, andGavin Mogan. The term will last until December 2022.

About Ewelina Wilkosz

Ewelina has been a Jenkins Contributor since 2017, when she got involved in Jenkins Configuration as Code Plugin development. Voted Most Valuable Jenkins Contributor in 2018. She has 14 years of experience in IT, working as a CI/CD consultant since the beginning of 2017. In that role she’s trying to solve numerous issues Jenkins users are facing daily - as developers, administrators, maintainers.

Ewelina Wilkosz - new governance board member

Here is Ewelina’s statement from the elections:

As a consultant I support my customers with their Jenkins issues since the beginning of 2017. And almost from the start it was some kind of "as code" approach. The experience I gained during that time resulted in getting myself involved in the development of Configuration as Code Plugin for Jenkins. I consider becoming a part of Jenkins Community one of the most valuable experiences in my career so far. I appreciate how much I have learned and how welcoming the community is.

I am not a very active contributor these days, at least when it comes to code, but what I have to offer is rather extensive experience with Jenkins end users - from small, single instance setups to environments with hundreds of controllers run in a different way on different operating systems. Every day I see challenges those users go through, I know what issues they are facing and which features they consider valuable or missing. As a Jenkins Governance Board Member I can represent those users.

Thanks to my involvement in Configuration as Code Plugin development I had a chance to deliver a number of public presentations where I focused on the benefits of the solution and tried to make it easier for newcomers to try it. Here are a few examples of my activities related to Jenkins Configuration as Code:blogpost,cdCON presentation,podcast recording. So my focus is not only on representing users but also on educating them, and educating myself, so I actually know what they need and why.

What’s next for the Jenkins Governance Board?

In February we had the Jenkins Contributor Summit. There we discussed many topics related to the Jenkins evolution and its roadmap. We identified several initiatives we would like to focus on, including but not limited to improving user experience, contributor onboarding, and securing Jenkins delivery pipelines. These initiatives will be a key focus until the next contributor summit we plan to organize in June. The board will also focus on maintaining the Jenkins governance processes (meetings, budget approvals, funding, etc.) and facilitating contributions to the project.

Participating in Jenkins Governance

Jenkins Governance Board has just a representative function in the community. The project and the community have a long history of open and inclusive governance driven by many contributors. We invite all community members to participate in the project by joining thegovernance meeting, participating in the mailing list conversations, and joining special interest groups driving particular topics.

See this page to know more about contributing to Jenkins in general.

↧

Welcome the SheCodeAfrica Contributhon participants!

April 7, 2021, 8:12 pm
$
0
0

The SheCodeAfrica Contributhon started April 1, 2021. The SheCodeAfrica Contributhon is a boot camp where African women are paid to work with open source organizations on selected projects with dedicated mentors. This program aims to create a more diverse, inclusive, and innovative culture within the African open source ecosystem by matching African women in technology with sponsor and mentor open source organizations. The 5 mentees joining the Jenkins project come from Nigeria, Kenya, and Rwanda.

The Jenkins project has been accepted as a Contributhon mentoring organization. Our project idea will introduce mentees to Jenkins and plugin development as they create Pipeline examples and create Pipeline help for Jenkins Pipeline plugins.

Twice weekly mentoring sessions are scheduled with the mentees and are listed in the events calendar. We’ve already received the first pull request to improve the embedded documentation that generates the Steps Reference and provides contextual help in the Snippet Generator. We’re looking forward to more pull requests and more improvements throughout April, 2021.

We’d like to introduce our mentees so that you recognize them and can welcome them during code review and online chat.

Onyinye Ezike

Onyinye Ezike

Onyinye is based in Lagos, Nigeria. She’s a junior fullstack software developer who is very passionate about learning. She uses Angular, React, Nodejs, and Spring Boot. She has spent the last two years building up herself in software development and she’s hoping to become a world-class software developer.

You’ll recognize her contributions as Onyimatics on GitHub and on the Jenkins issue tracker.

Sharon Jebitok

Sharon Jebitok

Sharon is based in Kenya. She’s a software development student at Microverse, a remote school for software developers. She has been an active member of She Code Africa and she’s presently a mentee on She Code Africa’s Contributhon Program with the Jenkins project. She has spent one year in the software industry since she switched her career into tech, learning to write meaningful code and collaborate with other developers/designers and the tech community.

You’ll recognize her contributions as jebitok-dev on GitHub and as jebitok16_ on the Jenkins issue tracker.

Esther Ejidike

Esther Ejidike

Esther is a frontend developer based in Lagos, Nigeria. She is one of the participants in the She Code Africa Contributhon Open Source Program working with Jenkins and will be contributing to the nodes and processes plugin. She loves to convert designs to exact replicas in the form of webpages and she likes to learn new things in her free time.

You’ll recognize her contributions as Esther101 on GitHub and as esther101 on the Jenkins issue tracker.

Cynthia Iradukunda

Cynthia Iradukunda

Cynthia Iradukunda is based in Kigali, Rwanda. She is currently a junior Computing student at the African Leadership University (ALU) in Mauritius. She wants to be a software engineer because it will help her solve real-world problems while also allowing her to use her coding skills. Her coding skills include but not limited to Java, JavaScript, and attention to detail. By contributing to the Git plugin, she hopes to help users have a smooth process using its documentation, get involved with the project, and connect with other community members.

You’ll recognize her contributions as ciradu2204 on GitHub and as ciradu2204 on the Jenkins issue tracker.

Lucy Karimi

Lucy Karimi

Lucy is based in Nairobi, Kenya. She is a software developer with experience in mobile app development. She is very passionate about tech and is currently involved in the SheCodeAfrica Contributhon.

You’ll recognize her contributions as luciahroyalty101 on GitHub and as luciahroyalty101 on the Jenkins issue tracker.

About the Contributhon projects

See the previous blog post for more information about SheCodeAfrica, the Contributhon, and the plans for Jenkins.

Conversations related to the Contributhon are happening in a Continuous Delivery Foundation slack channel.

Mentors

We’re very grateful to the mentors from the Jenkins project that are hosting mentoring sessions, reviewing pull requests, and encouraging the mentees. Thanks to:

We also thank Zainab Abubakar of SheCodeAfrica for her efforts to facilitate the Contributhon and encourage participation.

↧

Machine Learning Plugin project - coding phase 1 blog post

April 13, 2021, 10:13 am
$
0
0
jenkins gsoc logo small

Welcome back !

This blog post is briefing my coding phase 1 in Jenkins Machine Learning Plugin for this GSoC 2020.

After a fresh introduction of community bonding, On June 1st, coding of GSoC had started officially with phase 1. At this point, every GSoC student should be expected to have a rigid plan with their entire project. With the guidance of mentors I was able to complete a design document and timeline which can be slightly adjustable during the coding. The coding phase was more about coding and discussion.

Quick review

Week 1

I have to ensure that I have a solid architecture for implementing the core of this plugin such that perhaps I or future community will be able to develop R and Julia kernels for this plugin. Factory method design patterns are suitable when users need different types of products ( Python, R and Julia) without knowing much about the internal infrastructure ( Manager of these interpreters ).

All the base classes were implemented this week.

  • Design the Kernel connectors

  • Initiate the interpreter

  • Close the connection

  • Add simple tests

  • Update pom.xml

More than these changes, repo was updated with pull request template and licence header. Readme was extended a little at the end of the week.

Issues and Challenges

  • Git rebase and squash

  • Tests invokes ipython client in the server failed during the CI build

Week 2

With the help of a design document, I had a plan to do the configurations globally and using the Abstract Folder property I could save the configuration and retrieve for the job configuartion. I used to reference some other well developed plugin for the structure of code. That helped me a lot while I was coding. Our first official contributor has popped out his pull request.

Form validations and helper html will be a great help in the user point of view as well as developers. A minor bug was fixed with the guidance of mentors by writing tests with ‘Jenkins WebClient`. Until the end of the week, builder class of the plugin has been implemented with lots of research and discussion. Finally, Test connection was added to the global configuration page to start the connection and test it. A single issue that blocked me using py4j authentication about zeppelin-python was reported in Jira.

Server Configuration

global_config

Issues and challenges

  • Backend depends on Apache zeppelin-python API to connect IPython

  • Find relevant extension points to extend the plugin

Week 3

Earlier in this week, we were trying to merge our IPython builder PR without any memory leaks or bugs that will cause the system to be devastating while running this plugin. Later, this whole week I was implementing a file parser that could copy the necessary files and had the ability to accomplish the file conversion.

Supported file types

  • Python (.py)

  • JSON (Zeppelin notebooks format)

IPython builder was able to run Jupyter Notebooks and Zeppelin formatted JSON files at the end of the 3rd week. Minor issues were fixed in the code. We used ANSI color plugin to fix the abnormal view of error messages produced by the ipython kernel.

Copying and converting Jupyter Notebook

file_convert

Issues and Challenges

Python error messages could not be displayed in rich format If a job is running at user level, but if the python code access file/file path which is not authorized to the user, it returns a permission denied message. While running on agent, notebook has to be written/copied to agent workspace Artifacts should be maintained/reachable from controller after build.

Week 4

As all the major tasks has done, the demo preparation and plan for a experimental release was carried during the last week. There were lots of research on how to connect to a existing kernel in remote. Demo and presentation were prepared along the week.

Issues and Challenges

  • Releasing the first version was bit late

Knowledge transfer

How to debug the code through IntelliJ

  • Edit configuration → Add new Configuration → Maven

  • Command line → type hpi:run

  • Click the debug icon on the toolbar or go to Run menu then Debug

How to setup to test the plugin

  • Setup JDK 8 and Maven 3.5.*

  • Create a directory $ mkdir machine-learning-plugin

  • Create a virtual environment $ virtualenv venv

  • Activate your virtual environment $ source venv/bin/activate

  • Run $ which python to ensure your python path

  • $ git clone https://github.com/jenkinsci/machine-learning-plugin.git

  • Run $ mvn clean install from the machine-learning-plugin directory

  • Run $ mvn hpi:run to start Jenkins with the plugin

  • Set up the builder with localhost and other parameters

  • Create a job

  • Write python code like print(“plugin works”)

  • Build the job

Issues and bugs

Pull Requests

21

Jira Issues

11

Major Tasks

3

Completed

3

In progress

0

↧
↧

Windows Service Wrapper : YAML Configuration Support - GSoC Phase - 01 Updates

April 13, 2021, 10:13 am
$
0
0

Hello all, I am Buddhika Chathuranga from Sri Lanka and I am a final year undergraduate at the Faculty of IT, University of Moratuwa. I am participating in GSoC 2020 with Jenkins. I am working on the Windows Service Wrapper Project. So the Coding Phase 01 of GSoC 2020 is now over and this blog post describes what I have done so far.

Windows Service Wrapper is an executable, which we can use to run applications as Windows Services on Windows machines, which has almost one million downloads. In Jenkins, we use Windows service wrapper to run Jenkins server and agents as Windows services to gain more robustness. This feature is bundled into Jenkins’s core. Currently, the Windows Service wrapper is configured by an XML file. However, there is a limited number of configuration checks and there is no XML schema.

XML is not such a human-friendly way to do that. It is quite verbose and not easy to identify the schema without some effort. Usually, users misconfigure the service wrapper. This is a sample XML configuration file that we can use to provide configurations to Windows Service Wrapper.

Sample XML Configuration File

<service><id>jenkins</id><name>Jenkins</name><description>This service runs Jenkins automation server.</description><envname="JENKINS_HOME"value="%LocalAppData%\Jenkins.jenkins"/><executable>C:\Program Files\Java\jdk1.8.0_201\bin\java.exe</executable><arguments>-Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle
    -jar "C:\Program Files\Jenkins\jenkins.war" --httpPort=8081 --webroot="%LocalAppData%\Jenkinswar"</arguments><logmode>rotate</logmode><onfailureaction="restart"/><extensions><extensionenabled="true"className="winsw.Plugins.RunawayProcessKiller.RunawayProcessKillerExtension"id="killOnStartup"><pidfile>%LocalAppData%\Jenkinsjenkins.pid</pidfile><stopTimeout>10000</stopTimeout><stopParentFirst>false</stopParentFirst></extension></extensions></service>

The usage of YAML could simplify configuration management in Jenkins, especially when automated and configuration management tools are used. So what we are doing under GSoC - 2020 is to update the Windows Service Wrapper to support YAML configurations. After finishing this project, users will be able to provide configurations to the Windows Service Wrapper as a YAML file.

This is a sample YAML configuration file for Windows Service Wrapper and you can see it is less verbose than XML or JSON and much more human friendly. Users can read and edit this without a big effort.

Sample YAML Configuration File

id: jenkinsname: Jenkinsdescription: This service runs Jenkins automation server.env:_name: JENKINS_HOME_value: '%LocalAppData%\Jenkins.jenkins'executable: 'C:\Program Files\Java\jdk1.8.0_201\bin\java.exe'arguments: >-
    -Xrs -Xmx256m -Dhudson.lifecycle=hudson.lifecycle.WindowsServiceLifecycle
    -jar "C:\Program Files\Jenkins\jenkins.war" --httpPort=8081 --webroot="%LocalAppData%\Jenkinswar"logmode: rotateonfailure:_action: restartextensions:
    -pidfile: '%LocalAppData%\Jenkinsjenkins.pid'stopTimeout: '10000'stopParentFirst: 'false'_enabled: 'true'_className: winsw.Plugins.RunawayProcessKiller.RunawayProcessKillerExtension_id: killOnStartup

Advantages of YAML as a configuration file

  • It is less verbose and much more human friendly than XML.

  • Since YAML is not using extra delimiters, it is lightweight.

  • Nowadays YAML has become more popular among configuration management tools.

Project Scope

During this project, I will add the following features to Windows Service Wrapper.

  • YAML Configuration support

  • YAML Schema validation

  • New CLI for the Windows Service Wrapper

  • Support for XML Schema validation via XML Schema Definition (XSD)

Phase 01 Updates

In GSoC - 2020 phase 01, I have done the following updates to the Windows Service Wrapper.

You can find Phase 01 Demo slides in this link.

Below you can find more details about the deliverables listed above.

Project Structure overview

The project structure overview document describes how files and directories are organized in the Windows Service Wrapper project. It will help contributors as well as users, to understand the codebase easily. Also, it helps me a lot to understand the codebase. You can find the document from the given link.

YAML configurations support

As I explained before, in this project, configurations will be provided as a YAML file. I used YamlDotNet library which has more than 2.2k stars on GitHub, to deserialize the YAML file into an Object graph. In this YAML file, users can specify configurations in a more structured way than in XML configuration files. As an example, now users can specify all the log related configurations under the log config. Users can specify all service account related configurations under serviceaccount config etc.

At the moment, I am working on a design document for YAML configuration support. I will add it to the GitHub Issue once ready

New CLI

Before moving into Phase 01 updates, it’s better to explain why we needed a new CLI for Windows Service Wrapper. In the early phases of Windows Service Wrapper, we will keep the XML configuration support as well. So we should allow users to specify the configurations file separately. The current approach is, configurations file should be in the same directory, where Windows Service Wrapper executable exists and the file name of the XML file should be the same as the Windows Service Wrapper executable file name. Also, users should be able to redirect logs if they need to and they should be allowed to elevate command prompt using Windows Service Wrapper. Also, we thought that it’s better to allow users to skip schema validation if they needed. So we decided to move into a new CLI.

As I explained, after releasing this, users will have options in addition to commands. It will make the WinSW CLI more flexible so that we can easily extend it later. These are the options users are allowed to use. These options are available with all the commands except help and version

  • --redirect / -r [string]

    • Users can specify the redirect path for the logs if needed

    • Not required | Default value is null

  • --elevated / -e [boolean]

    • Elevate the command prompt before executing the command

    • Not required | Default value is false

  • --configFile / -c [string]

    • Users can specify the configurations file as a path

    • Not Required | Default value is null

  • --skipConfigValidation / -s [boolean]

    • Users can skip schema validation for configurations file if needed

    • Not required | Default value is true

  • --help / -h

    • User can find what options are available with a particular command with this option

This option is available with the install command

  • --profile / -f [boolean]

    • If this option is true, then users can provide a service account for installation explicitly.

    • Not required | Default value is false

We used commandlineparser/commandline library to parse the command line argument which has more than 2k stars in GitHub. At a glance, the library is compatible with .NET Framework 4.0+, Mono 2.1+ Profile, .NET Standard, and .NET Core.

XML Schema validation

As I mentioned before, there was no schema validation for XML in Windows Service Wrapper. Hence, I was working on schema validation for XML. I use XSD to validate XML files. The XSD file will be shipped as an embedded resource with the executable. You can find the XSD file in my pull request.

Future updates

In the next phase, for GSoC 2020 the listed deliverables features will be released and the YAML schema validation feature will be added. Also, we hope to publish a design document for the new features, which will help contributors.

How to contribute

You can find the GitHub repository in this link. Issues and Pull requests are always welcome. Also, you can communicate with us in the WinSW Gitter channel, which is a great way to get in touch and there are project sync up meetings every Tuesday at 13:30 UTC on the Gitter channel.

↧

GitHub Checks API Plugin Project - Coding Phase 1

April 13, 2021, 10:13 am
$
0
0

This blog post is about our coding phase 1 progress on GSoC project: GitHub Checks API Plugin.

The GitHub Checks API is a highly customized way to integrate CI tools to make reports for pull-requests (PRs). It allows users to see CI reports on GitHub pages directly.

github check run
Figure 1. GitHub Check Run Screenshot from GitHub Docs

What’s more exciting is that it can leave annotations on specific lines of code, just as the comments people left while reviewing.

github check annotations
Figure 2. Check Run Annotation Screenshot from GitHub Docs

While on Jenkins' side, the source code view provided by Warnings Next Generation Plugin does pretty much the same thing.

source view
Figure 3. Source Code View from Warnings Next Generation Plugin

Utilizing such features through GitHub Checks API, it would make Jenkins more convenient to GitHub users.

Features from Coding Phase 1

In the past month, our team was mostly working on the general checks API and an implementation for GitHub checks API.

GitHub Checks API Plugin Demo [starts from 50:15]

General Checks API

Although the general checks API is developed based on the semantic meaning of GitHub Checks API, we still want to prepare it for similar concepts on other platforms like Commit Status API from GitLab. Contributions for implementations on these platforms will be welcomed in the future.

GitHub Checks API Implementation

Our work on supporting GitHub Checks API is mostly done by now. Besides, we implemented a consumer to automatically create a check run that simply indicates the current stage of a Jenkins build. After the release, Jenkins developers (especially publisher plugin ones) can create their own GitHub checks for a GitHub branch source project by consuming our API.

Example: To create a check run like:

Created Check Run

Consumers need to use our API in this way:

ChecksDetails details = new ChecksDetailsBuilder()
        .withName("Jenkins")
        .withStatus(ChecksStatus.COMPLETED)
        .withDetailsURL("https://ci.jenkins.io")
        .withStartedAt(LocalDateTime.now(ZoneOffset.UTC))
        .withCompletedAt(LocalDateTime.now(ZoneOffset.UTC))
        .withConclusion(ChecksConclusion.SUCCESS)
        .withOutput(new ChecksOutputBuilder()
                .withTitle("Jenkins Check")
                .withSummary("# A Successful Build")
                .withText("## 0 Failures")
                .withAnnotations(Arrays.asList(new ChecksAnnotationBuilder()
                                .withPath("Jenkinsfile")
                                .withLine(1)
                                .withAnnotationLevel(ChecksAnnotationLevel.NOTICE)
                                .withMessage("say hello to Jenkins")
                                .withStartColumn(0)
                                .withEndColumn(20)
                                .withTitle("Hello Jenkins")
                                .withRawDetails("a simple echo command")
                                .build(),new ChecksAnnotationBuilder()
                                .withPath("Jenkinsfile")
                                .withLine(2)
                                .withAnnotationLevel(ChecksAnnotationLevel.WARNING)
                                .withMessage("say hello to GitHub Checks API")
                                .withStartColumn(0)
                                .withEndColumn(30)
                                .withTitle("Hello GitHub Checks API")
                                .withRawDetails("a simple echo command")
                                .build()))
                .build())
        .withActions(Collections.singletonList(new ChecksAction("formatting", "format code", "#0")))
        .build();

ChecksPublisher publisher = ChecksPublisherFactory.fromRun(run);
publisher.publish(details);

Future Works

The next step is integrating our API into Warnings Next Generation Plugin and Code Coverage API Plugin consume our API. After that, pipeline support will be added: users can publish checks directly in a pipeline script without requiring a consumer plugin that support the checks.

Resources

References

↧
Search

Git Plugin Performance Improvement: Phase-1

April 13, 2021, 10:13 am
$
0
0

Git Plugin Performance Improvement is a Google Summer of Code 2020 project. It aims to improve the performance of the git plugin, which provides fundamental git functionalities.

Internally, the plugin provides these functionalities using two implementations: command line git and JGit (pure java implementation).

git-intro

CLI git is the default implementation for the plugin, a user can switch to JGit if needed

The project is divided into two parallel stages:

  • Stage 1: Create benchmarks which evaluate the execution time of a git operation provided by CLI git and JGit using JMH, a micro benchmarking test harness.

  • Stage 2: Implement the insights gained from the analysis into the plugin to improve the overall performance of the plugin.

The project also aims to fix any existing performance bottlenecks within the plugin as well.

Benchmarks

The benchmarks are written using JMH. It was introduced in a GSoC 2019 project to Jenkins.

  • JMH is provided within the plugin through the Jenkins Unit Test Harness POM dependency.

  • The JMH benchmarks are created and run within the git client plugin

  • During phase-1, we have created benchmarks for two operations: "git fetch" and "git ls-remote"

Results and Analysis

The benchmark analysis for git fetch:

Git fetch results

git-fetch-results

  • The performance of git fetch (average execution time/op) is strongly correlated to the size of a repository

  • There exists an inflection point on the scale of repository size after which the nature of JGit performance changes (it starts to degrade)

  • After running multiple benchmarks, it is safe to say that for a large sized repository CLI-git would be a better choice of implementation.

  • We can use this insight to implement a feature which avoids JGit when it comes to large repositories.

Please refer to PR-521 for an elaborate explanation on these results

Note: Repository size means du -h .git

Fixing redundant fetch issue

The git plugin performs two fetch operations instead of one while performing a fresh checkout of a remote git repository.

To fix this issue, we had to safely remove the second fetch keeping multiple use-cases in mind. The fix itself was not difficult to code, but to do that safely without breaking any existing use-case was a challenging task.

Further Plan

After consolidating a benchmarking strategy during Phase 1, the next steps will be:

  • Provide functionality to the git plugin, which enables it to estimate the size of the repository without cloning it.

  • Broaden the scope of benchmarking strategy

    • Consider parameters like number of branches, references and commit history to find a relation with the performance of a git operation

    • The git plugin depends on other plugins like Credentials which might require benchmarking the plugin itself and the effects of these external dependencies on the plugin’s performance

  • Focus on other use-cases of the plugin

    • For phase-1, I focused on the checkout step and the operations involved with it

    • For the next phase, the focus will shift to other areas like Multibranch pipelines or Organisation Folders

How can you help?

If you have reached this far of the blog, you might be interested in the project.

To help, you can

Come visit our Gitter channel: https://gitter.im/jenkinsci/git-plugin

↧

Severity of cross-site scripting vulnerabilities

April 13, 2021, 10:13 am
$
0
0

Eagle-eyed readers of today’s security advisory may already have noticed that we consider the cross-site scripting (XSS) vulnerabilities to be 'High' severity. This is a change from previous security advisories, in which similar vulnerabilities got a 'Medium' score.

We follow the guidelines of CVSS version 3.0 for the severity we assign to these issues. Their examples for XSS vulnerabilities, as well as XSS vulnerabilities in other software, consider the most severe, immediate impact to be a modification of the HTML output, possibly also the extraction of the session cookie (something Jenkins prevents by declaring it to be HttpOnly).

Unfortunately, this does not adequately model the impact that a successful XSS exploitation in Jenkins can have: Jenkins administrators can perform far more sensitive actions than e.g. the admins of most content management systems could, as it is designed to allow users to execute code to build, test, and deploy their software. So this kind of vulnerability, that allows attackers to do anything their victims have permission to do, in Jenkins can mean execution of arbitrary code, perhaps via the script console, if the victim has the Overall/Administer permission. None of this requires chaining different actions in an attack, a well-chosen XSS payload will accomplish this.

Therefore, starting today, we score XSS vulnerabilities by the highest immediate impact a successful attack can have, which is a complete system compromise if admins can be attacked. For stored XSS requiring some permissions, like the ability to configure jobs, a typical score would be 8.0. Reflected XSS, which don’t require any permissions to exploit, will usually score 8.8.

↧
↧

Jenkins 2.248: Windows Support Updates

April 13, 2021, 10:13 am
$
0
0

In this article, I would like to announce the new Windows support policy which was introduced in the Jenkins project in June 2020. This policy sets an expectation about how we handle issues and patches related to Windows support for the Jenkins server and agents, and how we organize testing of Windows support in the project. We will also talk about .NET Framework 2.0 support removal in Jenkins 2.248, and about new Windows service management features and fixes Jenkins users get with this release.

header image
Figure 1. Jenkins on Windows

Why?

In theory, Jenkins can run everywhere where you can run Java 8 or Java 11, but, in practice, there are some limitations. The Jenkins core and some plugins contain native code, and hence they rely on operating systems and platforms. We use Java Native Access and Java Native Runtime libraries which provide wide platform support for low-level operations, but there are platform-specific cases not covered by such generic libraries. In the case of Windows platforms we use Windows Service Wrapper (WinSW) andWindows Process Management Library (WinP). These libraries depend on particular Windows API versions and, in the case of Windows services, on .NET Framework.

Historically Jenkins had no documented support policy for Windows, and we were accepting patches for all versions which existed since the Hudson inception in 2004. It became a serious obstacle for Windows component maintainers who had to be very conservative about incoming patches so that we could avoid breaking instances running on old platforms. Lack of testing for older platforms did not help either. And it is not just about maintenance overhead. Users were impacted as well, because it blocked us from adopting some new Windows features and making Jenkins more stable/maintainable on modern platforms.

New policy

To set proper expectations about Windows support, in the new policy we defined four support levels. See the Windows support policy page for the actual information about the support levels and the supported platforms. This blogpost captures the support state as of Jul 23, 2020:

Level 1 - Full Support

We run automated testing for these platforms, and we intend to timely fix the reported issues. This support level includes 64-bit (amd-64) Windows Server versions with the latest GA update pack, and versions used in the official Jenkins server and agent Docker images.

Level 2 - Supported

We do not actively test these platforms, but we intend to keep compatibility. We are happy to accept patches. This support level includes 64-bit (amd64) Windows Server and Windows 10 versions generally supported by Microsoft.

Level 3 - Patches considered

The platforms are generally expected to work, but they may have limitations and extra requirements. We do not test compatibility, and we may drop support if needed. We will consider patches if they do not put Level 1/2 platforms at risk and if they do not create maintenance overhead. This support level includes non-amd64 platforms like x86 (32-bit) and AArch64 (Arm). It also applies to non-mainstream release lines like Windows Embedded, preview releases, and versions no longer supported by Microsoft.

Level 4 - Unsupported

These versions are known to be incompatible or to have severe limitations. We do not support the listed platforms, and we will not accept patches. At the moment this level applies to platforms released before 2008.

When the policy was introduced, there were questions raised about platforms listed in the Level 3 support category. First of all, these platforms are still supported. Users are welcome to run Jenkins on these platforms. We recognize the importance of the platforms listed there, and we intend to keep compatibility with them. At the same time, particular functionality may break there due to the lack of testing when we update Jenkins or upstream dependencies. It may take a while until a fix is submitted by a user or contributor, because we do not maintain development environments for these platforms. By setting a Level 3 support level, we want to set an explicit expectation about those limitations.

If you are interested in expanding the official Windows support policy and adding more platforms there, we invite you to participate in quality assurance of Jenkins. You may contribute by expanding test automation for Jenkins, contributing test environments for your platforms, or participating in the LTS release candidate testing and reporting results. Please contact us via Platform SIG channels if you are interested.

Windows Service Management changes in Jenkins 2.248

winsw logo
Figure 2. WinSW Logo

Although the policy was introduced more than 1 month ago,Jenkins 2.248 is the first release where the new policy is applied. Starting from this release, we won’t support .NET Framework 2.0 for launching the Jenkins server or agents as Windows services. .NET Framework 4.0 or above is now required for using the default service management features.

This release also upgrades Windows Service Wrapper (WinSW) from 2.3.0 to 2.9.0 and replaces the bundled binary from .NET Framework 2.0 to 4.0. There are many improvements and fixes in these versions, big thanks to NextTurn and all other contributors. You can find the full WinSW changelog here, just a few highlights important to Jenkins users:

  • Prompt for permission elevation when administrative access is required. Now Jenkins users do not need to run the agent process as Administrator to install the agent as a service from GUI.

  • Enable TLS 1.1/1.2 in .NET Framework 4.0 packages on Windows 7 and Windows Server 2008 R2.

  • Enable strong cryptography when running .NET Framework 4.0 binaries on .NET 4.6.

  • Support security descriptor string in the Windows service definition.

  • Support 'If-Modified-Since' and proxy settings for automatic downloads.

  • Fix Runaway Process Killer extension so that it does not kill wrong processes with the same PID on startup.

  • Fix the default domain name in the serviceaccount parameter (JENKINS-12660)

  • Fix archiving of old logs in the roll-by-size-time mode.

As you may see, there are many improvements available with this version, and we hope that it will make Windows service installation even more reliable. Some of the changes in WinSW also replaced old workarounds in the Jenkins core, making the code more maintainable.

Use-cases affected by .NET Framework 2.0 support removal

If you use .NET Framework 2.0 to run the Jenkins Windows services, the following use-cases are likely to be affected:

  • Installing the Jenkins server as a Windows service from Web UI. The official MSI Installer supports .NET Framework 2.0 for the moment, but it will be changed in future versions.

  • Installing agents as Windows services from GUI. This feature is provided by in Windows Agent Installer Module from the Jenkins core.

  • Installing agents over Windows Management Instrumentation (WMI) via the WMI Windows Agents plugin

  • Auto-updating of Windows service wrappers on agents installed from GUI.

Upgrade guidelines

If all of your Jenkins server and agent instances already use .NET Framework 4.0 or above, there are no special upgrade steps required. Please enjoy the new features!

If you run the Jenkins server as a Windows Service with .NET Framework 2.0, this instance will require an upgrade of .NET Framework to version 4.0 or above. We recommend running with .NET Framework 4.6.1 or above, because this .NET version provides many platform features by default (e.g. TLS 1.2 encryption and strong cryptography), and Windows Service Wrapper does not have to apply custom workarounds.

If you want to continue running some of your agents with .NET Framework 2.0, the following extra upgrade steps are required:

  1. Disable auto-upgrade of Windows Service Wrapper on agents by setting the-Dorg.jenkinsci.modules.windows_slave_installer.disableAutoUpdate=true flag on the Jenkins server side.

  2. Upgrade agents with .NET Framework 4.0+ by downloading the recent Windows Service Wrapper 2.x version from WinSW GitHub Releases and manually replacing the wrapper ".exe" files in the agent workspaces.

What’s next?

We plan to continue expanding the Windows support in Jenkins, including providing official Docker images for newer Windows versions. For example, there is already a pull request which will introduce official agent images for Windows Server Core LTSC 2019 and for Windows Server Core and Nano Server 1909. We are also interested to keep expanding test coverage for Windows platforms. Any contributions and feedback will be appreciated!

We also keep working on improving Windows Services.Buddhika Chathuranga, a Google Summer of Code 2020 student, is working on support for YAML Configurations in Windows Service Wrapper, and on better verification of XML and YAML Configurations. See the details on the project page and in theCoding Phase 1 Report. In addition to that, there is ongoing work on a new Windows Service Wrapper 3.0 release which will redesign CLI and introduce a lot more improvements. If you are interested in contributing to Windows Service Wrapper, see the guidelines here. We will also appreciate your feedback on the WinSW Gitter channel.

↧

External Fingerprint Storage Phase-2 Updates

April 13, 2021, 10:13 am
$
0
0

As another great phase for theExternal Fingerprint Storage Project comes to an end, we summarise the work done during this phase in this blog post. It was an exciting and fruitful journey, just like the previous phase, and offered some great learning experience.

To understand what the project is about and the past progress, please refer to thephase 1 blog post.

New Stories Completed

We targeted four stories in this phase, namely fingerprint cleanup, fingerprint migration, refactoring the current implementation to use descriptors, and improved testing of the Redis Fingerprint Storage Plugin. We explain these stories in detail below.

Fingerprint Cleanup

This story involved extending the FingerprintStorage API to allow external storage plugins to perform and configure their own fingerprint cleanup strategies. We added the following functionalities to Jenkins core API:

  • FingerprintStorage#iterateAndCleanupFingerprints(TaskListener taskListener)

    • This allows external fingerprint storage implementations to implement their own custom fingerprint cleanup. The method is called periodically by Jenkins core.

  • FingerprintStorage#cleanFingerprint(Fingerprint fingerprint, TaskListener taskListener)

    • This is a reference implementation which can be called by external storage plugins to clean up a fingerprint. It is upto the plugin implementation to decide whether to use this method. They may choose to write a custom implementation.

We consume these new API functionalities in theRedis Fingerprint Storage plugin. The plugin uses cursors to traverse the fingerprints, updating the build information, and deleting the build-less fingerprints.

Earlier, fingerprint cleanup was always run periodically and there was no way to turn it off. We also added an option to allow the user to turn off fingerprint cleanup.

Fingerprint cleanup disable

This was done because it may be the case that keeping redundant fingerprints in memory might be cheaper than the cleanup operation (especially in the case of external storages, which are cheaper these days).

Fingerprint Migration

Earlier, there was no support for fingerprints stored in the local storage. In this phase, we introduce migration support for users. The old fingerprints are now migrated to the new configured external storage whenever they are used (lazy migration). This allows gradual migration of old fingerprints from local disk storage to the new external storage.

Refactor FingerprintStorage to use descriptors

Earlier, whenever an external fingerprint storage plugin was installed, it was enabled by default. We refactored the implementation to make use of Descriptor pattern so the fingerprint engine can now be selected as a dropdown from the Jenkins configuration page. The dropdown is shown only when multiple fingerprint storage engines are configured on the system.Redis Fingerprint Storage Plugin was refactored to use this new implementation.

Fingerprint Storage Engine Dropdown

Strengthened testing for the Redis Fingerprint Storage Plugin

We introduced new connection tests in theRedis Fingerprint Storage Plugin. These tests allow testing of cases like slow connection, breakage of connection to Redis, etc. These were implemented using the Toxiproxy module inside Testcontainers.

We introduced test for Configuration-as-code (JCasC) compatibility with the plugin. The documentation for configuring the plugin using JCasC was also added.

We introduced a suite of authentication tests, to verify the proper working of the Redis authentication system. Authentication uses the credentials plugin.

We strengthened our web UI testing to ensure that the configuration page for the plugin works properly as planned.

Other miscellaneous tasks

Please refer to the Jira Epic for this phase.

Releases 🚀

Changes in the Jenkins core (except migration) were released in Jenkins 2.248.

We drafted 1.0-rc-1 release for the Redis Fingerprint Storage Plugin to deliver the changes. This was an increment from the alpha release we had drafted at the end of the previous phase. The plugin is now available at https://plugins.jenkins.io/redis-fingerprint-storage/!

Trying out the new features!

The latest release for the plugin can be downloaded from the update center, instructions for which can be found in the README of the plugin. We appreciate you trying out the plugin, and welcome any suggestions, feature requests, bug reports, etc.

Acknowledgements

The Redis Fingerprint Storage plugin is built and maintained by the Google Summer of Code (GSoC) Team forExternal Fingerprint Storage for Jenkins. Special thanks to Oleg Nenashev,Andrey Falko, Mike Cirioli,Tim Jacomb, and the entire Jenkins community for all the contribution to this project.

Future Work

Some of the topics we aim to tackle in the next phase include a new reference implementation (possibly backed by PostgreSQL), tracing, etc.

Reaching Out

Feel free to reach out to us for any questions, feedback, etc. on the project’s Gitter Channel or the Jenkins Developer Mailing list. We use Jenkins Jira to track issues. Feel free to file issues under redis-fingerprint-storage-plugin component.

↧

Machine Learning Plugin project - Coding Phase 2 blog post

April 13, 2021, 10:13 am
$
0
0
jenkins gsoc logo small

Welcome back folks!

This blog post is about my coding phase 2 in Jenkins Machine Learning Plugin for this GSoC 2020. After successfully passing the evaluation and demo in the phase 1, our team went ahead for facing the challenges in phase 2.

Summary

This phase of coding was well spent by documentation and by fixing many bugs. As the main feature of connecting to an IPython Kernel is done in phase 1, we were able to focus on fixing minor/major bugs and documenting for the users. According to the JENKINS-62927 issue, a Docker agent was built to facilitate users without concerning plugin dependencies in python. In the act of deprecation of Python 2, we ported our plugin to support Python 3. We have tested our plugin in Conda, venv and Windows environments. Machine learning plugin has successfully passed the end to end test. A feature for a code editor is needed for further discussion/analysis as we have done a simple editor that may be useful in other ways in the future. PR#35

Main features of Machine Learning plugin

  • Run Jupyter notebook, (Zeppelin) JSON and Python files

  • Run Python code directly

  • Convert Jupyter Notebooks to Python and JSON

  • Configure IPython kernel properties

  • Support to execute Notebooks/Python on Agent

  • Support for Windows and Linux

Upcoming features

  • Extract graph/map/images from the code

  • Save artifacts according to the step name

  • Generate reports for corresponding build

Future improvements

  • Usage of JupyterRestClient

  • Support for multiple language kernels

    • Note : There is no commitment on future improvements during GSoC period

Docker agent

The following Dockerfile can be used to build the Docker container as an agent for the Machine Learning plugin. This docker agent can be used to run notebooks or python scripts.

Dockerfile
FROM jenkins/agent:latest

MAINTAINER Loghi <loghijiaha@gmail.com>

USER root

RUN apt update && apt install --no-install-recommends python3 -y \
    python3-pip \&& rm -rf /var/lib/apt/lists/*

COPY requirements.txt /requirements.txt

RUN pip3 install --upgrade pip setuptools && \
    pip3 install --no-cache-dir -r /requirements.txt && \
    ln -sf /usr/bin/python3 /usr/bin/python && \
    ln -sf /usr/bin/pip3 /usr/bin/pip

USER jenkins

Ported to Python 3

As discussed in the previous meeting, we concluded that the plugin should support Python 3 as Python 2.7+ has been deprecated since the beginning of 2020. Pull request for docker agent should be also ported to Python 3 support.

Jupyter Rest Client API

The Jupyter Notebook server API seemed to be promising that it can be also used to run notebooks and codes. There were 3 api implementations that were merged in the master. But we had to focus on what was proposed in the design document and had to finish all must-have issues/works. Jupyter REST client was left for future implementation. It is also a good start to contribute to the plugin from the community.

Fixed bugs for running in agent

There were a few bugs related to the file path of notebooks while building a job. The major problem was caused by the python dependencies needed to connect to a IPython kernel. All issues/bugs were fixed before the timeline given.

R support as a future improvement

This is what we tried to give a glimpse of knowledge that this plugin can be extended for multi language support in the future. There was a conclusion that the kernel should be selected dynamically using extension of the script file(like eval_model.rb or train_model.r), instead of scripting the same code for each kernel.

Documentation and End to End testing

A well explained documentation was published in the repository. A guided tutorial to run a notebook checked out from a git repo in an agent was included in the docs page. Mentors helped to test our plugin in both Linux and Windows.

Code editor with rebuild feature

Code editor was filtered as a nice to have feature in the design document. After grabbing the idea of Jenkinsfile replay editor, I could do the same for the code. At the same time, when we are getting the source code from git, it is not an elegant way of editing code in the original code. After the discussion, we had to leave the PR open that may have use cases in the future if needed.

Jenkins LTS update

The plugin has been updated to support Jenkins LTS 2.204.1 as 2.164.3 had some problems with installing pipeline supported API/plugin

Installation for experimental version

  1. Enable the experimental update center

  2. Search for Machine Learning Plugin and check the box along it.

  3. Click on Install without restart

The plugin should now be installed on your system.

Resources

↧
Viewing all 1088 articles
Browse latest View live
Search