Quantcast
Channel: Jenkins Blog
Viewing all 1088 articles
Browse latest View live

Third Party Repository Detection Probe

$
0
0

Introduction

A third-party repository is a repository that is not hosted at https://repo.jenkins-ci.org/.

The Jenkins Infra team was concerned about the dependencies used by third-party repositories. These repositories are a concern, not just for security reasons, but also for reliability.

The third-party repository detection probe ensures that plugins are built only from trusted and reliable Maven repositories.

Importance of the probe

This probe was requested by the community. In the ticket description, one of the plugin developers asked the Jenkins team to include a third-party repository in the plugin build. With the probe in place, the Jenkins team can now identify the plugins they should add to the build process.

Challenges

In a Maven project, all the dependencies are listed in the pom.xml file. A plugin can have multiple pom files, and each module may have its own pom. This means that there can be multiple child poms in the same repository.

A major challenge in this probe was considering the parent and child pom relationship from all angles.

Outcome

During code reviews and test cases, we noticed that there were assumptions made regarding Maven’s project hierarchies.

Furthermore, some edge-case scenarios were not considered while specifying the probe. Due to the number of open questions, this probe was put on hold until further research was completed. The goal of doing this was to adapt the project architecture to tackle the newly identified cases.

Learning

I implemented parameterized test cases to test the probe. Additionally, I extensively read the Maven documentation to understand how to test POM structure, inheritance, and Maven API.

Conclusion

This probe is currently partially completed, with approximately 60% of the code and test cases already implemented.

The objective is to create an efficient pom probe capable of aggregating all the parent-child pom relationships into a single file. By doing so, we can easily list the third-party repositories used in every pom file within the repository.

For more information or to find answers to any questions you might have, please visit the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.


Security Scan Probe

$
0
0

Introduction

The Jenkins Security team offers a GitHub action that automatically scans the plugin source code during Continuous Integration and reports security issues.

The Jenkins community considers using such tools to ensure a high-security level for plugins as a good practice.

The Security Scan probe identifies whether the plugin is configured to execute the security action.

Importance of the Probe

This probe is essential from a security perspective, as it checks for known security breaches and vulnerabilities in a plugin.

Its primary purpose is to verify that the security scan is properly configured in the plugin’s GitHub Action.

Challenges

This probe was originally started by another contributor. However, they were unable to continue, so I took over their work.

Additionally, this probe required refactoring existing classes to eliminate code duplication, as the JEP-229 (Continuous Delivery) probe also looks for CI configurations in GitHub Actions.

Outcome

This probe will be beneficial to the security team, as it helps identify gaps and improve the security of the Jenkins plugin ecosystem.

Learning

During this project, I accomplished the following:

  • Successfully rebased an existing Git branch.

  • Developed and wrote unit test cases.

  • Gained knowledge on leveraging the class hierarchy to build an abstract probe that can be implemented by any probes requiring GitHub Action configuration checks.

For more information or if you have any questions, please visit the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.

Unreleased Production Changes Probe

$
0
0

Introduction

A bug fix, security fix, or new features are of no use unless they are released.

A production release not only provides users with an updated plugin, but also gives the developers a chance to get feedback on their work.

The "unreleased production changes" probe looks for unreleased commits in the production files.

If an unreleased file is found in the production path, the probe returns a failure message. If no unreleased file is found, it returns a success message.

For now, we are focusing on Maven production files (pom.xml and src/main folder).

This is my first probe that went live as a GSoC contributor.

Importance of the probe

The probe aims to identify unreleased production commits in Maven production files.

Finding an unreleased production file can mean one or more of the following:

  • There are unreleased security fixes that should be released as soon as possible.

  • There are unreleased features requested that would improve the usefulness of the plugin.

  • The plugin maintainer may be inactive or too busy to further develop the plugin. This could indicate the plugin needs a new maintainer and should be up for adoption.

Challenges

Learning to use JGit, a Java library that implements a Git version control system, was one of the biggest challenges I faced when working on this probe.

Since I was not familiar with Git hierarchy, I struggled to find the parent of a child commit using JGit.

In the end, my mentors suggested a pair programming session to review the thinking process involved in implementing such a probe.

I am thankful for their kindness and patience.

Outcome

When unreleased production files are found, the probe returns a failure status and the list of filenames.

When no unreleased production file is found, the probe returns a success message.

Learning

I hope there comes a day when test cases become my friends, but I don’t see this happening anytime soon.

Mentors suggested I should start with writing the test cases to reproduce the probe requirement(s), and later add code to pass each of the test cases.

This gave me an understanding of what the probe should actually do, and I learned to identify the test case scenarios.

Reading commits wasn’t possible without JGit Java library.

To implement JGit library and understand the features it provides, you should have strong understanding of Git concepts. I lacked depth in Git concepts that made me struggle with JGit.

This led me to strengthen the Git concepts along the way.

Conclusion

I once again realised the open-source world values quality.

The community is fine with fewer "quality" probes, as opposed to many "not-so-useful" probes.

Starting from ground zero is tough, whether implementing a new Java library or starting new work in another field.

However, with correct mentorship and guidance, you can rise up faster.

For more information or to find answers to any questions you might have, head to the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.

Jenkins in Google Summer of Code Midterm Recap

$
0
0

The Jenkins in GSoC program recently reached its midterm milestone for GSoC 2023. This milestone served to assess performance, and precipitate pass/fail decisions. Taking time to evaluate the progress and workflow of the project provides an opportunity to correct course and address underlying issues.

Earlier this month, on the Jenkins Online Meetup, GSoC contributors presented midterm status and progress for their projects, below is a recap. Thank you Jagruti, Harsh, Vandit, and Ashutosh for your contributions to Jenkins!

Add Probes to "Plugin Health Score"

A probe collects data about each plugin. It is an automated system that ensures the Jenkins Community-defined rules are followed. The probe data is used by health scoring to compute a score, and a higher-quality probe receives a good score. Conversely, an obsolete or outdated plugin is given a lower score.

Sharing details about the probes that I worked on during the first phase of GSoC.

Unreleased Production Changes Probe

A bug fix, security improvement, or new feature can only be beneficial when it’s actually released. The "Unreleased Production Changes" probe is designed to detect unreleased commits in the production files.

This probe’s primary goal is to identify unreleased production commits specifically in the Maven production files.

Following my mentors' advice, I began by writing test cases to accurately reproduce the probe’s requirements. Subsequently, I added the necessary code to pass each of these test cases.

During the implementation, I utilized the JGit library and familiarized myself with its features. However, I encountered some challenges due to a lack of depth in Git concepts, which required me to delve deeper and strengthen my understanding of Git concepts along the way.

Read the unreleased production changes blog post for more details on the probe.

The Third-Party Repository Detection Probe

By definition, a third-party repository is a repository that is not hosted at https://repo.jenkins-ci.org.

The third-party repository detection probe ensures that plugins are only built from trusted and reliable Maven repositories.

This probe was requested by the Jenkins community.

One of the major challenges in implementing this probe was considering the parent and child pom relationships from all angles.

Certain edge-case scenarios were not initially taken into account while specifying the probe, and due to the number of open questions, the probe was temporarily put on hold until further research was conducted.

Throughout the process, I extensively read the Maven documentation to gain a better understanding of how to test POM structure, inheritance, and Maven API.

Read the third party repository detection probe blog post for more details on the probe.

GitHub Security Scan Workflow Probe

The Security Scan probe identifies whether the plugin is configured to execute the security action.

From a security perspective, this probe is crucial as it checks for known security breaches and vulnerabilities in a plugin.

Its primary purpose is to ensure that the security scan is correctly configured in the plugin’s GitHub Action.

Additionally, this probe involves refactoring existing classes to avoid code duplication, and it shares similarities with the JEP-229 (Continuous Delivery) probe, which also looks for CI configurations in GitHub Actions.

Throughout this process, I have gained knowledge in leveraging the class hierarchy to build an abstract probe that can be implemented by any probes requiring GitHub Action configuration checks.

The findings of this probe will be highly beneficial to the security team, as it helps identify any gaps and enables the improvement of the overall security of the Jenkins plugin ecosystem.

Read the security scan probe blog post for more details on the probe.

What about the second-half of GSoC?

In the second half of the GSoC project, I will be working on a few more probes, including the following:

  • Detect Renovate usage:

    • This probe will identify how many plugins use the Renovate bot CLI to detect outdated dependencies in the plugin.

    • If any outdated dependencies are found, it will automatically create a pull request (PR) to update the dependency.

  • Count open tickets:

    • This probe will detect open issues in both Github and JIRA.

In addition to these new probes, I will also be focusing on fixing bugs and making enhancements to the existing probes.

GitLab Plugin Modernization

The GitLab Plugin facilitates seamless interaction between Jenkins and GitLab. This project involves replacing the usage of the outdated RESTEasy library in the plugin with GitLab4J-API through the GitLab API Jenkins Library Plugin.

This migration holds significant importance for the following reasons:

  1. The migration will significantly reduce future maintenance issues.

  2. It makes the plugin more lightweight by reducing dependencies.

  3. It improves consistency with other Jenkins plugins.

  4. It enhances documentation and overall user experience.

During Coding Phase 1, the following achievements were made:

  1. Successfully completed the migration from RESTEasy to GitLab4J-API via the GitLab-API plugin.

  2. Conducted interactive testing of the partially migrated Plugin.

To use the upcoming modernized GitLab Plugin, please ensure the following requirements are met:

  1. Minimum GitLab version: 14.0.

  2. Minimum Jenkins version: 2.387.3.

Over the years, GitLab has evolved significantly. During the migration process, we discovered that the plugin will soon have the removal of GitLab API V3 support, as GitLab now only supports V4 of its API. We strongly encourage V3 users to switch to V4 as soon as possible.

The following tasks are still in progress during the 2nd Coding Phase:

  1. Adapting the Webhooks to GitLab4J-API’s events (completed)

  2. Migrating the Proxy settings

  3. Adapting the unit and integration tests (work in progress)

  4. Improving the documentation

Building Jenkins.io with Alternative Tools

Jenkins users frequently refer to jenkins.io for documentation, and a new LTS line is released and available to all Jenkins users worldwide every 12 weeks. However, the documentation for each prior edition is not currently maintained, and that’s where my project comes in to provide Versioned Documentation for the users of the Jenkins community.

With my project, users will be able to switch between different versions of the documentation, depending on the version of Jenkins they are running. To achieve this, we adopt Antora, which comes with versioning out-of-the-box and creates documentation pages using asciidoc. Additionally, Gatsby will be used to generate some of the YAML file-generated components. This way, we aim to enhance the accessibility and usability of the documentation for Jenkins users.

Till now, we have migrated all documentation from jenkins.io which leaves us only with the Gatsby part of the implementation. The completed components include:

  1. User Guide

  2. Solution Pages

  3. Tutorials

  4. Developer Guide

  5. Security

  6. Subprojects

  7. Community Section

With the successful migration of the above components, our next focus will be on working with the following components or single pages using Gatsby as the framework:

  1. Blogs

  2. Roadmap

  3. Security Advisories

  4. Download

  5. Changelog

  6. Upgrade Guide

  7. About

  8. Homepage

As we enter the 2nd Coding Phase of GSoC, completing these sections/pages will mark the end of the project and the full migration from Awestruct to Antora and Gatsby.

Docker-based Jenkins Quickstart Examples

The project’s goal is to address the problems and complexities users often face when setting up Jenkins with Docker. The current process involves multiple intimidating steps with security risks. So, how are we solving these problems?

The intimidating steps can be simplified by using Docker Compose to hide all the complexities of Docker. Additionally, we have created scripts to automate the running of the Jenkins container with Docker. One script is called jenkins_init.sh, and another script called jenkins_teardown.sh is used to clean everything.

To address the security risk of running jobs on the controller, we are using the controller and agent as separate containers. For this purpose, we use another script called keygen.sh that creates and updates SSH keys.

For the tutorial part, it is essential to lower the barrier to entry for Jenkins. Therefore, the user will only need to add a keyword to the jenkins_init.sh script. For example, for a Maven tutorial, the user can simply run ./jenkins_init.sh maven to start the tutorial.

Additionally, everything now works with Gitpod with just a click.

Things to do during the 2nd Coding Phase:

  • Add more tutorials and integrate them with ./jenkins_init.sh.

  • Provide support for Windows without WSL.

  • Regularly test all the tutorial files and scripts.

  • Develop concise and easy-to-understand documentation for new and updated tutorials.

End Note

A full recording from the meetup can be found HERE and slides are HERE.

Java 17 usage in Jenkins.io Documentation

$
0
0

Jenkins supports Java 17.

Jenkins has fully supported Java 17 since the 2.357 weekly release and 2.361.1 Long Term Support release. For this reason, we have transitioned to using Java 17 in the Jenkins user handbook. This includes the installation guides, Pipeline documentation, and Pipeline syntax. Refer to the GitHub issue for additional information and insight.

Background

Debian 12 was released on June 10, 2023, and it no longer delivers OpenJDK 11. Due to this change in Debian, we have transitioned to using Java 17 in the documentation. The idea is this will encourage users to upgrade to Java 17, which has been fully supported since September 2022. Aligning with the actively supported Java version will also help prevent user confusion or frustration when trying to follow installation and usage guides.

Upgrading your Java version

While updating the documentation to reflect Java 17 support, we also added instructions on upgrading Jenkins' Java version from 11 to Java 17. This page includes a video walkthrough created by Darin Pope that guides users through the process of upgrading the Java version for Jenkins.

As always, update all installed plugins prior to and after updating Jenkins. This ensures up-to-date plugins that remain compatible. If plugins are not updated both before and after the upgrade, you can encounter compatibility issues.

Jenkins July 2023 Newsletter

$
0
0

Jenkins July Newsletter

Key Takeaways

  • A Jenkins Core security advisory was published on July 26

  • The official documentation has migrated to Java 17

  • Operating system end of life notifications have been added

Security Update

Contributed by: Wadeck Follonier

During July, there were two Security Advisories published:

Governance Update

Contributed by: Mark Waite

  • The Jira upgrade was successful. Special thanks to the Linux Foundation for their continued support.

  • Mark Waite was elected to the CDF Governing Board as a committer representative.

  • Jenkins core continues to improve.

  • Jenkins plugins continue to improve.

Infrastructure Update

Contributed by: Damien Duportal

  • Rollout of Kubernetes 1.25.

  • There was a production outage on public services (download mirrors, LDAP, etc.) which led to a public IP change. More information can be found in the post mortem blog post.

  • Jenkins LTS'2.401.3 (security release) was deployed everywhere less than 2 hours after the security advisory.

  • ci.jenkins.io migrated to new (and more powerful) hardware and a new network for faster builds.

  • Windows 2022 agents reached general availability.

User Experience Update

Contributed by: Mark Waite

  • The browser confirm dialog has been replaced with friendly modal dialogs - Markus Winter

  • Safe restart has a better user experience - Jan Meiswinkel

  • The log manager page has more improvements - Jan Faracik

Platform Modernization Update

Contributed by: Bruno Verachten

Several platform updates occurred throughout June:

  • Updates on Docker Images

    • Ssh-agent(releases 5.6.0, 5.6.1, 5.6.2, 5.6.3, 5.7.0):

      • Tracking JDK versions for the next level of performance.

      • Bumping node alpine docker image to 18.16.1-alpine3.18.

      • Bump debian from bullseye-20230522 to bullseye-20230703

      • Install Debian ca-certificates package

    • Docker-agent (releases 3131.vf2b_b_798b_ce99-2, 3131.vf2b_b_798b_ce99-3, 3131.vf2b_b_798b_ce99-4)

      • Windows enthusiasts, rejoice! Introducing Windows Server and Nanoserver LTSC 2022 images.

      • More efficient image building with multi-stage image for Windows Server Core.

      • Bump debian from bullseye-20230502 to bullseye-20230703

      • Bump archlinux from base-20230430.0.146624 to base-20230702.0.161694

      • Bumping Git version on Windows to 2.41.0.windows.2

    • Inbound-agent (releases 3131.vf2b_b_798b_ce99-2 and 3131.vf2b_b_798b_ce99-3):

      • Bump the parent image jenkins/agent version to 3131.vf2b_b_798b_ce99-3

      • Now allows JENKINS_URL to be unset when JENKINS_DIRECT_CONNECTION is set.

    • Controller (releases 2.401.2, 2.412, 2.413, 2.414 and 2.415):

      • Keeping up-to-date with the latest Bullseye version and timezone changes.

  • Latest News

    • End of life operating systems:

      • Beginning with Jenkins 2.407, Jenkins administrators will be warned if they are running Jenkins on an operating system that is within 6 months of its end of life date.

      • The operating system end of life warning has been backported to Jenkins 2.401.2 after discussion in the Jenkins developer mailing list.

      • The Jenkins project does not test Jenkins on operating systems that are not supported by the operating system provider.

      • We don’t support Jenkins on operating systems that are not supported by the operating system provider.

      • These last months, we’ve seen OS vendors offer extended support contracts for their systems having reached their end of life. However, the Jenkins project has not extended its Linux support policy or its Windows support policy beyond the public end of life for any of those operating systems.

    • Java 17 instructions

      • Jenkins is transitioning from using Java 11 as the preferred version to using Java 17, which offers more functionality, speed, and development support.

      • The documentation and usage areas are being updated to reflect this change, while also clarifying that Java 11 support in Jenkins will continue.

Documentation Update

Contributed by: Kevin Martens

Over the course of July, six different blog posts were published from nine different authors, including updates on the Google Summer of Code projects that are in progress. Thanks to all the participants for their insights and work!

The Jenkins documentation has transitioned to using Java 17 within the installation guides and other documentation areas. This issue explains some more background of the transition and what work has been completed already. A blog post was published to highlight this change and provide background for the transition. The post also encourages users to upgrade so that they can enjoy the additional features and functions that come with Java 17.

Outreach and advocacy Update

Contributed by: Alyssa Tong

Google Summer of Code projects midterm evaluations are complete!

The midterm demos were presented via the Jenkins online meetup at the beginning of July, 2023. If you missed it, refer to the blog post for the recap.

Congratulations to all four GSoC Contributors for passing this milestone!

Onward to the second half of GSoC 2023.

Happy coding!

image

Linux containers rebuilt

$
0
0

A Jenkins job mistakenly rebuilt the Linux container images for recent Jenkins weekly releases and recent Jenkins LTS releases. Users that downloaded some of those Linux container images received container images that could not run the Jenkins controller. The incorrect container images would fail to run with the message that Jenkins is not supported with Java 8.

Running with Java 8 from /usr/lib/jvm/java-1.8-openjdk/jre,
which is older than the minimum required version (Java 11).
Supported Java versions are: [11, 17, 21]

Corrected Linux container images

The Linux container images for Jenkins 2.419, 2.401.1, 2.401.2, and 2.401.3 have been rebuilt August 21, 2023.

Container images for Jenkins agents are not affected.

More details are available in the original issue report.

Summer Internship in Jenkins security

$
0
0

Context

Jenkins is an open-source CI/CD solution that is extensible with a wide range of plugins that can be installed using the Jenkins plugin distribution repository or via manual installation.

This extensibility is a powerful feature of Jenkins, but it is a critical aspect that has to be secured to avoid risks and vulnerabilities that can impact the Jenkins system.

The internship took place while working in direct contact with the Jenkins Security Team, having daily meetings to discuss topics, possible blockers, and share ideas.

Internship Goals

The main goals of the internship were:

  • Understanding how some of the common vulnerabilities can be recognized in the Jenkins ecosystem by paying attention to common patterns related to best practices in terms of configuration and data flow control.

  • Auditing the plugins in a structural and ordered way, with comprehensive vulnerability scoring and classification using the MITRE standard, permitting them to be reviewed easily by the team.

  • Reporting findings to the team and plugin maintainers using the Jira issue tracker, having live sessions to help the latter ones have a full comprehension of the vulnerability reported.

Organizational Model

Good communication is a necessary basis for a good organizational model.

Regarding that, a Slack channel was opened where I could ask questions, open discussions, and share ideas.

In addition to that, there were scheduled weekly sessions with my manager, Wadeck Follonier, who leads the Jenkins Security Team, and with my co-pilot, Kevin Guerroudj.

During these sessions, I could open discussion for doubts, receive knowledge, and propose activities that I could do during the internship, as well as a daily session with the team.

To be more productive and get quick feedback from the team, a spreadsheet was made available to me to note the plugins audited with related security audit documents, where I reported my findings and associated security tickets.

Furthermore, CloudBees got me in touch with other interns working in different departments using a Slack channel and also scheduled monthly meetings to discuss our progress on our projects and the value added to them.

The journey

At the beginning of this security journey, I had the opportunity to quickly learn about the common vulnerabilities that we could find inside a Plugin, thanks to a cyber gym that my manager created and distributed on a GitHub repository.

It is essentially a Jenkins plugin, called Emmenthal Plugin, that is written with some bad patterns that the Jenkins documentation suggests to avoid and that permits to obtain common vulnerabilities like Cross Site Scripting, XML External Entities, Server Side Request Forgery, Path Traversal, and so on.

From that moment on, I enjoyed the journey.

I started the audit with easier and less popular plugins, and each day I increased the difficulty level with more popular plugins until the end of the journey.

It was obvious to me that the plugins with larger installation counts were also more secure due to the accrued attention they already got prior to now.

During the journey, I improved my skills on auditing thanks to a structured internal document that the team uses for auditing, and that helped me to be more precise and to not forget anything that caught my attention during the analysis of the plugins.

Following each finding, there was a brainstorming phase where I tried to classify the vulnerability in terms of CWEs and assign a CVSS score to understand the real impact and risks.

Finally, the reporting to the maintainers was done using Jira and described in a structured format that permits them to easily and quickly understand the point impacted inside the code and the reproduction steps to have a procedure to replicate it.

Results

The value that I added to the project can be measured in terms of vulnerabilities reported and confirmed by the Jenkins Security Team and based on the type of plugin affected.

In terms of numbers, I reported 23 vulnerabilities, split into two categories using “10k installations” as a threshold:

  • 19 vulnerabilities were found in less popular plugins.

  • 4 vulnerabilities were found in more popular plugins.

At the moment, only 3 vulnerabilities are known to the public in the Jenkins Security Advisories:

Conclusions

I enjoyed this opportunity.

It helped me to be more confident in a practical manner on the security aspects and also permitted me to learn new ways to do audits.

I obtained good results in terms of the contributions of the Jenkins ecosystem, and it will have a positive impact on my professional career.

Message from the co-pilot

Andrea quickly stood out during his internship.

His ability to understand and adapt to our tasks was impressive.

He always asked the right questions and showed a real passion for learning.

It was great to work with someone so eager to grow in this field.

Message from the security officer

Andrea was able since the beginning and until the end, to ask very smart questions, showing his deep interest in the cybersecurity field.

He was able to demonstrate great analytical skills.

This kind of internship is continuously showing me that this kind of project is worth it for multiple reasons: for the intern themself, to learn new topics and get practical experience, but also for the community in terms of security improvement and also for the rest of the team, to have mentoring opportunities.


GSoC GitLab Plugin Modernization Project

$
0
0

The goal of this blog is to showcase the work done on GitLab Plugin during the Google Summer of Code 2023. For a detailed description of the project, please refer to the project page.

About Project

Plugins grow old and become difficult to maintain with time. GitLab Plugin was one such plugin that needed some love and care. The GitLab Plugin Modernization project thus aimed to modernize the GitLab Plugin by replacing the (RESTEasy) library with modern (GitLab4J-API) library via Jenkins (GitLab API plugin). The replacement normalizes the use of the GitLab API with regard to the (GitLab Authentication), (GitLab Branch Source), and (GitLab Logo) plugins. This modernization reduces future maintenance burden and improves the overall quality of the plugin by making it more lightweight due to reduced dependencies and enhanced documentation.

The migration was performed in the following 2 coding phases:

Phase 1

In this phase, the following tasks were completed:

  • Migration of GitLab Plugin from RESTEasy to GitLab4J-API via the GitLab-API plugin, without affecting the webhook functionality of the plugin. (PR)

  • Interactive testing of partially migrated plugin.

The achievements of this phase were demonstrated in the Midterm Presentation.

Phase 2

In this phase, the following tasks were completed:

  • Adapting webhooks implementation to GitLab4J-API.(PR)

  • Migrating the Proxy Settings from RESTEasy to GitLab4J-API.(PR)

  • Partially adapting the Unit and Integration tests.

  • Improving the documentation of the plugin.(PR)

  • Interactive testing of the fully migrated plugin.

The results of this phase will be showcased in in the Final Presentation in the future.

Future Improvements

  • As GitLab has discontinued the GitLab API V3, the plugin should only support GitLab API V4 and above. Thus the plugin will be updated to support GitLab API V4 and above only.

  • The plugin lacks the test coverage, thus the plugin’s Unit, Integrations as well as Docker-based Tests will have to be updated.

  • GitLab 16.0 has introduced some breaking changes which will have to be handled in the plugin.

Acknowledgments

My gratitude knows no bounds that I got the opportunity to contribute to such impactful project under Google Summer of Code with such kind mentors. I would like to thank my mentors, Kris Stern and Mark Waite for their constant support and guidance throughout the project. They put in a lot of effort in interactively testing the migrated plugin and providing me with valuable feedback. They never let me be stuck in any difficulty for too long and always helped me in finding the solution to the problem. I would also like to thank Basil Crow for the guidance he provided me with during the initial phase of the project, which helped the team shape the project.

Also, thanks to the org admins, Jean-Marc Meessen, Alyssa Tong, Kris Stern, and Bruno Verachten, for always keeping me and other contributors on our toes and assisting us with any blockers and concerns by organizing weekly stand-up calls and personal 1-to-1 meets every phase.

Docker-based Jenkins quick start examples

$
0
0

Jenkins GSoC

About the Project

The aim of my project was to tackle the challenges and intricacies that users frequently encounter when configuring Jenkins with Docker. The current process comprises numerous daunting steps associated with security risks. My objective was to resolve this issue using Docker Compose by crafting a Docker Compose file and conducting regular testing. Furthermore, the project included the creation of user-friendly documentation, tutorials, and a streamlined Gitpod setup accessible with just one click.

What will change in Jenkins after this project:

  • Previously, users had to follow six intimidating steps to install Jenkins with Docker, which appear as follows:

    Creating docker in docker container

    Building dockerfile

    Creating container from previous image

  • After this project, users will only need two commands: git clone URL and docker-compose up -d.

  • This also applies to tutorials that utilize the same Docker setup as mentioned above. After cloning the repository, users will only need to execute docker-compose up -d <keyword>.

  • Additionally, users can take advantage of a one-click setup using Gitpod to easily experience Jenkins.

  • Currently, these Docker Compose files undergo regular testing via GitHub Actions. We plan to transition this process to Jenkins in the near future.

Phase 1

During Phase 1, we aimed to accomplish this objective using scripts. We developed jenkins_init.sh and jenkins_teardown.sh scripts to initiate and stop the containers. Additionally, we crafted scripts for SSH key generation and the cloning/forking of repositories.

Throughout Phase 1, I gained insights into:

  • Optimal Git and GitHub practices

  • Effective conflict resolution in merging

  • The structure of the Jenkins file system

  • Creation and utilization of Docker Compose files

  • Crafting of shell scripts

  • Building Dockerfiles

  • Understanding of SSH key mechanisms

  • Navigating the functionality of Gitpod

  • Composing technical documentation

Phase 2

In Phase 2, our focus shifted towards integrating Windows as well, given that the shell scripts from Phase 1 were incompatible without Windows Subsystem for Linux (WSL). During this phase, we uncovered the utility of Docker Compose profiles, which turned out to be an excellent solution. These profiles allowed the tutorials to function similarly to our previous scripts, employing designated keywords.

A crucial aspect of the project was automated testing, which introduced me to GitHub Actions, a tool I hadn’t previously experienced. GitHub Actions were employed to test the functionality of Docker Compose files, facilitate the uploading of Docker images to Docker Hub, perform plugin updates, and manage other various tasks.

During Phase 2, my learning encompassed:

  • Docker Compose files

  • GitHub Actions

  • The Jenkins Remote Access API

Next Steps

The upcoming steps for the project involve:

  • Full integration into jenkins.io.

  • Transitioning from GitHub Actions to jenkins.ci.

  • Incorporating the repository into Hacktoberfest with "good first issues" to engage beginner contributors.

  • As a stretch goal, expanding the project with additional tutorials.

Acknowledgments

I am grateful for the guidance and support provided throughout this project from my mentors Bruno Verachten, Bervianto Leo Pratama, and Jean-Marc Meessen.

Conclusion

In conclusion, participating in GSoC 2023 was an incredible experience that allowed me to learn new skills and contribute to an open-source project. I am grateful for the opportunity and would like to thank my mentors and the Jenkins community for their support and guidance. I look forward to continuing to contribute.

A glimpse of the future

Next: Number of Open Issues Probe
Previous: Docker-based Jenkins quick start examples
$
0
0

image

You’ve all seen it before: the bitter sting of botched predictions. Flying cars, nuclear ovens in the kitchen, killer robot dogs, and lunar living - all expected to be part of our daily lives by now. Well, some of these wild dreams did come to pass, but I’m not about to roll the dice and predict what our world will look like 50 years hence. Instead, let me paint a picture of Jenkins' future up until October 2024.

Why am I being so specific about this end date, you ask? It’s because I want to highlight two major projects that Jenkins might tackle before the leaves start to fall next year.

One of these projects is the potential leap to JDK21, and we’ve got a proposed roadmap for that. Jenkins will stop supporting JDK 11 in October 2024 when many Java providers (Eclipse Temurin, OpenJDK, …​) stop supporting Java 11. This transition is one of the more significant moves Jenkins might be considering in the coming months.

The next project doesn’t have a roadmap within Jenkins yet, but the progress is promising, and it’s intertwined with the first project like two vines on a trellis. As some of you may know, I’ve got a fondness for CPU architectures that were once seen as outlandish or still raise a few eyebrows. I’m referring to ARM and RISC-V, of course.

ARM is making strides, finding its way into everything from laptops to servers. So, my personal campaign to see ARM recognized as a first-class citizen has been successful.

Now, I’m turning my attention to RISC-V - my next personal endeavor and hobby. While it’s not a fresh face in the CPU crowd, it’s currently experiencing a notable uptick in popularity. The emergence of numerous single-board computers, workstations, and servers reminds me of the early days of ARM, but amplified. I’ve touched on this before in one of my blog posts.

Let’s explore how JDK21, Jenkins, and RISC-V might be connected, at least in the maze of my musings.

Java Version Planning for Jenkins

The ink’s barely dry on the plans for which versions of the JDK the Jenkins project will support this year and the next. What I’m about to spill is just a snippet from a proposal whipped up by Mark Waite. For now, it’s about as definitive as a weather forecast.

There’ll be more detailed blog posts down the line when the final decision is made and the roadmap is as clear as a bell.

Java 11:

  • Java 11 OpenJDK’s public support ends in late 2024.

  • Eclipse Temurin, Red Hat, and Microsoft will support Java 11 through October 2024.

  • Oracle extends support until October 2026.

  • Amazon Corretto supports Java 11 until September 2027.

  • Proposal: End-of-life for Java 11 in Jenkins is suggested for October 31, 2024.

  • Jenkins LTS support for Java 11 will end with the August 7, 2024 release.

  • Java 21 will become the required baseline starting with the September 4, 2024 LTS release.

  • Admin monitors announcing Java 11’s end of life will be enabled in Jenkins weekly releases by October 3, 2023 and in the December 13, 2023 LTS release.

Java 17:

  • Java 17 adoption is increasing in Jenkins. The Jenkins documentation now recommends Java 17 in the installation guides and in the tutorials.

Java adoption rates based on Java version.

Java 21:

  • Java 21 releases on September 19, 2023.

  • Oracle, Eclipse Temurin, Red Hat, and others plan to support Java 21 as a long term support release.

  • Proposal: Jenkins aims to support Java 21 in weekly releases by the end of October 2023.

Are we confident Jenkins will be running on top of JDK 21 next October?
I guess so. Thanks to the work of Stéphane Merle, JDK21 is already available at https://ci.jenkins.io to build plugins. You just have to add one line to your Jenkinsfile, and your plugin will be tested on JDK21.

configurations:[[platform:'linux',jdk:'17'],[platform:'linux',jdk:'21'],[platform:'windows',jdk:'11']]

Furthermore, thanks to the work of the community (and particularly Basil Crow), Jenkins 2.419 and 2.420 no longer require the --enable-future-java flag for JDK 21 beta. Jenkins 2.418 and prior require the flag for Java 21 beta.

$ java -version
openjdk version "21-beta" 2023-09-19
OpenJDK Runtime Environment Temurin-21+34-202308082331 (build 21-beta+34-202308082331)
OpenJDK 64-Bit Server VM Temurin-21+34-202308082331 (build 21-beta+34-202308082331, mixed mode, sharing)$ java -jar jenkins-2.417.war
Running with Java 21 from /opt/jdk-21, which is not yet fully supported.
Run the command again with the --enable-future-java flag to enable preview support for future Java versions.
Supported Java versions are: [11, 17]
See link:https://jenkins.io/redirect/java-support/ for more information.

$ java -jar jenkins-2.419.war
Running from: /home/mwaite/bugs/jenkins-2.419.war
webroot: /home/mwaite/.jenkins/war
2023-08-24 15:42:32.857+0000 [id=1]     INFO    winstone.Logger#logInternal: Beginning extraction from war file`

I also heard from Basil Crow during the latest Governance board meeting that the Jenkins BOM had already run with JDK21; that’s good news. We will have to stay alert because of things that won’t migrate easily (think of Groovy for example), but if it’s too easy, it’s no fun, right?

Jenkins and RISC-V

The Jenkins project is already churning out aarch64 (64-bit ARM) Docker images for both the controller and agents. On top of that, we’re putting aarch64 through its paces with regular testing, and some parts of the Jenkins infrastructure are already humming along on aarch64 hardware.
We can’t say the same for RISC-V, and for good reason. RISC-V isn’t a supported CPU architecture for Jenkins, it hasn’t been put to the test, Docker isn’t officially on board with this architecture yet (even if Kubernetes is already on the bandwagon), and the Jenkins project doesn’t own a single RISC-V machine.

A few months back, I whipped up a Jenkins agent for RISC-V. Sadly, the machine I had at my disposal wasn’t up to the task of hosting a Jenkins controller. At that time, I was using a nightly build of JDK19 by Temurin.

These days, I’ve got another RISC-V machine that’s got more cores and more memory than the last one, and it meets the Jenkins recommendations, which are:

  • 4 GB+ of RAM

  • 50 GB+ of drive space

Here I’m wielding the StarFive VisionFive2, which boasts 8GB of RAM, 4 RISCV64 cores clocking up to 1.5GHz, and for now, a 128GB SDCard. I’ll swap it out for an NVMe disk when I get around to it. It’s built on the JH7110 from StarFive. We’re seeing this SoC pop up on SBCs more and more these days (Star64, and so on).

Since the machine is (on paper) up to the task of running Jenkins, we should give it a whirl, right?
I loaded a snapshot version of Debian onto the board after giving the firmware a tune-up, and away we went.

RISC-V and JDK

JDK17

The logical first step would be to install a version of the JDK, then follow the official documentation to get Jenkins up and running on Debian, right?

I’ve already been burned by the default JDK on RISC-V with Debian, which turns out to be a Zero VM.
If I were to kick things off with the default JDK, I reckon its performance would leave me so frustrated, I wouldn’t bother going any further.
You’re not buying it? Alright, I see how it is. Let’s not install Jenkins the old-fashioned way then, let’s give it a whirl on the command line after installing the default JDK.

sudo apt install openjdk-17-jdk-headless

java -version
openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment (build 17.0.5+8-Debian-2)
OpenJDK 64-Bit Zero VM (build 17.0.5+8-Debian-2, interpreted mode)

Yes, we’re using a Zero VM. Now onto the Jenkins war download.

curl -L-o /tmp/jenkins.war link:https://updates.jenkins.io/latest/jenkins.war

Let’s launch Jenkins on the command line:

java -jar /tmp/jenkins.war

Running from: /tmp/jenkins.war

webroot: /home/user/.jenkins/war

2023-08-06 12:31:15.432+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file

Let’s say I let it run for a good while, and nothing else happened.
A Zero VM is about as useful for running a server as a chocolate teapot.
Let’s switch gears and go with something that’s got a bit more pep in its step.

JDK21

As I’m penning this article, there’s no official JDK21 release we can snag from the Eclipse Temurin Latest Releases page.
We’ll have to scrounge up a nightly build from the Adoptium Temurin 21 binaries repo.
RISC-V binaries aren’t exactly churned out daily, so you might have to do a bit of digging to find a release with RISC-V binaries.
At the time of writing this blog post, the latest available is release jdk21-2023-08-08-20-16-beta.
In this release, there are several RISC-V binaries up for grabs, but we don’t need static libs or a debug image, so we’ll opt for OpenJDK21U-jdk_riscv64_linux_hotspot_2023-08-08-20-16.tar.gz.

Let’s reel in the JDK21 binaries:

curl -L-Olink:https://github.com/adoptium/temurin21-binaries/releases/download/jdk21-2023-08-08-20-16-beta/OpenJDK21U-jdk_riscv64_linux_hotspot_2023-08-08-20-16.tar.gz

Now that we have them, let’s install them on the machine.

sudo mkdir /opt/jdk21
sudo tar-xzf OpenJDK21U-jdk_riscv64_linux_hotspot_*.tar.gz -C /opt/jdk21 --strip-components=1

Once it’s installed, let’s inform the system about this new set of java binaries:

sudo update-alternatives --install /usr/bin/java java /opt/jdk21/bin/java 1
sudo update-alternatives --install /usr/bin/javac javac /opt/jdk21/bin/javac 1
sudo update-alternatives --install /usr/bin/javadoc javadoc /opt/jdk21/bin/javadoc 1

These commands create alternatives for the java, javac, and javadoc commands and associate them with the respective binaries in the JDK 21 installation.

After installing the alternatives, you need to select the default one. Run the following command and choose the number corresponding to the JDK 21 alternative in the presented menu:

sudo update-alternatives --config java

Repeat this for javac and javadoc:

sudo update-alternatives --config javac
sudo update-alternatives --config javadoc

Verify the Default Java Version: After configuring the alternatives, you can verify that JDK 21 is the default Java version by running:

java -version

Jenkins and JDK21

On the command line

Let’s try to launch Jenkins with JDK21 now, and see if it gets any better than with JDK17:

java -jar /tmp/jenkins.war +
Running from: /tmp/jenkins.war
webroot: /home/user/.jenkins/war
2023-08-24 08:35:11.202+0000 [id=1] INFO winstone.Logger#logInternal: Beginning extraction from war file
2023-08-24 08:35:17.635+0000 [id=1] WARNING o.e.j.s.handler.ContextHandler#setContextPath: Empty contextPath
2023-08-24 08:35:17.947+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: jetty-10.0.15; built: 2023-04-11T17:25:14.480Z; git: 68017dbd00236bb7e187330d7585a059610f661d; jvm 21-beta+34-202308081713
2023-08-24 08:35:19.288+0000 [id=1] INFO o.e.j.w.StandardDescriptorProcessor#visitServlet: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
2023-08-24 08:35:19.521+0000 [id=1] INFO o.e.j.s.s.DefaultSessionIdManager#doStart: Session workerName=node0
2023-08-24 08:35:22.058+0000 [id=1] INFO hudson.WebAppMain#contextInitialized: Jenkins home directory: /home/user/.jenkins found at: $user.home/.jenkins
2023-08-24 08:35:22.647+0000 [id=1] INFO o.e.j.s.handler.ContextHandler#doStart: Started w.@2a9bc08f\{Jenkins v2.420,/,file:///home/user/.jenkins/war/,AVAILABLE}\{/home/user/.jenkins/war}
2023-08-24 08:35:22.698+0000 [id=1] INFO o.e.j.server.AbstractConnector#doStart: Started ServerConnector@43599640\{HTTP/1.1, (http/1.1)}\{0.0.0.0:8080}
2023-08-24 08:35:22.743+0000 [id=1] INFO org.eclipse.jetty.server.Server#doStart: Started Server@b83a9be\{STARTING}[10.0.15,sto=0] @14031ms
2023-08-24 08:35:22.746+0000 [id=35] INFO winstone.Logger#logInternal: Winstone Servlet Engine running: controlPort=disabled
2023-08-24 08:35:23.763+0000 [id=42] INFO jenkins.InitReactorRunner$1#onAttained: Started initialization
2023-08-24 08:35:23.820+0000 [id=40] INFO jenkins.InitReactorRunner$1#onAttained: Listed all plugins
2023-08-24 08:35:28.157+0000 [id=40] INFO jenkins.InitReactorRunner$1#onAttained: Prepared all plugins
2023-08-24 08:35:28.180+0000 [id=40] INFO jenkins.InitReactorRunner$1#onAttained: Started all plugins
2023-08-24 08:35:28.204+0000 [id=40] INFO jenkins.InitReactorRunner$1#onAttained: Augmented all extensions
2023-08-24 08:35:29.182+0000 [id=46] INFO jenkins.InitReactorRunner$1#onAttained: System config loaded
2023-08-24 08:35:29.185+0000 [id=40] INFO jenkins.InitReactorRunner$1#onAttained: System config adapted
2023-08-24 08:35:29.187+0000 [id=43] INFO jenkins.InitReactorRunner$1#onAttained: Loaded all jobs
2023-08-24 08:35:29.194+0000 [id=43] INFO jenkins.InitReactorRunner$1#onAttained: Configuration for all jobs updated
2023-08-24 08:35:29.366+0000 [id=60] INFO hudson.util.Retrier#start: Attempt #1 to do the action check updates server
2023-08-24 08:35:31.242+0000 [id=45] INFO jenkins.install.SetupWizard#init:
***************************************************************************************************************************************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
2c4d91ba22d24f639a59ad50e6d82686
This may also be found at: /home/user/.jenkins/secrets/initialAdminPassword
***************************************************************************************************************************************************************************************

Jenkins coughed up this log a few seconds after I fired off the command, so it looks like we’re in business.
Notice anything odd? Shouldn’t Jenkins give us a heads-up that it’s not meant to run with JDK21? Aren’t JDK17 and JDK11 the only ones getting the official nod?
Up until a few weeks ago, that was the score…​ And it’ll stay that way for the LTS versions until next October. At the start of August 2023, this PR got the green light, and since then, there’s no need to add the --enable-future-java flag to give JDK21 versions the go-ahead.

Let’s take the current LTS for a spin:

curl -L-o /tmp/jenkins.war link:https://get.jenkins.io/war-stable/latest/jenkins.war
java -jar /tmp/jenkins.war
Running with Java 21 from /opt/jdk21, which is not yet fully supported.
Run the command again with the --enable-future-java flag to enable preview support for future Java versions.
Supported Java versions are: [11, 17]
See link:https://jenkins.io/redirect/java-support/ for more information.

As you can see, the current LTS does not support JDK 21 yet.

Jenkins standard package installation

As we’re using Debian, let’s go with the standard installation of the weekly release now.
Unfortunately, we get an error when installing the Jenkins package the official way.

Job for jenkins.service failed because the control process exited with error code.
See "systemctl status jenkins.service" and "journalctl -xeu jenkins.service"for details.

These commands don’t say much to help with understanding what the problem is.
Let’s try another way:

/usr/bin/jenkins
jenkins: invalid Java version: openjdk version "21-beta" 2023-09-19
OpenJDK Runtime Environment Temurin-21+34-202308081713 (build 21-beta+34-202308081713)
OpenJDK 64-Bit Server VM Temurin-21+34-202308081713 (build 21-beta+34-202308081713, mixed mode, sharing)

Now it’s clear as day: we’ve installed a JDK21 version that plays nice with the WAR file, but the scripts tied to systemd aren’t up to speed with this JDK version yet.
They’re still checking if we’re using JDK11, 17 or 21, but they’re not ready for prime time, hence the failure.
So, how do we fix this mess?

Tweaking the package installation

The official documentation tells us we can override systemd service configurations thanks to

sudo systemctl edit jenkins.

This gives us something like:

### Editing /etc/systemd/system/jenkins.service.d/override.conf### Anything between here and the comment below will become the new contents of the file[Service]
Environment="JAVA_OPTS=-Djava.awt.headless=true -Xmx1024m"Environment="JENKINS_OPTS=--enable-future-java"

I have just added the last line in the hope of getting Jenkins to start.

sudo systemctl daemon-reload and sudo systemctl start jenkins should now be enough to get Jenkins started.

Yes, we’re almost good to go:

sudo systemctl status jenkins
● jenkins.service - Jenkins Continuous Integration Server
Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; preset: enabled)
Drop-In: /etc/systemd/system/jenkins.service.d
└─override.conf
Active: activating (start) since Thu 2023-08-24 09:28:34 UTC; 1min 2s ago
Main PID: 7138 (java)
CPU: 2min 32.701s
CGroup: /system.slice/jenkins.service
└─7138 /usr/bin/java -Djava.awt.headless=true-Xmx1024m-jar /usr/share/java/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8>

Why almost? We’ve still got a couple of hitches. The first one is the timeout. These RISC-V machines don’t have an optimized kernel yet, so a lot of things are slower than molasses in January. Too slow.
We’d best give Jenkins a bit more time to get its act together, just to be on the safe side.
Let’s bump up the timeout. Like before, we’ll tweak the configuration: courtesy of sudo systemctl edit jenkins.

### Editing /etc/systemd/system/jenkins.service.d/override.conf### Anything between here and the comment below will become the new contents of the file[Service]
Environment="JAVA_OPTS=-Djava.awt.headless=true -Xmx1024m"TimeoutStartSec=390

### Lines below this comment will be discarded

You may have spotted I removed Environment="JENKINS_OPTS=--enable-future-java" from the settings. We’re running the last weekly release of Jenkins that can handle JDK21, so why should we keep that?
We’ve got another fly in the ointment: the existing jenkins script in /usr/bin/jenkins is pulling a rabbit out of its hat with sed, trying to pin down a valid release. Unfortunately, our Java binary spits out something like “21-beta”. The script then trips up on validation, and in the end, throws a tantrum and refuses to start Jenkins.
Our workaround (while we twiddle our thumbs waiting for this PR to build the next package) is to tweak this file. Change line 40 so it reads:

awk-F'"''/version/ \{print $2}' | awk-F'.''\{match($1, /^[0-9]+/); print substr($1, RSTART, RLENGTH)}')

Once you’ve done that, fire off sudo systemctl daemon-reload and sudo systemctl restart jenkins and voilà, you’ve got a Jenkins instance running on JDK21 on a RISC-V machine.
For those of you still hanging in there, yes, there’s a quicker workaround…​
Just stick the Environment="JENKINS_OPTS=--enable-future-java" line back in the override.conf file, and you’ll have a Jenkins instance up and running with JDK21 faster than you can say "compile".

image

What’s next?

We’ve now got a fresh-off-the-press Jenkins instance humming along thanks to a nightly build of the JDK21 by Temurin.
That’s all well and good, but how do we install newer versions of the JDK when they roll off the assembly line?
One of these days, we’ll have regular JDK21 releases available straight from the package manager, but until that day comes, how are we supposed to update our installed version?

Well, I don’t have a silver bullet, but how about using Jenkins?
I’ve whipped up a nifty little Jenkinsfile that checks every day at 2:30 AM UTC if Temurin has pushed out a new nightly build for RISC-V.
If it strikes gold, it installs it into /home/jenkins/jdk-21.
It’s not exactly a masterpiece, since the version we’re using is in /opt/jdk21, but we could tweak it to shoot us an email when it finds something.
We’d then have to put Jenkins on ice, shuffle the contents of /home/jenkins/jdk-21 over to /opt/jdk21, and then wake Jenkins back up.

If you’re itching to give it a whirl, just whip up a new pipeline from source control and point it at the repo I mentioned earlier.
I’ll be the first to admit, even with an email notification, this workflow isn’t exactly a walk in the park, but it should be a stopgap solution, since Java 21 is set to launch on September 19, 2023.

Keep your ears to the ground for more updates and thrilling developments as we forge Jenkins' future together.

Number of Open Issues Probe

Next: DevOps World Tour 2023
Previous: A glimpse of the future
$
0
0

Introduction

An alarming number of issues in a plugin may signify that the plugin is outdated, in need of a new maintainer, or in need of a complete revamp. Open issues might also mean that there are a lot of new feature requests, which in turn means the plugin has quite a number of active users. The number of open issue probe counts the number of open issues in the issue tackers.

As of now, the probe only counts open issues in GitHub and the JIRA issue tacker.

Importance of the probe

The goal of the probe is to help maintainers and contributors identify plugins they want to invest their time into. This would lead to maintainers keeping plugins updated and useful for the users.

Tasks and Challenges

During the Google Summer of Code project duration, I was intended to:

  • Figure out the correct class design.

  • Understand the Update Center and how it works.

  • Implement the GitHub and JIRA APIs.

Outcome

The probe was approved and ready to be merged.

Learning

Over the course of the project, I learned about the correct usage and implementation of the abstract class. This was a more complex version of the class I was used to.

Conclusion

This probe will help maintainers in identifying the number of open issues a plugin has. This will also keep the plugin relevant to the latest Jenkins updates.

For more information or to find answers to any questions you might have, please visit the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.

DevOps World Tour 2023

Next: Stop Caching Maven Central
Previous: Number of Open Issues Probe
$
0
0

image

DevOps World Tour is coming to a city near you with events in multiple locations across the US, UK, and Asia. Jenkins project contributors will be attending and presenting in each city on the tour.

Tim Jacomb - London

image

Tim Jacomb will be presenting in London. Tim is a lead software engineer at Kainos and has been the the Jenkins release officer since December 2020.

He led the delivery of significant improvements to Jenkins core, including user interface modernization, major internal component upgrades, and more. He automated the creation of the Jenkins changelog. He was recognized by his peers in the Jenkins project as the Most Valuable Jenkins Contributor for 2020 and 2021.

Olivier Lamy - Singapore

image

Olivier Lamy will be presenting in Singapore. Olivier is a Jenkins core maintainer and a maintainer of more than 80 Jenkins plugins. He’s been a Jenkins committer for more than 13 years.

He is a member of the Apache Software Foundation and an active committer on multiple projects. He has served as an Apache project management committee member on multiple projects including Maven, Jetty, Tomcat, Commons, Cloudstack, Archiva, and more.

Mark Waite - New York, Chicago, Santa Clara

image

Mark Waite will be presenting in New York, Chicago, and Santa Clara.
Mark is a member of the Jenkins board, a Jenkins core maintainer, and the maintainer of the Jenkins git plugin and a number of other plugins.

He served as Jenkins documentation officer in the past and was one of the creators of the "Improve a plugin" tutorial. Mark is involved in many Jenkins efforts, including Google Summer of Code. user experience, documentation, and platform special interest groups.

Let’s talk about Jenkins

Each of the conferences will include a wide range of speakers in a single day of presentations. After the day of presentations, we’ll host a "Let’s talk about Jenkins" session. Conference attendees are invited to join us as we discuss Jenkins, listen to users, and identify ways that Jenkins can help users succeed.

Register for the DevOps World Tour near you at the conference website. Jenkins users and contributors are invited to use the DW23JENKINS discount code. See you at the conference!

Stop Caching Maven Central

Next: Detecting deprecated JSR-305 imports Probe
Previous: DevOps World Tour 2023
$
0
0

image

JFrog has been a sponsor of the Jenkins project for many years. We’re delighted that they continue to sponsor the Jenkins project and continue to provide our artifact hosting service, repo.jenkins-ci.org.

Releases, incremental development builds, and snapshots of Jenkins core, Jenkins tooling, Jenkins plugins, and Jenkins infrastructure components are hosted on JFrog Artifactory. The worldwide Jenkins community has been well served for many years by JFrog’s hosted artifact repository.

Maven Central

Effective Thursday September 7, 2023, we will stop caching the Maven Central repository from repo.jenkins-ci.org. The change will be transparent to Jenkins developers because Apache Maven automatically appends the Maven Central repository to its list of repositories. Beginning Thursday September 7, 2023, Jenkins developers will receive Maven Central artifacts from Maven Central instead of receiving them from the Jenkins Artifactory.

In case of issues, please open a Jenkins help desk ticket.

If you’d like more details of the bandwidth misuse investigation, refer to the help desk ticket.

Bandwidth misuse - the long story

Bandwidth misuse can happen in many different ways. Most of our bandwidth misuse was innocent misconfiguration. You can benefit users, the Jenkins community, and Jenkins sponsors by better configuration.

When the bandwidth misuse was not innocent misconfiguration, it was bad programming and bad monitoring of a server. You can benefit users, the Jenkins community, and Jenkins sponsors by monitoring bandwidth used by your servers.

Correct tool configuration

Two cases involved a Jenkins tool binary that was being downloaded repeatedly from Artifactory. Ephemeral Jenkins agents were configured to download the tool every time they started. The download source was configured to read the tar.gz tool installer from the Jenkins Artifactory repository. Once the tool was downloaded from Jenkins Artifactory, it was unpacked, installed, and used.

The download process took time because it was reading from the public internet to a local agent. The unpack and install took time because it was performed every time an ephemeral agent started.

Please install remote tools onto the agent before connecting the agent to the controller

Please download a local copy of the tool if you must download tools onto the agent

Don’t waste the time and bandwidth to download and install a tool from a remote location after the agent starts. The OpenJDK project, Eclipse Temurin project, and the Apache Maven project all thank you for not repeatedly downloading the same binary from their servers. Jenkins thanks you as well.

Abuse reports to cloud providers

Another case involved a rogue server that was repeatedly downloading all versions of the Jenkins war files. That rogue server was using over 20TB per month for its repeated downloads of the same files.

We reported the misuse to the abuse reporting service of the cloud provider that was hosting the server. The abuse reporting service told us they would investigate. The cloud provider took no further actions.

Please choose cloud providers that honor abuse reports

We blocked the IP address of that server, hoping that the server would not switch to a new IP address and resume its abusive downloads. The IP block stopped the abusive requests.

Use local caching

Another case involved agents from the Jenkins project that were repeatedly downloading dependencies as part of their regular build processes. Because we prefer ephemeral agents, the dependencies were being downloaded to the newly launched agent while it was building Jenkins core and Jenkins plugins.

Herve Le Meur and Damien Duportal implemented an artifact caching proxy using nginx for each of the cloud providers that host agents for the Jenkins project. The artifact caching proxy is hosted on the same cloud provider that runs the agents. It caches requests from agents running on that cloud provider.

Mark Waite’s home lab appeared in the top 5 consumers in a slightly different case. Mark corrected job configuration errors in his home lab that were causing individual jobs to download all their dependencies to a job-specific cache folder. Job specific caching caused the discs to fill on his agents and wasted bandwidth from the repository server. Fixing the job configuration error removed that home lab from the list. Mark implemented a central cache in his home lab in an attempt to prevent the issue in the future.

Cache your dependencies

Use local drives and local servers to cache your dependencies so that the can be downloaded faster.

Summary

We’ve learned to analyze log files with SQL queries thanks to the Artifactory SQL tool provided by Basil Crow. We upload artifactory logs into a SQLite database and can then use SQL select statements to identify patterns and trends. Thanks to Basil for a very helpful analysis tool.

Special thanks to JFrog for their patience and perseverance while we worked through these improvements.

Detecting deprecated JSR-305 imports Probe

Next: Jenkins Board and Officer Elections 2023 - Nominations Open
Previous: Stop Caching Maven Central
$
0
0

Introduction

This blog post describes the latest addition to the plugin health scoring system developed as part of my Google summer of Code Project: a probe that detects usages of the JSR-305 framework in plugins.

The JSR-305 framework provides annotations to detect software defects. Its executive committee voted it as dormant since May 2012. It is important to stop using the framework for the following reasons:

  • It is no longer maintained.

  • The annotations have questionable license.

  • The annotations are in the reserved javax namespace and there is no public release of the spec. Since it is not actively maintained, there is little chance this would be published.

For more insight, refer to the Jenkins developer mailing list conversation.

The @Nonnull and @CheckForNull annotations, which are part of JSR-305, have been deprecated since 2016.

The JSR-305 probe checks whether the deprecated annotations are used in a plugin.

Importance of the probe

This probe identifies the outdated annotations used in a plugin and highlight them in the scoring system.

Challenges and Learning

The outdated annotations are imported in .java files. My first approach was to go for an abstract class because I assumed there could be other probes that may need to look for code in .java files. After a series of reviews, one of the mentors pointed out that to comply with the OpenWrite rule, all imports from the javax.annotation package should be checked for deprecation.

Also, based on the YAGNI principle ("You Aren’t Gonna Need It"), which means features are developed only when you need them, the abstract wasn’t the current requirement. I restructured the code and used a single class to implement the probe.

Conclusion and Outcome

This probe will identify the deprecated annotations and let maintainers know which imports in a plugin require replacement.

For more information or to find answers to any questions you might have, please visit the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.


Jenkins Board and Officer Elections 2023 - Nominations Open

Next: Hacktoberfest 2023
Previous: Detecting deprecated JSR-305 imports Probe
$
0
0

We are excited to announce the 2023 Jenkins Governance Board and officer elections!

Nominations

Nominations can be submitted for two governance board positions, and all five officer positions (Security, Events, Release, Infrastructure, and Documentation).

During the registration period, we invite community members to nominate candidates by sending a message to the election-committee group. In your message, please include the name of the nominee, the specific position and your reasons for nominating that person. The nomination period ends on October 27. Nominees will be notified, and asked to confirm that they are interested in running as a candidate. The list of candidates will be announced on November 06.

Anyone can nominate anyone as candidate!

Many thanks to Oleg Nenashev and Mark Waite for serving on the Jenkins Governance Board. We also want to thank Damien Duportal as Infrastructure Officer, Wadeck Follonier as Security Officer, Alyssa Tong as Events Officer, Tim Jacomb as Release Officer, and Kevin Martens as Documentation Officer for all of their work over the last term.

The election consists of 4 phases:

  • Nomination of candidates (September 18 - October 27)

  • Voter registration (September 18 - November 05)

  • Voting (November 06 - December 1)

  • Results announcement (December 11)

Voting

Participation in the election process requires registering an account on the Jenkins community forums and at least one contribution made before September 1, 2023. When registering, you can use an existing GitHub account or create a new account specifically for Jenkins community forums. We ask all community members who are interested in voting and meet the requirements to join the election-voter-2023 group during the registration period. Previous elections utilized their own groups, so joining the election-voter-2023 group is required for participation.

We have made the decision to trust participants and not require that you provide evidence of contribution as part of your voter registration. We reserve the right to ban any account from the election process if we identify abuse. Once registration is over, a list of email addresses will be sent to the Condorcet Internet Voting Service (CIVS).

Election

On November 06, candidates will be announced, and all registered voters will be notified by email to participate using the Condorcet Voting System. Registration to vote closes on November 05. All votes must be submitted by December 1.

Results

As soon as the votes are counted, the results will be published. The new term starts on December 11, when the newly elected members will transition to their roles.

Important Dates

Important DatesTime

Nomination of candidates open

September 18

Voter registration opens

September 18

Nomination of candidates closes

October 27

Voter registration closes

November 05

Candidates announced & Voting starts

November 06

Voting closes

December 01

Announcing Results

December 11

Troubleshooting

If you are new to Jenkins elections, unfamiliar with the Condorcet Internet Voting Service (CIVS) or the community forums, or feel overwhelmed with the process, don’t worry, we’ve got you covered. We published a step-by-step guide, outlining how to to join the election-voter-2023, activate your CIVS account and cast your vote.

If you encounter issues still or need assistance, don’t hesitate to contact the election committee.

Thank you, as always, and don’t forget to register to vote by December 1!

Hacktoberfest 2023

Next: Renovate Probe
Previous: Jenkins Board and Officer Elections 2023 - Nominations Open
$
0
0

Hacktoberfest

Once again, Hacktoberfest is back! During this October event, everyone can celebrate and support open source by contributing changes.

This year’s edition is special because it is Hacktoberfest’s tenth anniversary Once again, the Jenkins community will participate in the event, and we invite you to contribute to Jenkins projects. We also encourage maintainers to welcome and help newcomers in the project.

The event is a good opportunity to cross the "imposter syndrome" threshold and contribute back to Open Source. A lot of helpful folks will be available to guide less experienced contributors in their first steps. It is also a great opportunity to acquire and exercise new skills, both hard and soft, while improving your experience with CI/CD in general and Jenkins in particular. On top of this, it is a great way to "build the Jenkins dev muscle", should one want to be part of the Jenkins Google Summer of Code 2024 adventure.

September is the perfect time to prepare for Hacktoberfest. Get a jump start by finding projects to contribute to, adding the "Hacktoberfest" tag to your projects, or familiarizing yourself with Git.

Due to the increasing popularity of Hacktoberfest, and to ensure its sustainability for another decade, the organization is moving away from a free t-shirt reward to a digital reward.


Contributors

This is what contributors need to know to participate and complete Hacktoberfest:

  • Register anytime between September 26 and October 31 on the Hacktoberfest site.

  • Pull requests can be made in any jenkinsci or jenkins-infra GitHub project that is participating in Hacktoberfest (look for the "hacktoberfest" topic).

  • Project maintainers must accept your pull requests for them to count toward your total.

  • The new digital reward kit will include a customizable badge that evolves with each pull request accepted by maintainers, representing the participant’s journey in open source and Hacktoberfest. Additionally, participants will receive unique badges featuring a delightful surprise and gifts from sponsors. Participants will also be able to proudly share their badge on the Holopin Hacktoberfest Badge Board of Fame.

  • In previous years, Hacktoberfest sponsors have given participants who completed 4 PRs the option to plant a tree through Tree Nation, instead of receiving a t-shirt. This year, a tree will be purchased for the first 50,000 participants that complete their first PR/MR.

Jenkins specific details can be found on the Jenkins Hacktoberfest page.

Some good resources for beginners can be found here:

A good introduction to Jenkins contribution is the series of Modernizing Jenkins plugins recorded livestreams.


Maintainers

Jenkins and plugin maintainers also need to get ready for Hacktoberfest by preparing your project for contributions by following these best practices:

  • Add the "hacktoberfest" topic to your repository to OPT-IN TO HACKTOBERFEST and indicate you’re looking for contributions.

  • Apply the "hacktoberfest" label to issues you want contributors to help with in your GitHub project.

  • Add a CONTRIBUTING.md file with contribution guidelines to your repository.

  • Choose issues that have a well-defined scope and are self-contained.

  • Be ready to review pull requests, accepting those that are valid by merging them, leaving an overall approving review, or by adding the "hacktoberfest-accepted" label.

  • Reject any spammy requests you receive by labeling them as "spam" and any other invalid contributions by closing them or labeling them as "invalid".

Renovate Probe

Next: Jenkins August 2023 Newsletter
Previous: Hacktoberfest 2023
$
0
0

Introduction

Bots are used to automate tasks in a plugin. The Plugin Health Scoring System tracks usage of two bots - Dependabot and Renovate.

Dependabot looks for dependencies that are outdated in a plugin. It can also raise a security alert if a security breach is found, like when the code depends on an insecure package.

Renovate provides automated dependency updates. It can also schedule PRs and customize behavior using configuration files.

Importance of the probe

This is a community requested probe to track the usage of Renovate. Bot tracking probes help in identifying which bot is used most by the plugins. In the future, the probe will aid in decision-making when the need arises to add or remove a bot.

The pull requests created by Renovate will be counted when scoring the probe.

Challenges

Dependabot and Renovate probes both had similar functionality: finding their respective configuration in GitHub workflow.

Implementing the solution with the correct class design was a major challenge while working on this probe.

Outcome and Conclusion

Having implemented a similar class in the Security Scan probe, this probe was easy to complete. The probe has been successfully merged and pushed into production. The probe will help identify usage of Renovate in plugins.

For more information or to find answers to any questions you might have, please visit the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.

Jenkins August 2023 Newsletter

Next: Incremental Build Detection Probe
Previous: Renovate Probe
$
0
0

Jenkins July Newsletter

Key Takeaways

  • Jenkins project reports growth of 79% in Jenkins Pipeline, used to propel software delivery.

Security Update

Contributed by: Wadeck Follonier

  • Andrea Chiera completed his 3 months internship within the Security team, auditing 100 plugins and finding 20+ vulnerabilities.

  • A Plugin security advisory was published on July 12

  • The Security team is progressing on the CSP compliance project for Jenkins Core, encountering interesting corner cases that have to be covered.

Governance Update

Contributed by: Mark Waite

Voter registration and candidate nomination for Jenkins elections will begin in September. The blog post contains more details.

The ten year old Prototype.js JavaScript library will be removed from Jenkins core beginning with the weekly release October 3, 2023. Special thanks to Basil Crow and Tim Jacomb for their work preparing Jenkins core and Jenkins plugins for a successful removal of that outdated library.

Infrastructure Update

Contributed by: Damien Duportal

  • JDK21 (Early Access Temurin Edition) and Maven 3.9.4 are generally available for developers on ci.jenkins.io.

  • JDK17 is the default for all Jenkins controllers and agents on the Jenkins public infrastructure.

  • Migration of the last two VMs to a new IPv6-enabled network removed a monthly cost of $1000 for unused cloud resources.

User Experience Update

Contributed by: Mark Waite

The Jenkins user experience continues to improve. Jenkins 2.414.1 include significant updates to many UI elements, including form and page modernization.

  • Dropdown links have been improved and are now part of a common framework

  • Log recorder administration has been updated to be consistent with other pages

  • Sign-in page has been modernized and simplified

  • Many delete dialogs have been standardized and prepared for future improvements

  • Builds widget has a better layout

Platform Modernization Update

Contributed by: Bruno Verachten

  • There has been an issue with Docker Images republishing unexpectedly, causing old tags to be rebuilt. This is now solved.

  • JDK21:

    • Work is ongoing to keep JDK21 updated to an early access version in Jenkins’ infrastructure.

    • Jenkins can run on JDK21 since version 2.419.

    • A proposal from Mark Waite regarding Java 11, Java 17, and Java 21 adoption roadmap in Jenkins is being considered.

  • Docker agents and controller:

    • Ssh-agent: breaking change in 5.10.0 with the replacement of bullseye in favor of bookworm.

    • Controller:

    • Windows:

      • Windows agent images are now using a Windows server image, which is a breaking change.

      • Windows 2022 images are being considered due to Windows 2019 reaching end-of-life.

      • Instructions for Java 17 are still to be updated in the Windows docs.

Documentation Update

Contributed by: Kevin Martens

Over the course of August, there were seven blog posts published by a combination of 12 different authors. This included updates from the Google Summer of Code participants, a retrospective on a Jenkins security internship, considering what the future holds for Jenkins, and notices regarding bandwidth reduction, Linux containers, and Java 17 use in the Jenkins documentation.

Incremental Build Detection Probe

Next: GSoC Building Jenkins.io with alternative tools
Previous: Jenkins August 2023 Newsletter
$
0
0

Introduction

The Incremental Build Detection Probe was developed as a part of Google Summer of Code 2023 for the plugin health scoring system project.

Incremental builds aid in continuous delivery of plugins, deploying only the modules affected by new commits [1]. Jenkins core and plugins can be deployed incrementally, rather than waiting for the developer to manually deploy the release [2].

This probe identifies the plugins that have configured incremental-builds.

Importance of the Probe

This probe encourages delivery automation and saves the maintainer from manual deployment.

Challenges

Understanding the configuration and implementation posed a major challenge in this probe. To me, it seemed that I needed to check the configuration for Maven Friendly CI versions.

My mentor pointed out the following and explained that the actual configuration goes deeper:

  • The setup is actually done in .mvn/maven.config and it is essential that I look for the configuration in .mvn/maven.config.

  • Some plugins configure it in extensions.xml in the .mvn section such as kubernetes-plugin.

  • The final step was to look for the configuration in the CD workflow. As the jep-229 (Continuous Delivery Probe) is already live, I did not need to look for this configuration.

Outcome and Learning

I learned more about GitHub Workflows and its strength. This probe is merged and ready to be live.

For more information or if you have any questions, please visit the official GSoC 2023 project Adding Probes to "Plugin Health Score" description page.

Viewing all 1088 articles
Browse latest View live