Quantcast
Channel: Jenkins Blog
Viewing all 1088 articles
Browse latest View live

See what's next for the Jenkins user interface - DevOps World 2022

$
0
0
This is a speaker blogpost for a DevOps World 2022 talk in Orlando, Florida
jenkins redesign preview

... at the 'Transformation of the Jenkins User Interface and Where it’s Going Next' talk at DevOps World 2022, Thursday, September 29 10:45 AM - 11:30 AM EDT.

We’ll be giving a talk on the history of the Jenkins user interface, where we stand today, and a look into the future of where we plan to take Jenkins.

Expect to hear about the lessons we’ve learned and the challenges we’re tackling as we update Jenkins for the modern web.

Topics include:

  • A brief history of the Jenkins UI

  • What’s wrong with the current UI

  • The work we’re doing currently

  • Live demo showcasing a futuristic Jenkins


If you want to get involved in the UI/UX discussions of Jenkins please join the User Experience SIG.

You can watch our monthly meetings on YouTube and you can view in-progress work on GitHub.


Moving Jenkins to a New Frontier of Performance with Java 11 - DevOps World 2022

$
0
0

It’s 2022, and the Jenkins project is approaching 18 years old, boasts over 1,800 plugins and over 1 million users around the world, and is widely considered the most popular automation server in use today. The latest hurdle we’ve cleared was delivering Java 11 support, partially motivated by Oracle’s ending of public support for Java 8 back in March of 2022.

Starting as a hack-a-thon project in 2018, the Jenkins community announced full support for Java 11 in March of 2019, and new versions of Jenkins since the June 2022 release now require Java 11.

So what does this mean exactly? Well, for starters, Java 8 was released in 2014 and had become outdated in terms of new features and security enhancements that were happening in many key third-party libraries consumed by the Jenkins project. The requirements to move to Java 9, Java 10, or Java 11 were becoming more and more painful from a development standpoint. This led to an unprecedented response from Oracle, stating they would extend fee-based support for JDK8 into 2030. However, the benefits of moving to newer versions of Java were clear, most notably surrounding performance.

Java memory management

It’s one of the core tenets of the programming language, removing the complexity of memory management from the developer’s responsibilities. At the core of the JVM (Java Virtual Machine) is the garbage collector. The garbage collector frees up memory when objects are no longer needed, allowing space for new objects to be created. While the G1GC garbage collector is our recommended GC for Jenkins, it was not the default for Java 8. Only in Java 9 and later did G1GC become default. Now with Java 11 there are many improvements to the G1GC collector which deliver significant gains in performance.

Join us at DevOps World in-person this year in Orlando, Florida, where Dylan Dewhurst and I will be presenting our learnings about the benefits of moving Jenkins from Java 8 to Java 11. We will provide data showing the performance improvements this shift has brought. We will showcase some experimental garbage collectors that we hope to see utilized in the next generation of JVM’s, and we will show you how to move your Jenkins instance from Java 8 to Java 11 successfully.

DW 2022

A near Feature-complete version of Jenkinsfile Runner Actions as GitHub Actions

$
0
0

Abstract

Jenkinsfile Runner Action for GitHub Actions aims at providing customized environments and useful GitHub Actions for the users. Therefore, users can execute the Jenkins pipeline in the GitHub Actions runners. It enables running Jenkins pipelines in a Function-as-a-Service context. Users can customize their environment by setting up base images, configuring the Jenkins Configuration-as-Code plugin, extending vanilla images or providing initial hooks for the temporary Jenkins controller. Several examples about how to set up secrets, Jenkins plugins and integrate with other GitHub Actions can be seen below. This project is based on the Jenkinsfile runner, which is a command line tool for the Jenkins pipeline execution engine.

Implementation

Jenkinsfile Runner Action for GitHub Actions help users fetch necessary dependencies, and wrap the entrypoint for Jenkinsfile Runner in one GitHub Action. Users can use them in the workflow definitions just like other GitHub Actions. Based on this big idea, I developed five GitHub Actions.

Runtime type actions

These actions run the pipeline in the GitHub Actions runners host machines directly.

  • jfr-setup-action - It’s used to set up the Jenkins environment, including but not limited to Jenkins core, plugin installation manager and Jenkinsfile Runner.

  • jfr-plugin-installation-action - Users can use it to install extra plugins. jfr-setup-setup needs to be run before.

  • jfr-runtime-action - It runs the Jenkins pipeline in the host machine. jfr-setup-setup needs to be run before.

Container type actions

These actions run the pipeline in the customized container.

  • jfr-container-action - It runs the Jenkins pipeline in the declared container. As the prebuilt container is declared in advance, it can integrate with other GitHub Actions theoretically. It’s based on the idea of Running jobs in a container.

  • jfr-static-image-action - It runs the Jenkins pipeline in the runtime built container. It doesn’t need to declare the container in advance so users cannot use other GitHub Actions except using actions/checkout to set up the workspace. But the users can override the base image by providing the docker image URL. For instance, if users want to combine the Jenkins environment with nodejs environment, they can use node:18.3.0 as input. It’s based on the idea of GitHub Docker container action.

Phase 1

When I started to write code based on my previous proposal, I found my implementation is different from the expectation of the maintainer and the mentors. What I wanted to do is make Jenkinsfile runner capable of generating necessary configurations for users. So they can use these configuration files to complete their Jenkinsfile runner actions. However, this method will couple the configuration generation step with Jenkinsfile runner which is a command line tool for Jenkins pipeline execution engine. The maintainer and the mentors proposed to make several useful GitHub Actions instead of coupling this unnecessary logic into Jenkinsfile runner. Therefore, I changed my project plan based on their advice and worked on developing the GitHub Actions for Jenkinsfile runner.

In this stage, I developed jfr-container-action and jfr-static-image-action, which run in the customized containers. Its main advantage is that most dependencies are prepared in advance in the container. I also made several useful options in Jenkinsfile runner available in our GitHub actions such as the initial hook option. To make users get started in our actions quickly, I refactored the documentation several times, and created several useful examples. It is worth noting that my examples aim at applying Jenkins pipeline in GitHub Actions runner which is a FaaS context.

Phase 2

In the second phase, I found current actions can only be supported by GitHub Actions Linux runner. Then I decided to develop the runtime type actions so they can run in all kinds of GitHub Actions runners. Its advantages are obvious. The runtime type actions are jfr-setup-action, jfr-plugin-installation-action and jfr-runtime-action. These actions will download all the dependencies at the runtime compared to container type actions' behavior. Its main disadvantage is the possibility of suffering from a download center outage.

What’s more, I kept developing examples for the actions and updated the documentation. As the project and documentation grow infinitely, I decided to split my project into several subprojects, and host the documentation including the user guide and developer guide on the GitHub Pages.

Deliverables

I write a central documentation page above the interface, basic usage, advanced usage, demos and developer guide for these Jenkinsfile-runner GitHub actions. If you feel confused about the definitions or examples, you can check the documentation page mentioned above.

How to use jfr-container-action

The user needs to declare the image usage of ghcr.io/jenkinsci/jenkinsfile-runner:master at the start of the job. Then the user can call this action by using jenkinsci/jfr-container-action@master. Because the runner declares the container environment at the start, other GitHub Actions can also run with jfr-container-action theoretically.

name: CIon: [push]jobs:
  jfr-container-action-pipeline:
    runs-on: ubuntu-latestcontainer:
      image: ghcr.io/jenkinsci/jenkinsfile-runner:mastersteps:
      - uses: actions/checkout@v2
      - name: Jenkins pipeline in the containeruses:
          jenkinsci/jfr-container-action@masterwith:
          command: runjenkinsfile: Jenkinsfilepluginstxt: plugins.txtjcasc: jcasc.yml

How to use jfr-static-image-action

jfr-static-image-action can also run the pipeline in the container. But the users don’t need to declare the container by themselves like jfr-container-action. Instead, jfr-static-image-action wraps the preparation and invocation of a customized container. Therefore, the users cannot use other GitHub Actions except using actions/checkout to set up the workspace. The users can call this action by using jenkinsci/jfr-static-image-action@master.

name: CIon: [push]jobs:
  jfr-static-image-action-pipeline:
    runs-on: ubuntu-lateststeps:
      - uses: actions/checkout@v2
      - name: Jenkins pipeline in the containeruses:
          jenkinsci/jfr-static-image-action@masterwith:
          command: runjenkinsfile: Jenkinsfilepluginstxt: plugins.txtjcasc: jcasc.yml

How to use jfr-runtime-action with other supporting runtime type actions

jfr-runtime-action will run the pipeline in the host machine directly. The users need to use jfr-setup-action in advance. If the users want to install extra plugins, they can use jfr-plugin-installation-action. The advantage of jfr-runtime-action is that it can run in all kinds of runners provided by GitHub Actions.

name: CIon: [push]jobs:
  jfr-runtime-action-pipeline:
    strategy:
      matrix:
        os: [ ubuntu-latest, macOS-latest, windows-latest ]runs-on: ${{matrix.os}}name: jfr-runtime-action-pipelinesteps:
      - uses: actions/checkout@v2
      - name : Setup Jenkinsuses:
          jenkinsci/jfr-runtime-action@master
      - name: Jenkins plugins downloaduses:
          jenkinsci/jfr-plugin-installation-action@masterwith:
          pluginstxt: plugins.txt
      - name: Run Jenkins pipelineuses:
          jenkinsci/jfr-runtime-action@masterwith:
          command: runjenkinsfile: Jenkinsfile

Reflection

The most valuable lesson I learned in GSoC is not about how to write code. It’s about the method of communicating with other people in the Jenkins community. When I was new in the Jenkins community, I kept making assumptions when I communicate with my mentors and project maintainers. Therefore, it’s easy for them to misunderstand my idea. The best way of communication in Jenkins open source community is over-explanation. Sometimes, people who lack background information cannot understand your ideas about a specific topic. Therefore, providing some extra explanations or descriptions is necessary.

The second lesson I learned is still about communication. When you feel confused about the words under a specific context, it’s better to ask the contributors or your mentors directly instead of spending plenty of time by yourself. The people in Jenkins open source community are always willing to give you the answers. Furthermore, if you have obstacles or milestones when developing the tools or plugins, it’s still necessary to ask the community developers first instead of making the decisions arbitrarily.

Jenkins Contributor Summit 2022 Agenda - Orlando, Florida

$
0
0
Jenkins DevOps

We wanted to share the agenda to the Contributor Summit in Orlando, Florida - as it stands at the moment.

Entry fee:

No cost to attend. Sign up via the sign up sheet (room capacity: 30). Event is in person only.

Location:

Orlando World Center Marriott: 8701 World Center Dr, Orlando, FL 32821 (Room: Magnolia 19)

Key dates:

  • Sept 27 - Contributor Summit @ 9AM - 5PM

  • Sept 28 & 29 - Main conference (use discount code: DW22JENKINSRTE)

Contact

Description

The Jenkins Contributor Summit brings together current and future contributors to the Jenkins project. At this event we will talk about the current state of the project and its future evolution.

The summit will include project updates by the Governance Board, working and special interest groups. We will also organize breakout sessions to discuss key initiatives in the project and its roadmap. Topics include but are not limited to: architecture changes, Jenkins Security, Cloud Native Jenkins, interoperability, documentation, and user experience. We will also have lightning talks and demos of the new and upcoming Jenkins features. We will also organize a track for newcomer contributors who want to participate in the project.

Agenda

  • Intro, summit overview - 3 min - Bruno Verachten

  • State of Jenkins - one of Jenkins Board members, 7 min - Mark Waite

  • Project/SIG Updates, 5-10 min each. SIG leaders, officers, et al. to summarize the current state

    • Release officer - Tim Jacomb

    • Security officer - Wadeck Follonier

    • Infrastructure officer - Damien Duportal

    • Platform SIG - Mark Waite

    • Events officer - Alyssa Tong

    • Documentation officer - Mark Waite

    • Advocacy and Outreach SIG - Mark Waite & Jean-Marc Meessen

      • Google Summer of Code

      • She Code Africa

  • User Experience SIG - Tim Jacomb & Jan Faracik

  • What’s next for Java - Mark Waite

    • Java 11

    • Java 17

  • Plugin health scoring project - Jake Leon

    • Inspiring plugin health improvements by scoring the current status

  • Ignite Talks / Demos

    • Darin Pope

    • Volunteers from audience

    • Rapidly build and test your iOS Apps with Jenkins on EC2 Mac - Sudhir Reddy [10 mim]

    • Migrating Jenkins Unity CI/CD build farms to AWS EC2 Mac instances - Sergey Kruson [10 min]

    • End user presentations/demos (5 min each]

    • Pipeline experiences - how they use Pipeline

    • Mobile development - Bruno Verachten

    • Administering large Jenkins instances - Damien Duportal

    • Pre-built Jenkins test configurations with Docker - Mark Waite

    • Choosing your tests wisely - Kohsuke Kawaguchi (tbc)

    • Security - Wadeck Follonier

  • Hacktoberfest and onboarding new contributors - Jean-Marc & Bruno Verachten

  • Future of Jenkins

    • UI - Jan Faracik & Tim Jacomb

    • Testing improvements

    • Internationalization and localization - Mark Waite and Alex Brandes

    • First time contributor tasks - Mark Waite & Darin Pope

    • More cloud

If you’re in the neighborhood please join us and meet the many Jenkins contributors and users. We look forward to seeing you there!

Thank you,

Alyssa Tong and Bruno Verachten

Create iOS Unity build pipelines on AWS with Jenkins and EC2 Mac instances

$
0
0
Jenkins DevOps

Create iOS Unity build pipelines on AWS with Jenkins and EC2 Mac instances

Every mobile game, whether it is a multiplayer or standalone game, must build binaries. Developers of high budget, high profile AAA and AA games tend to lean towards managing a build farm, whereas independent developers (indies) may rely on local or third-party solutions. Managing compute infrastructure is a time consuming and ongoing task for many companies and developers, especially in relation to iOS builds, which require applications to be signed before submission to the App Store. Given the overhead required to run and maintain a local build environment, it rarely makes sense to maintain physical build farms for many teams.

Amazon EC2 Mac instances launched at re:Invent 2020, introduced macOS as a supported instance type for the first time. Later in 2022, Apple M1-based Mac instances were released, enabling Apple silicon in the cloud alongside the x86 architecture. Today, EC2 Mac instances are being leveraged by thousands of customers today to build and run their macOS workloads on the AWS cloud. By using EC2 Mac instances in combination with EC2 Spot instances and Jenkins CI/CD, Unity developers were able to build scalable, cost efficient, and fast iOS pipelines for building, testing, signing, and publishing to the App Store.

In this blog post, we discover how to build a scalable and cost-efficient iOS build pipeline on AWS that can be deployed within hours.

Overview of solution

This solution is possible due to Unity`s ability to split a build process into two parts:

  • Xcode Project preparation: a first phase involving processing compute-intensive images, videos, music, and compiling additional assets.

  • Xcode Build, packaging, and signature: the second and final phase of building an iOS app which must be done using macOS, an EC2 Mac instance in our solution.

The high-level architecture looks like the following:

image

We will build upon these concepts and will use EC2 Spot instances for the compute-heavy phase of asset compilation in conjunction with the EC2 Autoscaling feature. We will also use an EC2 Mac instance to complete the second, final Xcode build step. For the whole pipeline, orchestration is done with Jenkins CI/CD. We will also use the ability of the Unity engine to be Dockerized to implement scalability and flexibility.

Services and dependencies used in our solution are shown on the following diagram:

image

Key pieces

In this blog I will not explain the solution step by step; instead, I will emphasize key pieces of the solution, which you can setup on your own. You can follow the whole procedure by going through the workshop here, a repository with code and a sample project can be found here.

Networking template

To setup the whole solution, I have created the following CloudFormation template. It prepares the environment by creating a VPC, subnets, a Jenkins Manager installation and an EC2 bastion host. To deploy the stack in us-east-2 (Ohio) region, click the button: image

The stack incorporates VPC with two availability zones (AZs), and with two private and two public subnets. To follow best practices, we will place Jenkins Manager and execution nodes within the private subnet. The stack also sets up a bastion host in order to access resources in the private subnet.

Internal resources will use NAT gateway to reach the internet in two availability zones to reduce inter-AZ traffic costs.

image

Unity container and build server

The demonstration repository includes a sample Unity project to build. The Unity engine can be containerized—the Linux-based solution uses Docker to pull a specific Unity version from a repository, set up a license, and build the Xcode project. Since the process is compute-heavy, the types of instance used in this case is a C5.4xlarge instance, which are compute-optimized instances.

Unity part that needs to be legally validated

In order to build a Unity project and create an Xcode project in a production environment, you will need a license. For that, Unity provides several options: in our example, we will use “Unity Build Server” licenses. (You can read more here.) Here are some AWS cloud-specific steps to implement in order to ensure the Build Server is setup in resilient way.

  1. Build server installations maintain a number of “seats” or licenses which are loaned to workers and released after the build process is done; however, the build server is bound to number of cores on the instance, as well as the network interface’s MAC address. This means that once you “bind” seats to an instance of Build Server and you need to launch it on a new instance, if the number of cores or network interface do not match, you will lose your seats. In order to avoid such a problem, you’ll need to provision an additional ENI, attach it to the instance, and use its MAC address to bind Build Server to that ENI’s address. Once that is done, you can launch new instances from AMIs with Build server, attach the ENI, and your licenses are secure and assigned.

  2. To ensure that the process is automated, set up an autoscaling group (ASG) with the Build Server AMI— a single instance—and set it to attach the specific ENI to any new instance each time it is launched.

  3. For a multi-AZ setup, you need two separate build servers with their own distinct seat groups.

  4. You can expose Build Server via AWS Service Endpoint by setting up a Network Load Balancer in front of the Build Server instance’s ASG and providing a link to the endpoint to your consumers.

  5. Unity editor within workers need to be configured in order to connect to Build server. This can be achieved by providing configuration file each time a Docker container is started via Jenkinsfile. Configuration might look like the following:

{
    "licensingServiceBaseUrl": "\{http(s)://\{server dns name}}",
    "enableEntitlementLicensing": true,
    "enableFloatingApi": true,
    "clientConnectTimeoutSec": 5,
    "clientHandshakeTimeoutSec": 10
}

and the file should be saved as '/usr/share/unity3d/config/services-config.json' of the container. The License server DNS name can be stored in Secrets Manager.

End of Unity Part

EC2 Mac and Secrets Manager

In order to build and sign iOS applications, we need an EC2 Mac instance. To launch an Amazon EC2 Mac instance, you must first allocate a dedicated host in Amazon EC2. A dedicated host is a physical server that is wholly allocated for your use. Please keep in mind that currently EC2 Mac dedicated hosts cannot be released earlier than 24 hours after being launched. There is no restriction on how often you can launch an EC2 Mac instance on a dedicated host, however. Depending on the architecture (mac1 for x86, mac2 for Apple silicon), you will need different dedicated hosts.

image

Once the instance is launched, you will need to enable VNC to connect to it via the graphical user interface (GUI). That process is described here.

image

EC2 Mac instances use most of the same tools that you have already been using with other EC2 instances. These instances live in the VPC, support IAM, support user data, and can boot from EBS volumes, so you can create golden AMIs with all the required software installed; for example, Xcode. EC2 Mac instances can be configured by Systems Manager, for example to install patches. It is also integrated with Cloudwatch for logs. Basically, treat it the same as any EC2 instance that you need to configure and then use as a part of the unattended CI/CD pipeline to run your builds.

In our case, the instance also needs to have Java and Xcode installed, as well as an IAM role to call to other services, such as Amazon Secrets Manager.

Build signing and AWS Secrets Manager

To sign the build for the App Store, we need to have a signing certificate and a provisioning profile. It is possible to also generate an application package that can later be signed by another certificate. This is common for companies that want to separate test environments’ certificates from their production environment.

The development certificate, its password, and the provisioning profile can be stored in AWS Secrets Manager, a great way to save your secrets and have secure access to resources.

To set up a temporary keychain with signature files, you can use either Fastlane or create the temporary keychain manually.

Linux worker instances

In order for Linux workers to be able to run build instructions from Jenkins, several features have to be enabled for Linux worker instances:

  • Docker engine and Java must be installed

  • The attached EBS storage has to be large enough to incorporate the docker images

  • The instance’s security group should allow communication from Jenkins Manager node port 22 (Manager node uses ssh to connect to the worker and setup agent)

Jenkins setup

Jenkins uses a manager node to orchestrate builds. Build jobs will run either on EC2 Mac or on EC2 Spot instances managed by the EC2 Fleet plugin. Jenkins manager stores configurations, provides a user interface, and orchestrates build jobs. It can also store build artifacts, however I find it’s better to use Amazon S3 as it provides virtually unlimited storage, and can easily be used from within Jenkins pipeline.

image

In order to orchestrate Linux build instances, Jenkins needs the following plugins:

image

The Docker and Docker pipeline plugins allow us to run docker pipeline steps. These are required to launch Unity containers and run the first part of the build within.

The EC2 Fleet plugin allows for simple integration with Autoscaling groups, significantly reducing the overhead of launching new instances, starting a Jenkins agent, and running a build. The EC2 Fleet plugin is decoupled from the Auto Scaling group, which allows for full control of the kinds of instances to be launched. This way, we can utilize the full power of Spot instances as temporary workers, thus significantly reducing costs.

In order to orchestrate Mac instances, the instance has to have port 22 enabled as well as Java installed. You can add the instance manually, or launch it via CloudFormation or Terraform and use the self-registration method described here.

Please note that currently dynamic provisioning of EC2 Mac instances via Auto Scaling groups is not possible due to the minimum 24 hour reservation time for which an EC2 Mac dedicated host has to be reserved. However, we can use several Jenkins executors on a single EC2 Mac instance:

image

Also, labels can be used in order to separate kinds of workers within a pipeline. Note that on a screenshot above I use label “mac”.

Build pipeline and running a build

Every Jenkins pipeline can be described using a Jenkinsfile file. It is a YAML-formatted document which describes all the steps for the pipeline. You can read more here. I already have such a file stored in my repository. The file contents are following:

Click to reveal the Jenkinsfile
pipeline {
    agent none

    environment {
        UNITY_PROJECT_DIR='UnityProjectSample'
        IMAGE='unityci/editor'
        UNITY_VERSION='2021.3.6f1-ios-1.0'// Build parameters
        UNITY_LICENSE_FILE='UNITY_LICENSE_FILE'
        PROVISIONING_PROFILE_NAME='UnityBuildSample-profile'// secret from Secrets Manager
        TEAM_ID_KEY='TEAM_ID'
        LICENSE_SERVER_ENDPOINT='LICENSE_SERVER_ENDPOINT'
        SIGNING_CERT='SIGNING_CERT'
        SIGNING_CERT_PRIV_KEY='SIGNING_CERT_PRIV_KEY'
        SIGNING_CERT_PRIV_KEY_PASSPHRASE='SIGNING_CERT_PRIV_KEY_PASSPHRASE'
        APPLE_WWDR_CERT='APPLE_WWDR_CERT'
        PROVISIONING_PROFILE='PROVISIONING_PROFILE'
    }

    stages {
        stage('build Unity project on spot') {
            agent {
                docker {
                    image 'unityci/editor:2021.3.6f1-ios-1.0'
                    args '-u root:root'
                }
            }
            steps {
                // install stuff for Unity, build xcode project, archive the result
                sh '''
                    printenv
                    echo "===Installing stuff for unity"
                    apt-get update
                    apt-get install -y curl unzip zip
                    curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o
                    "awscliv2.zip"
                    unzip -o awscliv2.zip
                    ./aws/install
                    apt-get install sudo
                    # Following section can be uncommented if Unity Build server is used
                    # just to push it through
                    # sudo mkdir -p /usr/share/unity3d/config/
                    # endpoint=`aws secretsmanager get-secret-value \
                    # --secret-id $LICENSE_SERVER_ENDPOINT --output text --query
                    # 'SecretString' | cut -d '"' -f4`
                    # configfile='\{
                    # "licensingServiceBaseUrl": "'$endpoint'",
                    # "enableEntitlementLicensing": true,
                    # "enableFloatingApi": true,
                    # "clientConnectTimeoutSec": 5,
                    # "clientHandshakeTimeoutSec": 10
                    # }'
                    # Copying Unity .ulf license file from S3 to container
                    # aws s3 cp "s3://$\{S3_BUCKET}/Unity_2021.3.6f1-ios-1.0.ulf"
                    # "/root/.local/share/unity3d/Unity/Unity_lic.ulf"
                    # mkdir -p "/root/.local/share/unity3d/Unity"
                    # aws secretsmanager get-secret-value --secret-id $UNITY_LICENSE_FILE
                    # --output text --query SecretBinary |
                    # base64 -d > "/root/.local/share/unity3d/Unity/Unity_lic.ulf"
                    # echo "===Building Xcode project"
                    # We also pull in additional repository with actual Unity Project.
                    # We have several configuration files for our build configuration
                    # You can find those in UnityProjectSample folder
                    rm nodulus -rf
                    git clone https://github.com/Hyperparticle/nodulus.git
                    cp -nR nodulus/* UnityProjectSample/
                    cd $UNITY_PROJECT_DIR
                    mkdir -p ./iOSProj
                    mkdir -p ./Build/iosBuild
                    xvfb-run --auto-servernum --server-args='-screen 0 640x480x24'\
                        /opt/unity/Editor/Unity \
                        -quit \
                        -batchmode \
                        -nographics \
                        -executeMethod ExportTool.ExportXcodeProject \
                        -buildTarget iOS \
                        -customBuildTarget iOS \
                        -customBuildName iosBuild \
                        -customBuildPath ./Build/iosBuild \
                        -logFile /dev/stdout
                    echo "===Zipping Xcode project"
                    zip -r iOSProj iOSProj
                    '''// pick up archive xcode project
                    dir("$\{env.UNITY_PROJECT_DIR}") {
                        stash includes: 'iOSProj.zip', name: 'xcode-project'
                    }
                }
                post {
                    always {
                        sh "chmod -R 777 ."
                    }
                }
            }
            stage('build and sign iOS app on mac')\{
                // we don't need the source code for this stage
                options {
                    skipDefaultCheckout()
                }
                agent {
                    label "mac"
                }
                environment {
                    HOME_FOLDER='/Users/jenkins'
                    PROJECT_FOLDER='iOSProj'
                }
                steps {
                    unstash 'xcode-project'
                    sh '''
                    pwd
                    ls -l
                    # Remove old project and unpack a new one
                    rm -rf $\{PROJECT_FOLDER}
                    unzip iOSProj.zip
                    '''// create export options file
                    writeFile file: "$\{env.PROJECT_FOLDER}/ExportOptions.plist", text: """<?xml version="1.0" encoding="utf-8"?>
                    <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN""http://www.apple.com/DTDs/PropertyList-1.0.dtd">
                    <plist version="1.0">
                        <dict>
                            <key>signingStyle</key>
                            <string>manual</string>
                        </dict>
                    </plist>
                """

                sh '''
                PATH=$PATH:/usr/local/bin
                cd $\{PROJECT_FOLDER}
                # Update project settings
                # sed -i ""'s|^#!/bin/sh|#!/bin/bash|' MapFileParser.sh
                # extra backslash for groovy
                TEAM_ID=`aws secretsmanager get-secret-value \
                    --secret-id $TEAM_ID_KEY --output text --query 'SecretString' | cut -d '"' -f4`
                # extra backslash for groovy
                sed -i """s/DEVELOPMENT_TEAM = \\"\\"/DEVELOPMENT_TEAM = $TEAM_ID/g" Unity-iPhone.xcodeproj/project.pbxproj
                #############################################
                # setup certificates in a temporary keychain
                #############################################

                echo "===Setting up a temporary keychain"
                pwd
                # Unique keychain ID
                MY_KEYCHAIN="temp.keychain.`uuidgen`"
                MY_KEYCHAIN_PASSWORD="secret"
                security create-keychain -p "$MY_KEYCHAIN_PASSWORD""$MY_KEYCHAIN"
                # Append the temporary keychain to the user search list
                # double backslash for groovy
                security list-keychains -d user -s "$MY_KEYCHAIN" $(security list-keychains -d user | sed s/\\"//g)
                # Output user keychain search list for debug
                security list-keychains -d user
                # Disable lock timeout (set to "no timeout")
                security set-keychain-settings "$MY_KEYCHAIN"
                # Unlock keychain
                security unlock-keychain -p "$MY_KEYCHAIN_PASSWORD""$MY_KEYCHAIN"
                echo "===Importing certs"
                # Import certs to a keychain; bash process substitution doesn't work with security for some reason
                aws secretsmanager get-secret-value --secret-id $SIGNING_CERT --output text --query SecretBinary | base64 -d -o /tmp/cert && security -v import /tmp/cert -k "$MY_KEYCHAIN" -T "/usr/bin/codesign"
                rm /tmp/cert
                PASSPHRASE=`aws secretsmanager get-secret-value \
                    --secret-id $SIGNING_CERT_PRIV_KEY_PASSPHRASE --output text --query 'SecretString' | cut -d '"' -f4`
                aws secretsmanager get-secret-value --secret-id $SIGNING_CERT_PRIV_KEY --output text --query SecretBinary |
base64 -d -o /tmp/priv.p12 &&
                security -v import /tmp/priv.p12 -k "$MY_KEYCHAIN" -P "$PASSPHRASE" -t priv -T "/usr/bin/codesign"
                rm /tmp/priv.p12; PASSPHRASE=''
                #aws secretsmanager get-secret-value --secret-id $APPLE_WWDR_CERT --output text --query SecretBinary | \
                # base64 -d -o /tmp/cert &&
                # security -v import /tmp/cert -k "$MY_KEYCHAIN"
                # rm /tmp/cert
                # Dump keychain for debug
                security dump-keychain "$MY_KEYCHAIN"
                # Set partition list (ACL) for a key
                security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k $MY_KEYCHAIN_PASSWORD $MY_KEYCHAIN
                # Get signing identity for xcodebuild command
                security find-identity -v -p codesigning $MY_KEYCHAIN
                # double backslash for groovy
                CODE_SIGN_IDENTITY=`security find-identity -v -p codesigning $MY_KEYCHAIN | awk '/ *1\\)/ \{print $2}'`
                echo code signing identity is $CODE_SIGN_IDENTITY
                security default-keychain -s $MY_KEYCHAIN
                #############################################
                # setup provisioning profile
                #############################################
                echo ===setting up a provisioning profile
                pwd
                # # if the provisioning profile already exists, don't overwrite
                # PROV_PROFILE_FILENAME="$\{HOME}/Library/MobileDevice/Provisioning Profiles/$\{PROVISIONING_PROFILE_NAME}.mobileprovision"
                # if [ ! -f "$PROV_PROFILE_FILENAME" ]; then
                # aws secretsmanager get-secret-value --secret-id $PROVISIONING_PROFILE --output text --query SecretBinary | \
                # base64 -d -o "$\{PROV_PROFILE_FILENAME}"
                # fi
                # # lock, since multiple jobs can use the same provisioning profile
                # if [ -f "$\{PROV_PROFILE_FILENAME}.lock" ]; then
                # n=`cat "$\{PROV_PROFILE_FILENAME}.lock"`
                # n=$((n+1))
                # else
                # n=1
                # fi
                # echo $n > "$\{PROV_PROFILE_FILENAME}.lock"
                #############################################
                # Build
                #############################################
                echo ===Building
                pwd
                # xcodebuild -scheme Unity-iPhone -sdk iphoneos -configuration AppStoreDistribution archive -archivePath "$PWD/build/Unity-iPhone.xcarchive" CODE_SIGN_STYLE="Manual" PROVISIONING_PROFILE_SPECIFIER_APP="$PROVISIONING_PROFILE_NAME" CODE_SIGN_IDENTITY=$CODE_SIGN_IDENTITY OTHER_CODE_SIGN_FLAGS="--keychain=$MY_KEYCHAIN" -UseModernBuildSystem=0
                xcodebuild -scheme Unity-iPhone -sdk iphoneos -configuration AppStoreDistribution archive -archivePath "$PWD/build/Unity-iPhone.xcarchive" CODE_SIGN_STYLE="Manual" CODE_SIGN_IDENTITY=$CODE_SIGN_IDENTITY OTHER_CODE_SIGN_FLAGS="--keychain=$MY_KEYCHAIN" -UseModernBuildSystem=0 CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO
                # Generate ipa
                echo ===Exporting ipa
                pwd
                # xcodebuild -exportArchive -archivePath "$PWD/build/Unity-iPhone.xcarchive" -exportOptionsPlist ExportOptions.plist -exportPath "$PWD/build"
                #############################################
                # Upload
                #############################################
                # Upload to S3
                # /usr/local/bin/aws s3 cp ./build/*.ipa s3://$\{S3_BUCKET}/
                #############################################
                # Cleanup
                #############################################
                # Delete keychain - should be moved to a post step, but this would require a global variable or smth
                security delete-keychain "$MY_KEYCHAIN"
                # Delete a provisioning profile if no jobs use it anymore
                n=0
                if [ -f "$\{PROV_PROFILE_FILENAME}.lock" ]; then
                n=`cat "$\{PROV_PROFILE_FILENAME}.lock"`
                n=$((n-1))
                echo $n > "$\{PROV_PROFILE_FILENAME}.lock"
                fi
                if [ "$n" -le "0" ]; then
                rm -f "$\{PROV_PROFILE_FILENAME}"
                rm -f "$\{PROV_PROFILE_FILENAME}.lock"
                fi
                '''
            }
            post {
                always {
                    sh '''
                    #############################################
                    # cleanup
                    #############################################
                    zip -r iOSProj/build/Unity-iPhone.zip iOSProj/build/Unity-iPhone.xcarchive
                    rm -rf iOSProj/build/Unity-iPhone.xcarchive
                    '''
                    archiveArtifacts artifacts: '**/Unity-iPhone.zip', onlyIfSuccessful: true, caseSensitive: false
                }
            }
        }
    }
    post {
        success {
            echo 'Success ^_^'
        }
        failure {
            echo 'Failed :('
        }
    }
}

The key pieces of the file:

  • agent – describes which instances the build should run on. In my case label “mac” is used to separate EC2 Mac instances from Spot instances

  • environment – describes environment variables used by the build

  • stage(‘Name’) – describes separate stage

  • docker – describes docker image that is pulled and is later used to run Unity build in

  • xvfb-run /opt/unity/Editor/Unity – runs unity editor in headless mode within a container

  • security create-keychain – creates private keychain to store secrets like signing certificate. The keychain is later deleted.

  • To pass Artifacts between stages, Jenkins` stash function is used

And once done, the basic pipeline should look like the following:

image

Final architecture

Once the all the key pieces mentioned in this post are assembled, the final picture is as following:

image

Also refer to the full diagram with additional details explained. For this diagram, we assume that code and Docker images are located within the AWS account as well, to reduce data transfer charges and improve latency.

Main solution benefits and costs

There are several factors that are important to consider when building this solution:

  1. Unity version control.
    This solution allows for flexible control of which Unity version is used for the build. By simply using tagging for Docker images, the pipeline can run different versions of Unity without changing the configuration of the pipeline in general.

  2. Flexible Xcode version control
    By utilizing AMI images for EC2 Mac, it is possible to build a library of iOS and preinstalled Xcode versions to quickly launch on EC2 Mac hosts. This process can be further automated by using tools like Packer or EC2 Image Builder to create AMIs for different versions of environments.

  3. Cost benefits when using Spot instances and using less Mac instances
    Since this solution implements a split-build approach, it allows us to take 30 to 70% of the computing time from EC2 Mac instances. This allows for better parallelization of builds and reduces time required by the EC2 Mac instance to process the code, resulting in much faster builds in general. Spot instances are also used instead of on-demand instances. Thus, depending on a build, the approach can reduce the cost by around 30-40%.

  4. It is also possible to setup “layers” of EC2 Mac instances by using several Macs for development and production builds separately, this allows for secure separation of environments.

  5. Automatization of the pipeline via versioned Jenkinsfiles and Amazon CloudFormation templates allows for consistent and controllable approach to build environments.

  6. Conclusion

    This post explains key pieces of the of cost-effective Unity build pipeline. It utilizes a mechanism of separation of the build to Linux instances and EC2 Mac instances. The compute-heavy part can be done on cost-efficient Spot instances, which reduces load to Mac instances and allows for more parallel builds at a time. We went through Unity and iOS build environments, key elements, licensing, workers and overall CI/CD process automation with Jenkins.

    This approach has already been adopted by our Game tech clients: https://aws.amazon.com/ec2/instance-types/mac/customers/ - Riot Games, Pokemon Company and others. The pipelines speed being improved up to 400% (Pokemon Company), Improving management time (Riot games) and reduced complexity (Jamcity).

    We will be speaking more on this topic at the Jenkins Contributor Summit, on September 27 at DevOps World in Orlando, Florida. Hope to see you there!

Hacktoberfest Preptember

$
0
0

Hacktoberfest

Once again, Hacktoberfest is back! During this October event, everyone can celebrate and support open-source by contributing changes and can earn limited edition swag.

September is the perfect time to prepare for Hacktoberfest. Get a jump start by finding projects to contribute to, adding "Hacktoberfest" tag to your projects, or familiarizing yourself with Git.

The Jenkins community will participate once again in the event. We invite you to contribute to Jenkins projects but also, as maintainers, to welcome and help newcomers.


Contributors

This is what contributors need to know to participate and complete Hacktoberfest:

  • Register anytime between September 26 and October 31

  • Pull requests can be made in any jenkinsci or jenkins-infra Github project that’s participating in Hacktoberfest (look for the "hacktoberfest" topic)

  • Project maintainers must accept your pull requests for them to count toward your total

  • Have 4 pull requests accepted between October 1 and October 31 to complete Hacktoberfest

  • The first 40,000 participants (maintainers and contributors) who complete Hacktoberfest can elect to receive one of two prizes: a tree planted in their name, or the Hacktoberfest 2022 t-shirt

Jenkins specific details can be found on the Jenkins Hacktoberfest page.

Some good resources for beginners can be found here:

A good introduction to Jenkins contribution is the series Modernizing Jenkins plugins recorded livestreams. Or even better, join us at our workshop at DevOps World on September 27, 2022 (use discount code: DW22JENKINSRTE).


Maintainers

Jenkins and plugin maintainers need also to get ready for Hacktoberfest by preparing your project for contributions by following these best practices:

  • Add the "hacktoberfest" topic to your repository to OPT-IN TO HACKTOBERFEST and indicate you’re looking for contributions

  • Apply the "hacktoberfest" label to issues you want contributors to help with in your GitHub project

  • Add a CONTRIBUTING.md file with contribution guidelines to your repository

  • Choose issues that have a well-defined scope and are self-contained

  • Adopt a code of conduct to create a greater sense of inclusion and community for contributors

  • Be ready to review pull requests, accepting those that are valid by merging them, leaving an overall approving review, or by adding the "hacktoberfest-accepted" label

  • Reject any spammy requests you receive by labeling them as "spam" and any other invalid contributions by closing them or labeling them as "invalid"

Digital Ocean and Jenkins partnership continues to grow!

$
0
0

DO horizontal

We are excited to announce that the relationship between DigitalOcean and Jenkins continues to grow! We want to thank DigitalOcean for providing such incredible support to the Jenkins project.

DigitalOcean has provided a substantial donation of $18,000 to the Jenkins project. This is in addition to their previous sponsorship described here. Combining these, DigitalOcean has granted over $20,000 to the Jenkins project. The funding aspect is clear, but there are several other benefits to working with DigitalOcean in this capacity. Jenkins has access to more resources than ever before, such as utilizing the DigitalOcean platform to continue development and share progress made. We can dedicate our time and resources more fully, allowing users to engage and make an impact.

This sponsorship also increases the impact that we can have as a community, when it comes to ongoing projects and events like Hacktoberfest, which DigitalOcean organizes and hosts. DigitalOcean is proving their commitment to open source, which matches our desire to improve Jenkins in any way and by any means.

If you are new to DigitalOcean and would like to learn more, use our promo code DOJENKINS100 to get $100 credit to use over your first 60 days (only valid for new accounts).

Improve a plugin tutorial

$
0
0

Many new contributors may hesitate to contribute to a Jenkins plugin. They may be concerned that the time commitment is too great. They may be worried that they lack the technical skills to maintain a plugin. They may not feel adequate to handle issues related to a Jenkins plugin.

This blog post introduces the "Improve a plugin" developer tutorial for new contributors. The tutorial is a result of the "Contributing to Open Source" workshop (document and slides) that was held at DevOps World 2021.

It includes links to segments of the five part video series, Modernizing Jenkins plugins. Special thanks to Darin Pope for hosting the video series.

Small steps

A series of small steps can help new contributors contribute to Jenkins development, improve a plugin, and develop the confidence to adopt a Jenkins plugin. Each of the steps is valuable even if the new contributor decides that they do not want to adopt the plugin. Each of the steps will assist the maintainers of the plugin.

The steps include:


Scaling out iOS builds on AWS with EC2 mac

$
0
0
Jenkins DevOps

Scaling out iOS builds on AWS with EC2 mac

Introduction

The number of mobile application subscriptions are increasing annually, trending towards 8 billion and combined number of apps in the app stores are close to 6 million. Mobile application development has become prevalent, not just for consumer facing businesses but also with the remote workforce. Many large enterprises also support an internal app store catering to organization specific functionalities. As a result of this growth combined with the cloud first approach, there is a need to deploy mobile application development infrastructure with scale in mind from day 1.

Let’s start by looking at a skeleton view of typical iOS continuous integration and continuous delivery (CI/CD) pipelines. For the purpose of this blog, we can split it into two distinct phases, (1) Local Integrated Development Environment (IDE) based design and development of the iOS app which typically takes place on a macOS device, and (2) Building, testing, and publishing the app to the Appstore. This can occur on the same macOS device if it is with a a single developer project, however team based development, typically runs on separate build and test infrastructure. Although some aspects of this discussion can be applied to stand up a fleet of developer machines for phase 1, the focus will be placed on the topic at hand which is scalable build pipelines for the second phase.

image

Figure-1: iOS Application development steps

Amazon EC2 Mac instances allow customers to bootstrap macOS environments in the cloud and use these for building, testing, signing, and publishing iOS applications with the security, scalability, and flexibility of the cloud. EC2 Mac instances are offered as bare-metal instances running on top of single-tenant, Dedicated Hosts in compliance with macOS licensing.

Integration with services like AWS Auto Scaling and AWS Systems Manager with the SSM agent included in the Amazon Machine Images (AMIs), means that the fundamental building blocks to facilitate automation needed for setting up a CI/CD pipeline for iOS app builds is readily available. The AWS Partner Network (APN) is a global community of partners that leverages programs, expertise, and resources to build, market, and sell customer offerings. AWS Marketplace is built to accelerate innovation available via the cloud while balancing the needs for speed and agility with governance and control. Both software delivery automation products like CI from CloudBees and virtualization solution like Anka from Veertu are offered on AWS Marketplace. These products work with the fundamental building blocks to further remove the undifferentiated heavy lifting of scaling out the iOS app build workloads. Let’s look at the options to scale out build pipelines with EC2 Mac instances.

Queuing builds using SQS and AWS Systems Manager

EC2 Mac instances based AMIs have a Systems Manager agent included. This enables EC2 Mac instances to become a managed node under the Systems Manager inventory. We can then use “Run Command”, a capability of AWS Systems Manager, to remotely and securely run automation scripts on EC2 Mac instances. In this case, we will use Run Command to execute our build scripts. Configuring the event source trigger for Run Command could be achieved by a simple well-known pattern utilizing CodeCommit, SNS topic, SQS queue and Lambda worker as depicted below.

image

Figure-2: Multiple Build Jobs invoked by SSM Agent on EC2 Mac.

This pattern can be extended to a fleet of EC2 Macs instances all while relying on Systems Manager capabilities to retrieve the state of the fleet and invoke operations on individual nodes. Complex scenarios with multiple build pipelines would necessitate the need for a more feature rich build orchestrator than the rudimentary event-based sourcing pattern shown. For that, let’s look at what we can achieve with a popular automation server called Jenkins.

Orchestrating multiple builds with Jenkins

Jenkins has a powerful extension and plugin system that allows developers to write plugins affecting nearly every aspect of Jenkins' behavior. Jenkins can be run on AWS in several different ways, directly on EC2, using ECS Fargate as serverless or even EKS. To focus more on the iOS builds let’s assume a simple single node Jenkins setup running on EC2 Linux like shown in Figure-3. Using plugins like Configuration as code, SSH build agents you can define several parameters that simplifies deploying Jenkins to AWS, more on this here

image

Figure-3: Multiple Build Jobs invoked by Jenkins Agent on EC2 Mac instances.

Jenkins has the notion of a node, which in this case is an EC2 Mac instance, and executors, think of them as multiple processes within the node. If you configure the number of executors on the node as 2, you can in theory run two Xcode build jobs on a single EC2 Mac instance. Jenkins will handle the job orchestration as well as queuing up multiple jobs for each executor on the node. You can have several iOS build pipelines consuming the same build infrastructure. However, once the amount of time the build jobs waiting in the queue for an executor to be available increases, you may want to scale out into another EC2 Mac instance.

Auto-scaling with EC2 Mac instances

AWS Auto Scaling lets you build scaling plans that automate how groups of different resources respond to changes in demand. One of the features includes manual scaling operations typically used when you want an external event to make the decision of scaling up or down the number of instances in the group. In the case of scaling out Jenkins iOS builds, the queue depth can be one of those variables. EC2 fleet plugin has several options, including scaling up based on jobs waiting in the queue and scaling down when nodes are idle.

Adding a new launched EC2 Mac instance from the auto scaling group to the Jenkins controller is achieved by either SSH-based or JNLP-based registration. More on those here. Note however that EC2 Mac dedicated hosts require a 24-hour minimum allocation period, adhering to the macOS Software Licensing Agreement (SLA), before they can be released.

image

Figure-4: Multiple EC2 Mac instance executor nodes under an Auto Scaling group.

Type-2 virtualization with EC2 Mac instances

Apple Silicon and macOS has virtualization and a hypervisor framework built into it that lets you create guest virtual machines (VM) on top of the host. EC2 Mac instances are bare metal EC2 instances and will let you use this virtualization features to run up to 2 guest VM’s adhering to the macOS SLA’s. Anka build is one option that lets you leverage this granularity along with their CI/CD plugin for Jenkins to orchestrate multiple build jobs across a fleet of EC2 Mac instances. Another option is Tart that integrates into Cirrus labs CI to accomplish similar goals. Here is a quick depiction on how this would work from a Jenkins perspective, removing the well documented additional components included in the individual products to make this happen.

image

Figure-4: Type-2 virtualization enabling two guest virtual machines on EC2 Mac.

An example of how to combine the benefits of these several layers of scale out from AWS Autoscaling and macOS type-2 virtualization into one cohesive build fleet is illustrated in the diagram below.

image

Figure-5: Combining all scale-out options together.

Conclusion:

In this blog post we have walked through several options available to scale out iOS builds using Amazon EC2 Mac instances. We have also looked at the integration options available with a popular automation tool, Jenkins. Several of the options discussed here are implemented as solutions published by AWS with links available in the reference section for further reading.

Plugin Health Scoring System

$
0
0
GSoC

The goal of this blog is to showcase the work done during the Google Summer of Code 2022 coding phases. For a detailed description of the project, please see the project page.


Overview

About Project

Plugin maintenance is an extremely important phase of a plugin’s lifecycle. Jenkins is constantly developing, which raises the need for thousands of plugins to catch up with the latest developments. This allows them to stay useful for a broader set of users.

We aim to fulfill this need, by introducing a metric system to calculate the health score of Jenkins plugins. This health score measures the plugin’s development maturity level with the help of accurate numeric figures. This score attempts to provide an accurate picture of how much care and help a plugin needs, and makes it easier for contributors to contribute. This score also allows users to make a conscious decision before installing or using a plugin.

There were two coding phases in the GSoC 2022:


Phase 1

In this phase, we worked on setting up the project and followed best practices while doing so. The whole application is dockerized so that it is easier for anyone to spin up this project and test it out locally. There’s continuous integration (CI) in place for the project to make sure that new changes are tested automatically. The useful details from the Jenkins Update Center are stored inside the database to run analysis without constantly polling the Update Center.

Deliverables

  • Create a spring boot project connected with a PostgreSQL DB (PR)

  • Dockerize the whole application (PR)

  • Setup continuous integration for the project (PR)

  • Store plugin details from the update center, inside the DB in JSON format (PR)

Resources


Phase 2

In this phase, the probe engine for the probes was designed. Processes like reading the update center and executing the probe engine are now running on a schedule, without the need to restart the application. Probes created at this time are getting run by the probe engine, and are listed on the UI to highlight the current state of the project. Thanks to the helm chart that has been created to allow an easy deployment on Kubernetes, the site plugin-health.jenkins.io is live.

Deliverables

  • Design the Probe engine for the probes (PR)

  • Schedule the reading of the update center and probe engine execution (PR#1 and PR#2)

  • List the available probes of the application (PR)

  • Add a probe (PR)

  • Create a Helm chart to deploy the application on Kubernetes (PR)

  • Visit plugin-health.jenkins.io to view a list of active probes. A big thanks to the Jenkins Infrastructure team (especially Hervé Le Meur and Damien Duportal) for their help and support throughout.

Resources


Next Steps

  • Add more probes to the project.

  • Generate the plugin health scores based on the data extracted by the probes.

  • Deploy the health scores via a JSON file, similar to how Jenkins Update Center does it.

  • Render the detailed report of the health score of each plugin by fetching the JSON data generated above.

  • [Stretch Goal] Display Plugin health score on Plugin Manager.


Acknowledgments

  • I’m extremely grateful to have been given this opportunity to contribute to the Jenkins project. I owe it to my mentors for being able to help take this project forward and learn lots of things along the way. Shoutout to Adrien Lecharpentier and Jake Leon. They invested tremendous time and energy in mentoring me and driving this project forward. They synced with me weekly and made sure that I was learning and that we were taking this project forward, 1 PR at a time.

  • I asked all kinds of questions about this project, and Jake has been kind enough to answer all of them and help me understand this project and its future with his expertise. And a BIG shoutout to him for devoting his time to helping me prepare for my presentations by offering many tips on making effective slides and speaking well. His coaching helped me put across my points more powerfully, and made it all so easy.

  • I also want to thank Adrien for being one of the best mentors I’ve ever had. The amount of time he spent moving this project forward and sharing his expertise with me is unparalleled. And for that, I’m deeply grateful to be mentored by him. It’s an absolute privilege to get this opportunity to learn from him.

  • Also, thanks to the org admins, Jean-Marc Meessen, Alyssa Tong, and Kris Stern, for always keeping me and other contributors on our toes and assisting us with any blockers and concerns by organizing weekly stand-up calls.


Pipeline Steps Documentation Generator Improvements

$
0
0

About

Pipeline Steps Documentation Generator Improvements is a project under Google Summer of Code 2022, and this project aims to improve the steps documentation generated for Pipeline jobs, which is used by thousands of Pipeline developers worldwide.

Some initial parts of the project involved changes to the UI of the website. I spent most of the community bonding period thinking about possible improvements, understanding the code base (for jenkins.io and pipeline-steps-doc-generator), and creating website wireframes that can be shared with the community. This phase helped me communicate with my mentors and plan my project out, at least to a level where I could distinguish tasks for the two coding phases. I created an epic to generate and maintain tickets. Below is a summary of the two coding phases.

Coding Phase - I

  • Updating sidebar scrolling for Jenkins documentation (ticket) - The sidebar required independent scrolling from the main content to improve the navigation on longer pages.

  • Listing plugins on Pipeline Steps Reference (ticket) - I had included this idea in my proposal, but seeing the outcome, we decided that this does not add much value to the steps documentation for the users. This gave rise to the idea of adding a search filter instead. See the discussion in this PR.

  • Adding search filter (ticket) - There are more than 1500 steps listed on the Pipeline Steps Reference page, and filtering them becomes a much required feature for the users. With this change, users can now type in any string, and the content of the page is changed dynamically according to the input.

  • Separating declarative steps generation from main class (ticket) - A small but helpful change was to separate all the functions that generate declarative Pipeline steps from the main (PipelineStepExtractor) class. This would not only make the main class more readable but also isolate the specific functions related to declarative steps, making the code more modular.

  • Shifting parameter data types (ticket) - This was an initiative to reduce the overall length of each plugin page listed on Pipeline Steps Reference. The idea was to move the parameter data type (an essential piece of information regarding a step’s parameter that a user might want to know) from the bottom of the help text to inline with the name of the parameter/property.

  • Releasing pipeline-metadata-utils (ticket) - pipeline-metadata-utils is a tool that can be imported onto a project as a maven dependency. It provides a HyperLocalPluginManager that can be used to query plugins and build documentation with its help. My work involved updating and refining the code base and writing tests for the same. The released artifact is available in the Jenkins artifact repository.

  • Midterm Meetup

    • Slides

    • Presentation Recording -

Coding Phase - II

  • Label deprecated plugins (ticket) - Earlier, the only way to determine if a plugin was deprecated was to visit the plugin site. This can be remediated by adding a label saying so on Pipeline Steps Reference itself. In this way, users can also find all deprecated plugins that provide Pipeline steps by simply searching "(deprecated)" in the search filter provided.

  • Separate configured parameter classes (ticket) -

    • Many plugin pages included enormous amounts of documentation on a single page.

    • Currently, there are 625 plugins listed on Pipeline Steps Reference. The total size of the Asciidocs that hold their respective documentation is around 16.7 MB. Out of these 625 plugins, 8 of them contribute to 85% of the total size of the Asciidocs.

    • There were many issues associated with this -

      • Slow loading - some pages, such as workflow-multibranch took more than 15 to 20 seconds to load. This was due to the laggy action on javascript in collapsing the lists on the 4 MB large page.

      • Difficult navigation - once the user expanded a parameter containing deeply nested documentation, it became tough to keep track of the hierarchy levels within the documentation (it’s not easy to eyeball indentation :P). The only possible way to get back to a legible starting point was to refresh the page, which was not a quick thing to do, as mentioned above.

      • Redundant content - upon some investigation, it was found that there are several occurrences of redundant documentation. Redundant content is never good for any piece of documentation, and hence, this problem needed to be solved.

    • This problem is solved with the help of configuration-based post-processing of the generated AsciiDocs. A file would contain a list of parameter classes that are then searched in all AsciiDocs and localized in a single document, one by one. This new document is then referenced by all the depending documents with the help of a hyperlink. The tasks done under this ticket were -

      • Create Java class ProcessAsciiDoc.

      • Write tests and Javadoc comments for it.

      • Add instructions to use the configuration file in the README of pipeline-steps-doc-generator.

      • Add around 36 parameter classes to the configuration as a starting point.

    • The results obtained after running the processing layer with the initial configuration file were pretty encouraging. More than 2 MBs of redundancy was removed, and the new size of the AsciiDocs is around 14.4 MBs.

    • The larger pages saw a significant drop in size -

      Name

      Previous Size (in KBs)

      New Size (in KBs)

      workflow-multibranch

      4967

      688

      pipeline-groovy-lib

      2267

      564

      workflow-scm-step

      373

      96

    • Final Meetup

      • Slides

      • Presentation Recording -

Future Scope

  • Identify the plugin that a particular parameter class belongs to. This can be done by manipulating the getPluginNameFromDescriptor method supplied by pipeline-metadata-utils` such that it takes the class name and returns the plugin name corresponding to that.

  • Reduce the manual work required to configure the parameters and make the processing layer more robust towards inconsistencies.

  • Improve the time complexity associated with running the processing layer.

  • Possible future GSoC goal - Integrate the snippet generator with jenkins.io.

Acknowledgements and Insights

I am grateful to my mentor, Kristin, and the community at docs-sig. Their support was essential in making this project successful. I got consistent ideas and feedback from them throughout the project’s tenure. Here are some tips for new contributors who wish to participate in GSoC at Jenkins.

  • Make sure you ask your queries in the right channel. This will maximize the chances of an accurate and fast reply.

  • Don’t rely on others to solve every error you get. Try to figure it out yourself, and after an honest attempt, mention your query on the channel and all that you have tried.

  • Attend office hours regularly as soon as they begin for the next edition of GSoC. They are a great way to communicate with the mentors and understand the project idea.

  • Draft your proposal as soon as possible and gather feedback to maximize your chance of getting accepted. Make sure you add value to the original idea and include some implementation details in the proposal.

Project-specific guidance

  • After the separation of pipeline-metadata-utils, the code has become more abstract and relatively straightforward to dive into for newer contributors. You need not understand everything to start making changes to the code.

  • PipelineStepExtractor is the main class responsible for initializing the reactor in which the mock Jenkins instance is set up. It then uses the HyperLocalPluginManager to query the plugins and return all the information as a Java map.

  • ToAsciiDoc is responsible for formatting the Java map as an AsciiDoc and contains several functions to handle the different sections in a plugin page. Hence, if your goal is to change the presentation of the documentation while keeping the content static, you will probably need to make changes in this class only.

  • ProcessAsciiDoc is a string algorithm-based class responsible for matching the configuration keywords to their occurrences in the produced AsciiDocs. It currently follows a brute-force approach and is not very immune to complex configurations. Hence, there is a lot of scope for improvement in this class. If you want to improve something, feel free to tag my GitHub handle (@vihaanthora) in the issue/pull request you create.

  • The other classes will not require change unless a particular requirement arises.

  • Try to find bugs in the generated documentation by browsing through random AsciiDocs under Pipeline Steps Reference and create an issue on the project’s GitHub repository. If you want to seek clarification about some anomaly, you can write a brief description about it on the docs-sig gitter channel, and we’ll try to respond whenever possible.

You can find all the important links on the project page.

Jenkins September 2022 Newsletter

$
0
0

September 2022

image

Welcome to the Jenkins Newsletter! This is a compilation of progress within the project, highlighted by Jenkins Special Interest Groups (SIGs) for the month of September.

Happy reading!

Security Update

  • Two security advisories during September with "special" nature for Jenkins core

    • One was only impacting very specifically configured instance

    • The other was only impacting three successive weekly releases

Infrastructure Update

  • Dropping JDK8 from all agents and controllers processes with the 2.361.1 September LTS

  • Joint-work with JFrog (our sponsor) to decrease the bandwidth usage (50 Tb per month, should be under 10 Tb per month) of the Jenkins' Maven repository repo.jenkins-ci.org

  • Digital Ocean renewed their sponsorship with increased credits ($18,000 for 1 year)

  • Observability of ci.jenkins.io for plugin developers using Datadog’s (also a sponsor) metric collection on ephemeral agents. We hope to also use traces in the future

  • New "All-in-one" agent template built with Packer: whether your builds are running on a VM or a container, in AWS, Azure or Digital Ocean, you have the same tools in the same locations

Platform Update

  • Loaner computers for ppc64le are gone

    • IBM suspended the loaner machine program

  • arm/v7 agent Docker images

  • Linux packages use systemd instead of System V init

  • Exit lifecycle change in the Docker images

  • Blue Ocean container image deprecation (jenkinsci:blueocean)

  • Container repository management for Jenkins agents

    • Simplifying the release process

  • JDK 11 now required since Jenkins 2.357 and 2.361.1 LTS

    • Ongoing minor tasks

      • Dropping Java 8 from the Docker agents - merged, not yet released

      • Plugins beginning to require Java 11, ongoing process

    • Jenkins works with Java 17 since 2.355 and 2.346.1 LTS

    • Ongoing, a few recent reports, working well

    • Jenkins infra planning to put one or more of the infra machines on JDK 17

    • Timeline indicates end of Java 11 first level maintenance Sep 2023, security fixes for several more years after end of first level maintenance

    • More than 11,000 Java 17 installs reporting worldwide with Jenkins in September

Documentation Update

DigitalOcean and Jenkins partnership continues to grow!

Thanks to DigitalOcean’s generosity, Jenkins has access to more resources than ever before, such as utilizing the DigitalOcean platform to continue development and share progress made. We can dedicate our time and resources more fully, allowing users to engage and make an impact. Read the story on our blog.

image

Java 11 or Newer

Beginning with Jenkins 2.357 (released on June 28, 2022) and the forthcoming 2.361.1 LTS release, Jenkins requires Java 11 or newer. Additionally, beginning with Jenkins 2.355 (released on June 14, 2022) and Jenkins 2.346.1 LTS (released on June 22, 2022), Jenkins supports Java 17. Plugins have already been prepared in JENKINS-68446. Use the Plugin Manager to upgrade all plugins before and after upgrading to Jenkins 2.357. Read the blog post about Java 11 requirement for more information.

image

Thinking About Contributing to Open Source?

Ponder no more! We’ve created the "Improve a plugin" developer tutorial for new contributors. We’d love to work with you!

About Blue Ocean

Blue Ocean status

Blue Ocean will not receive further functionality updates. Blue Ocean will continue to provide easy-to-use Pipeline visualization, but it will not be enhanced further. It will only receive selective updates for significant security issues or functional defects.

Alternative options for Pipeline visualization, such as the Pipeline: Stage View and Pipeline Graph View plugins, are available and offer some of the same functionality. While not complete replacements for Blue Ocean, contributions are encouraged from the community for continued development of these plugins.

The Pipeline syntax snippet generator assists users as they define Pipeline steps with their arguments. It is the preferred tool for Jenkins Pipeline creation, as it provides online help for the Pipeline steps available in your Jenkins controller. It uses the plugins installed on your Jenkins controller to generate the Pipeline syntax. Refer to the Pipeline steps reference page for information on all available Pipeline steps.

Advocacy & Outreach Update

Jenkins & She Code Africa Contributhon

This program aimed to create a more diverse, inclusive, and innovative culture within the African open source ecosystem, by matching African women in technology with sponsor and mentor open source organizations. The 6 mentees joined the Jenkins project came from Nigeria, Kenya, and Ghana. They brought 3 different projects to the Jenkins community:

Many thanks to the dedicated mentors from the Jenkins project: Angélique Jard, Kevin Martens, Kristin Whetstone, and Mark Waite.

See the full blog post Expanding Open Source in Africa.

image

SCaLE19X

It was great to have returned to in person events! Thank you SCaLE for hosting us. See you again in March 2023!

image

Google Summer of Code Midterm Status Update

2022 GSoC Contributors worked hard to improve the following projects. In a midterm webinar, they presented their progress, lessons learned and achievements:

Hacktoberfest

jmMeessen announced …

September was the perfect time to prepare for Hacktoberfest. We got a jump start by finding projects to contribute to, adding "Hacktoberfest" tag to projects, or getting familiarized with Git. To get the deets see the blog post about Preptember.

image

Jenkins Board and Officer Elections 2022

$
0
0

We are excited to announce the 2022 Jenkins Governance Board elections! The nomination and voter registration period is now open. Nominations can be submitted for two governance board positions, and all five officer positions (Security, Events, Release, Infrastructure, and Documentation).

Many thanks to Gavin Mogan and Ewelina Wilkosz for serving on the Jenkins Governance Board. We also want to thank Damien Duportal as Infrastructure Lead, Wadeck Follonier as Security Officer, Alyssa Tong as Events Officer, Tim Jacomb as Release Officer, and Mark Waite as Documentation Officer for all of their work over the last term.

These roles are crucial to the expanding community and continued progress of Jenkins. This is an opportunity for Jenkins contributors and community members to sign up to vote in elections, and acknowledge other community members by nominating them for a position.

The election consists of 4 phases:

  • Nomination of candidates (October 20 - November 10)

  • Voter registration (October 20 - November 17)

  • Voting (November 17 - December 2)

  • Results announcement (December 7)

Voting

Participation in the election process requires registering an account with community.jenkins.io and at least one contribution made before September 1, 2022. When registering, you can use an existing Github account or create a new account specifically for the Jenkins community. We ask all community members who are interested in voting, and meet the requirements, to join the election-voter-2022 group on community.jenkins.io during the registration period.

We have made the decision to trust participants and not require that you provide evidence of contribution as part of your voter registration. We reserve the right to ban any account from the election process if we identify abuse. Once registration is over, a list of email addresses will be sent to the Condorcet Internet Voting Service.

Contributing

As shown on the Participate and Contribute page, there are many different ways to contribute to Jenkins. You can contribute by:

Since contributions can vary, there is no way to measure contributions accurately. All contributions to Jenkins and its community are welcome!

Nominations

During the registration period, we invite community members to nominate candidates by sending a message to the election-committee group. In your message, please include the name of the nominee, the specific position and your reasons for nominating that person. The nomination period ends on November 10. Nominees will be notified, and asked to confirm that they are interested in running as a candidate. The list of candidates will be announced on November 17.

Anyone can nominate candidates.

For the 2022 election, nominations are open for the following positions:

Additional information about the roles can be found on the team leads page.

Election

On November 17, candidates will be announced, and all registered voters will be notified by email to participate using the Condorcet Voting System. Registration to vote closes as soon as candidates are announced on November 17. All votes must be submitted by December 2.

Results

As soon as the votes are counted, the results will be published. The new term starts on December 3, when the newly elected members will transition to their roles.

Key dates for the 2022 elections

  • October 20: Voter registration and nominations open

  • November 10: Nomination deadline

  • November 17: Candidates announced, voter registration closes, voting period begins

  • December 02: Voting period ends

  • December 07: Results announced

Key information

Thank you, as always, and don’t forget to register to vote by November 17!

Jenkins In Google Summer of Code 2022

$
0
0

The Jenkins project participated in Google Summer of Code 2022. Congratulations to all participants and we thank all participants for their contributions. Now that the event is over, this page contains an archive of the 2022 specific information.

Projects

All four projects have been successfully completed in 2022. You can find the project information on the project pages:

These were proposed and selected from the list of GSoC 2022 project ideas.

Demos and results

Recordings

Midterm project demos
Final project demos

Org admins

GSoC Organization admins in 2022: Jean-Marc Meessen, Kris Stern, Alyssa Tong

Other materials

Jenkins October 2022 Newsletter

$
0
0

October 2022

Jenkins September Newsletter

Welcome to the Jenkins Newsletter! This is a compilation of progress within the project, highlighted by Jenkins Special Interest Groups (SIGs) for the month of October.

Happy reading!

Security Update

  • One security advisory during October about plugins

    • 28 impacted plugins

    • 8 without fixes, as explained in our process

    • 4 plugins were suspended from distribution

    • Due to the corrections applied for SECURITY-2824 in the Groovy sandbox, we strongly recommend you to update as soon as possible

  • We got several questions / reports related to Text4Shell (CVE-2022-42889)

    • The Security team did an investigation and nothing in the Jenkins ecosystem is affected

    • Due to the communication from Apache about why it’s not a vulnerability like Log4Shell, we do not plan to publish a blog post about our analysis.

Infrastructure Update

  • Hacktoberfest support: ~15% more builds for a cost decrease of $2.000 emoji:clap[]

  • Fall cleanup of accounts.jenkins.io and wiki.jenkins.io (Unified designs, dependencies updates, bug fixes)

  • Upgrade of our controllers to the new LTS 2.361.2

  • Upgrade of Kubernetes to 1.23.x for the 6 clusters (Amazon, Digital Ocean and Azure)

  • Request for volunteers to host download mirrors in India

  • Security Advisories support (Jenkins Infrastructure is the first user!)

Platform Update

  • Very last JDK8 docker images have been released

  • Preparation of the three agents repos merging while solving a bug.

  • Loss of the ppc64le loaner machine from IBM, some reference cleaning to do.

  • JDK17 aarch64 docker image bug discovered and solved.

  • First RISC-Vmachine received, tests should happen in the coming months

  • JVM stats for October: Java 8 fell from 153,205 to 139,827; Java 11 rose from 137,332 to 141,387; and Java 17 rose from 11,173 to 14,420. There are now more users on Java 11 than Java 8.

User Experience Update

The accounts.jenkins.io page has been significantly updated thanks to the work of Tim Jacomb, Alex Brandes, and Gavin Mogan.

At a high level, Tim and Alex made major dependency updates. Tim was also able to update the UI to be more modern and synchronize with the jenkins.io interface. Gavin was able to create and add the page header & footer, so that page transitions are seamless. This work to keep Jenkins aligned and up to date is greatly appreciated, and necessary to keep it modern.

image

The old Jenkins wiki site has also received a face lift, as it has been updated with the stylesheets used for the accounts page update. Despite the wiki being deprecated, finding and reading information on the site is much easier. This is yet another way to make the page transitions more seamless throughout Jenkins.

image

Documentation Update

image The annual Jenkins board members and officers election is here! Thanks Kevin Martens for thoroughly laying out the voter registration and nomination process. Nominate yourself or someone you know. But hurry because nominations end on November 10, 2022.

Can’t think of anyone to nominate? We’d gladly take your vote! Voter registration is open now until November 17. The voting period will begin on November 17 - December 2, 2022. The election process page in Jenkins has been updated accordingly.

Advocacy & Outreach Update

Google Summer of Code 2022 Ended

image

Congratulations to all GSoC contributors! Thank you for your contributions!

Jenkins GSoC 2022 ended in October with the successful completion of all 4 projects. Below are the final reporting from each projects:

Demos were represented during the Jenkins Online Meetup. The recording can be found here.

This program isn’t possible without the dedication of all the mentors. Hats off to all Jenkins GSoC mentors!

Hacktoberfest

image Participation in the 2022 edition of this worldwide event was strong. 117 seasoned but also first-time contributors submitted 613 eligible PRs. From these, 531 PRs are "Hacktoberfest complete" (merged or flagged as hacktoberfest-approved). They were submitted by 95 contributors (among them 42 qualify for the swag just with Jenkins contribution).

Congratulations and many thanks to those who contributed but also to those who advised/guided/reviewed/coached these contributors.


Google Summer of Code 2023 Call for Project Ideas and Mentors

$
0
0

Google Summer of Code (GSoC) is a global, online mentoring program focused on introducing students and new contributors to open source software development. GSoC contributors work on a 10-22 weeks programming project with the guidance of mentors from their open source organization. During GSoC, participating contributors are paired with mentors from open source organizations, gaining exposure to real-world software development techniques. GSoC contributors will learn from experienced open source developers while writing code for real-world projects! A small stipend is provided as an incentive.

See GSoC contributor eligibility here.

We are looking for project ideas and mentors to participate in GSoC 2023. GSoC project ideas are coding projects that potential GSoC contributors can accomplish in about 10-22 weeks. The coding projects can be new features, plugins, test frameworks, infrastructure, etc. Anyone can submit a project idea, but of course we like it even better if you mentor your project idea.

Please send us your project ideas before the beginning of February so they can get a proper review by the GSoC committee and by the community.

How to submit a project idea

Create a pull-request with your idea in a .adoc file in the project ideas. It is not necessary to submit a Google Doc, but it will still work if you want to do that. See the instructions on submitting ideas which include an .adoc template and some examples.

What does mentoring involve?

Potential mentors are invited to read the information for mentors. Note that being a GSoC mentor does not require expert knowledge of Jenkins. Mentors do not work alone. We make sure that every project has at least two mentors. GSoC org admins will help to find technical advisers, so you can study together with your GSoC contributor. Mentoring takes about 5 to 8 hours of work per week (more at the start, less at the end). Mentors provide guidance, coaching, and sometimes a bit of cheerleading. They review GSoC contributor proposals, pull-requests and contributor presentations at the evaluation phase. They fill in the Google provided final evaluations at the end of the coding period.

What do you get in exchange?

Mentor interaction is a vital part of GSoC. In return for mentoring, a GSoC contributor works on your project full time for 10-22 weeks. Think about the projects that you’ve always wanted to do but never had the time… Mentoring is also an opportunity to improve your management and people skills, while giving back to the community. GSoC is a fantastic program and the Jenkins project is looking forward to participating in GSoC again in 2023!

For any question, you can find the GSoC Org Admins, mentors and participants on the GSoC SIG Gitter chat.

Jenkins Elections Announcement

$
0
0

Hello all, voter registration and candidate nominations for the 2022 Jenkins elections are now complete. Thanks to everyone who registered to vote and to all those who submitted nominations. Special thanks to the candidates that have accepted the nominations.

As in years past, when a particular role receives only one nomination, an election is not required. This year, all the board and officer positions are uncontested, with one person nominated to fill each available position. This announcement shares the candidates and their statements, and serves as an announcement that the candidates have been elected and will begin their service December 3, 2022.

New Board Members

Board Member - Alexander Brandes

I am a Jenkins user, contributor to the Jenkins project, and core maintainer. I maintain many plugins, and have helped organize Jenkins online meetups, which I also have been part of. I attend and take part in documentation and user experience office hours frequently. Previously, I led community initiatives, such as the Jenkins plugin integration with Crowdin and participated in the Java 11 and 17 support campaigns.

I truly believe in Jenkins' future and would be happy to serve on the Jenkins governance board. I would like to make Jenkins a great place to contribute, and to get more beginners and regular contributors involved in the community on a regular basis. Onboarding more individual contributors would be one of my key targets, for example, through Jenkins' continued participation in Google Summer of Code.

On the technical aspect, I plan to move automatic plugin and component releases forward, and focus on the support and extension of modern technologies, such as Java 17 and beyond.

Affiliations: Student of business science

Board Member - Ullrich Hafner

I have been an active contributor in the Jenkins project since 2007. Currently, I am the maintainer of several Jenkins plugins, mostly in the area of build result visualization using modern JS based user interfaces: for example, I am the maintainer of the warnings, code coverage and git-forensics plugins as well as several UI related JS wrapper plugins like Bootstrap or ECharts. I am a member of the Jenkins UX SIG and I was a member of the Jenkins Governance Board during the 2019/2021 term. I am a professor of Software Engineering at the University of Applied Sciences Munich, where I also try to win new Jenkins contributors by letting students develop new features in their student projects and theses. I am also helping new contributors (and students) in our Jenkins forums or by giving some talks in our Jenkins Meetups. It would be a pleasure for me to serve again as a member of the governance board. On the board, I can help the Jenkins project evolve and follow a strict vision that will focus on the needs of the users.

Affiliations: University of Applied Sciences Munich

New Officers

Release Officer - Tim Jacomb

I have been a user of Jenkins for the last 10 years and a regular contributor since 2018. I began with maintaining the Slack plugin, and over the last couple of years, I have expanded my experience with several more plugins and the Jenkins core. These are some of the components I maintain when I have time: Slack, Azure Key Vault, Junit, most of the Database plugins, Dark theme, Plugin installation manager, Jenkins Helm chart, and Configuration as code plugin. I am also a member of the Jenkins infrastructure team, and I was involved in the release automation project and the mirrors modernisation effort, along with the day to day support helping people regain access to accounts etc. As a Release Officer I would like to increase automation, ease onboarding of new contributors to the release team, and ensure that responsibilities rotate among people so that I wouldn’t be a bottleneck for any task.

Affiliations: Kainos

Events Officer - Alyssa Tong

It has been a privilege for me to be part of the Jenkins community and serve as Events Officer in 2022. I am honored to serve as Events Officer for another term. I am passionate about open source, collaboration, and education. My goal is to build welcoming communities, and coordinating events is part of this role. More importantly, I strive to bring the community together through events and conferences, so we can share best practices and collaborate for the betterment of the project.

Affiliations: CloudBees

Security Officer - Wadeck Follonier

I joined the Jenkins adventure in 2017, as part of the Jenkins Security team and was coached by Oleg Nenashev and Daniel Beck about how to best contribute to the community effort, especially related to security tasks. I am working full time at CloudBees, managing two teams, including the security team dedicated to Jenkins. I was inspired by Daniel’s work in terms of tooling, to create an application helping both the security people and the maintainers through multiple bot commands (in SECURITY tickets and in Hosting request). During the year as the Security Officer, I spent a good amount of my time onboarding the future experts, finding ways to increase efficiency of the team by left shifting tasks, to reduce the size of our backlog and to put the focus on important tasks. For the next year, I would like to continue the effort to make Jenkins an even more secure place. One of the topic that I want to push forward is about the Content Security Policy introduction in the Jenkins ecosystem.

Affiliations: CloudBees

Documentation Officer - Kevin Martens

I am a technical documentation writer, and have been contributing to the Jenkins project consistently over the last seven months. I helped create the Blue Ocean status statement that is now in the documentation, DevOps World material, and was a mentor for the Jenkins Documentation area of Hacktoberfest 2022. I have recently been added to the copy-editor team due to my involvement with the Jenkins Documentation SIG and contributions.

I continue to learn by contributing blog posts to Jenkins and the CD Foundation, documentation updates and creation, and screenshot updates. I have also contributed to recent LTS release changelogs and upgrade guides, and am currently reviewing the weekly release lines. I review open pull requests, issues, and feedback, so that we can discuss further with the community and work to improve as needed. I’ve also developed a passion for open source, through reviewing documentation and building my own Jenkins environments.

I am currently hosting the EU Docs office hours, and would love to expand my knowledge, along with others. I want to empower the community further to increase activity and investment in the Jenkins project. My main goal is to help extend the community and ensure the documentation is accessible for both new and experienced users.

Affiliations: CloudBees

Infrastructure Officer - Damien Duportal

I am a Software Engineer working as a SRE for the Jenkins public infrastructure and as an IT teacher. I am active in the Platform SIG, and have been using Jenkins for over a decade. I am a fervent open-source citizen and active in other communities including Docker, Asciidoctor and Updatecli.

Affiliations: CloudBees

We want to congratulate the nominees and share thanks to the community for joining us in this year’s election.

Hacktoberfest in Jenkins 2022

$
0
0

As November continues, we want to share some highlights from Hacktoberfest 2022! This year’s edition was incredibly busy with lots of interesting ideas, great work, new Jenkins contributors, and returning contributors. We want to thank all of the contributors and mentors for all of their hard work, as none of this would be possible without them.

Hacktoberfest is an event to highlight and celebrate open-source projects like Jenkins. The main idea of the event is contributing to and improving open-source software. People are invited to submit pull requests and engage with open-source software in a variety of ways, such as contributing to plugins, creating documentation, providing localization, and reviewing code. Many products and services are based on open-source software, so giving back in this way continues the cycle of development.

For Jenkins, Hacktoberfest is an opportunity to collaborate with and expand the community, while learning and sharing with all experience levels. These contributions are what keeps Jenkins moving forward, so having an opportunity to celebrate both the project and the people is amazing.

Hacktoberfest results

During October, the Jenkins project received 613 pull requests from 117 different contributors. All of these helped to improve Jenkins, create new resources, and meet the needs of Jenkins users. From these, 531 pull requests are completed, meaning they have been merged or flagged as "hacktoberfest-approved". They were submitted by 95 different contributors, with 42 participants qualifying for swag with Jenkins contributions alone! A total of 1,183 manually created pull requests were submitted during October in the jenkins-ci and jenkins-infra organizations. Overall, this appears to be a 10 to 15 percent increase in the average number of monthly submissions. With this in mind, we want to highlight some new and returning Jenkins contributors, including their experiences and advice for other users. These contributors were selected at random, as Hacktoberfest had such high levels of participation.

New Jenkins contributors

First, we’d like to share some insight from Jagruti Tiwari. Jagruti is based in Mumbai, India and active in various areas of open source. This immersion is the main component of her strategy to explore and learn new things. She is not afraid of challenges and have received assistance from the friendly, open-source communities. She is inspired by thought leaders such as Ankita Tripathi. Jagruti’s contributions to Jenkins during Hacktoberfest include documentation, code examples, and testing. She has also provided reviews for other community members, which supports the open-source spirit.

Next, we want to introduce Omkar Borhade from Pune, India. Omkar just started as a software engineer, after graduating as a Mechanical Engineer. He is new to open source, and this was his first time participating in Hacktoberfest. His goal was to become more familiar the Jenkins project. This was achieved by a lot of reading, approaching issues with an open mind, and providing documentation. As a new contributor, he shared that it can be difficult to understand the issue and what is needed for resolution. This knowledge comes with experience, so have some patience when getting started.

Omkar Borhade

Kris Stern is a full-stack software engineer based out of Hong Kong SAR, China. He has previously trained as an observational astrophysicist with a PhD in physics, and is currently studying part-time for a master’s degree in computer science. This is his third Hacktoberfest, and first Hacktoberfest contributing to Jenkins. He has contributed to open source previously for astronomy, but shifted focus since graduating. His goals during Hacktoberfest were giving back to the Jenkins community and acquiring more software development skills along the way.

Kris Stern

Returning Jenkins contributors

Jim Klimov is a regular Jenkins contributor. Jim is based in the Czech Republic, and has been working in open source for quite some time. He has worked on various enterprise document workflow and messaging systems, and is currently a DevOps architect. He is the creator and maintainer of several Jenkins plugins. Along with his own experience, he also learned a lot from previous contributors. Having this knowledge feels like standing on the shoulders of giants. This is his second Hacktoberfest in Jenkins, but Jim contributes all year round. Challenges that have come up for Jim include maintainability expectations to get pull requests merged, and architecture becoming more costly as time goes on.

Teona Mushambadze is another contributor who submitted work to Jenkins during Hacktoberfest. Teona created three new logos: Turkey, Georgia, and a Jenkins nerd themed logo. These logos help connect Jenkins to users globally and extend the community even further. She is a freelance developer, and this is her first time contributing to Jenkins in Hacktoberfest. Her motivation when starting was curiosity and learning more about open source. Through her contributions, she has learned how to work with someone else’s code. There have been challenges when new tools are installed, but don’t work as expected. However, these are learning experiences, as each one is an opportunity to understand what is causing the behavior.

Teona Mushambadze

Stefan Spieker is a solutions architecht and DevOps engineer from Aachen, Germany. He has been participating in Hacktoberfest since 2019, with a strong focus on Jenkins core and plugin development. Hacktoberfest was the trigger to start participating, and Stefan has continued to contribute regularly. Stefan likes to contribute to open source in his free time and has found that there are always ways to improve. For Hacktoberfest, this included updating projects that still have spotbugs disabled and adopting a plugin to become a maintainer. Since he uses open-source software daily, especially in his professional life, Stefan embraces this by giving back as an OSS consumer. He is an advocate for others contributing to open source, and has encouraged colleagues to participate in any capacity they can. Stefan feels that the first pull request is always toughest, due to the challenges of setting up an initial environment and meeting approval expectations of a maintainer.

Kayla Altepeter is a senior engineer from Minneapolis, Minnesota and has been participating in Hacktoberfest since 2018. Despite having less bandwidth to dedicate to this year, Kayla shared the Hacktoberfest information with her colleagues. This resulted in another person contributing, which encouraged Kayla to participate in this year’s event. Open source is mission critical for her own fun projects and the projects built at work. Kayla contributes to Jenkins because Jenkins is how product code is delivered. She also shared that Jenkins maintainers have been responsive and helpful when it comes to creating, reviewing, and merging pull requests. This collaboration is a hallmark of open source and important to the continued progress of Jenkins.

Kayla Altepeter

Contributor insights

The thrill you get from knowing that countless people are using something you built is something a swag can never match.

If an issue is too hard to solve at the moment, take a break from it and try out a different one.

— Jagruti Tiwari

Contributing to open source also gives a feeling of satisfaction that the projects you are contributing to are used by several people on this planet and your contributions are benefiting them in one way or another.

When you are new, it does feel scary and confusing. Patience is important in the beginning. Don’t be afraid to ask your doubts. Find good first issues that you are comfortable with.

Since, my all contributions for this Hacktoberfest were to Jenkins project, I would like to thank Jenkins and team to accepts my commits, guiding me in the resolution of issues and helping me to learn and grow by the means of the project.

— Omkar Borhade

For people hesitating, there is nothing to fear. Start with the easiest issues, and step after step it gets more fun. You will notice how you grow as a developer.

— Teona Mushambadze

I think open source is important because besides its utility it is also a great way to organize knowledge and to build communities with a common set of interests or purposes.

Sometimes engaging in open source means tinkering with new tech and going at it alone. It is challenging at times but also tremendously rewarding.

Sometimes it is hard to get started, and it takes time and perseverance to make things work the way they are intended.

— Kris Stern

Find a project that interests you, with technologies or approaches you want to learn, go tinker, and post pull requests.

In any case, you would learn more about the world, project, yourself, interactions and patience.

Documentation is one area almost everyone can do better, and almost anyone can help improve. It is easy to overlook something as "apparent" after a decade of experience with a project when it is really non-trivial for a newcomer.

In case of Jenkins core, plugin or shared library contributions, keep in mind that Java IDEs like NetBeans or IDEA can be very helpful to step through the server sources with a debugger. Peppering code with temporary printlns only goes so far, sometimes you will need real tools.

— Jim Klimov

To those that hesitate: I encourage everyone to try it out. We have within Jenkins a great community, which tries to help so that the PR also gets merged eventually.

— Stefan Spieker

If you are afraid to contribute, find a repo with clear steps to remove that hurdle and just try to set it up locally. If you can do that, you can open a pull request and someone will probably offer to help if you get stuck. Finding orgs that have a good chat or helpful maintainers makes it easier.

Fixing a small bug that affects you is great because you know the issue.

— Kayla Altepeter

GSoC 2023: how to get prepared

$
0
0

We are happy to see a lot of questions on how to prepare efficiently to submit a powerful project submission. This is key to be selected and participate in the coming Google Summer of Code (GSoC). This blog post is addressed to those who ask these questions.

Jenkins GSoC

First of all, welcome to the Jenkins Community! Thank you for your interest in Jenkins in general and for GSoC 2023 in particular.

Only a limited number of GSoC slots are made available by Google. We will select the best proposals with the highest chance for a succesful outcome. It is thus a wise step to prepare early. It will put all the chances on your side. You will find hereafter some advice about activities to build the muscle required to be selected.

The objective is to build

  • functional knowledge of Jenkins,

  • understanding of the basics of plugins and Jenkins core internals and development,

  • improvements of your java development skills (the main programing language of this project),

  • practical experiences by contributing to an Open Source project.

First of all, install and run your own Jenkins server. You can install it on your own machine, on a spare server, or even in the Cloud (some Cloud vendors offer free tiers). Configure it to build a real application, preferably using descriptive pipeline syntax. Install, configure, and use a couple of plugins. This will help you to get a functional understanding and a practical experience of the product.

To build your Java/Jenkins "development muscle", start proposing fixes or enhancements in the form of pull requests. You can find open newbie friendly issues via this query. Issues describe problems that need fixing or enhancement ideas. To move forward, use the documentation available on Jenkins.io (https://www.jenkins.io/doc/developer/) or ask for advice/help on the developer mailing list, Discourse, or the Gitter channels (see https://www.jenkins.io/projects/gsoc/#contacts for the details).

Another way to get experience is to follow the suggestions of the "Improve a Plugin Tutorial".

And of course read the rules and general advice specific to Google Summer of Code: https://www.jenkins.io/projects/gsoc/students/ and https://opensource.googleblog.com/2022/11/get-ready-for-google-summer-of-code-2023.html. Also a useful reading are previous year’s submission and recorded meetings.

Remember it is always a good idea to let others know about your contributions to the community especially via IRC conversations, GitHub issues, as well as pull requests. Be sure to ask questions when and where appropriate. Open-source software development is a group endeavour, and we are here to offer guidance to help you to become a part of the community. Therefore open and honest communication is always encouraged and is highly appreciated.

We will hold an online meeting where we will come back on these advises and answer any question you might have. The meeting will take place on Tuesday December 20, 2022 at 16:00 UTC. The Zoom meeting link is https://zoom.us/j/92664728323. It will be recorded.

Next steps

The jenkins GSoC team is currently assembling project ideas for candidates to explore and submit. We are also recruiting mentors and preparing our application.

In January, we will organise one or several meetings where the project ideas will be explained in details. Candidates will have the possibility to ask questions to clarify any doubts. We will also explain how to proceed to prepare a good submission.

Jenkins Sponsor Appreciation

$
0
0

The Jenkins project would like to take a moment and share deep gratitude to our sponsors. Sponsorship of the Jenkins project is key to the maintenance and development of Jenkins. There are different ways in which our sponsor organizations support the Jenkins project, such as infrastructure, necessary development tools, funding, and providing mirrors to connect Jenkins to the community.

For supporting Jenkins distribution, build, test, and deployment infrastructure, we would like to thank JFrog, AWS, Oracle, DigitalOcean, Datadog, Docker, PagerDuty, Fasty, GitHub and OSU Open Source Labs.

JFrog      AWS      Oracle      DigitalOcean      OSU Open Source Labs      Datadog      Docker      PagerDuty      Fastly     

We also want to recognize and thank GitHub, JFrog, Atlassian, Linux Foundation, Netlify, and 1Password for providing tools to track Jenkins development.

GitHub      JFrog      Atlassian      1Password      Netlify      Linux Foundation     

Massive thanks to Algolia for providing site search for the primary Jenkins documentation and plugins sites.

Algolia

We also want to share deep gratitude for operational funding from CD Foundation, CloudBees, AWS, and DigitalOcean. Without funding, the community would have less opportunities to work on and develop Jenkins.

CD Foundation      CloudBees      AWS      DigitalOcean     

Finally, we want to thank the various organizations that host mirrors for Jenkins worldwide distribution. Thank you to OSU Open Source Labs, XMission, Tsinghua University, Yamagata University, Gruenehoelle NL, Belgian Education and Research Network, and RWTH Aachen University.

2022 has been a fantastic year for the Jenkins project, and it would not be possible without all of the support from our sponsors and community!

Viewing all 1088 articles
Browse latest View live