Day 28 - Jenkins Agents

Introduction

Jenkins, an open-source automation server, plays a crucial role in orchestrating workflows through pipelines. In larger and more complex environments, the need for distributing workload and scaling up becomes apparent. This is where Jenkins Agents come into play. In this task, we'll explore the concept of Jenkins Agents, their role, and how to set them up for efficient workload distribution.

Jenkins Master (Server)

The Jenkins master, or server, serves as the control center for Jenkins. It houses all key configurations and orchestrates the execution of workflows defined in pipelines. From scheduling jobs to monitoring their progress, the master server is responsible for managing the overall Jenkins environment.

Jenkins Agent

A Jenkins agent is a machine or container connected to the Jenkins master, responsible for executing the steps outlined in a job. When creating a Jenkins job, an agent must be assigned to it, and each agent has a unique label for identification. When a Jenkins job is triggered, the actual execution takes place on the agent node configured for that specific job.

For small teams and projects, a monolithic Jenkins installation might suffice. However, as the number of projects and workload increases, scaling up becomes essential. Jenkins provides a solution known as "master to agent connection," where agents handle job execution, allowing the master to focus on serving the Jenkins UI and acting as a control node.

Pre-requisites

Assuming a fresh Ubuntu 22.04 Linux installation, setting up a Jenkins agent involves installing Java (matching the version on the Jenkins master server) and Docker. It's crucial to manage rights, permissions, and ownership for Jenkins users when creating agents.

Task-01: Setting Up a Jenkins Agent

Step 1: Create an AWS EC2 Instance

Begin by launching a new AWS EC2 instance. This instance will serve as the agent node connected to the Jenkins master. Follow these steps:

1. Log in to your AWS Management Console.
2. Navigate to the EC2 dashboard.
3. Click on "Launch Instance" to create a new virtual machine.

Step 2: Configure EC2 Instance

During the instance configuration process:

1. Choose an Amazon Machine Image (AMI), preferably a recent version of Ubuntu.
2. Select an instance type based on your requirements.
3. Configure instance details, including the number of instances, network settings, and storage.

Step 3: Add Security Group Rules

Ensure that your security group allows SSH access. Add an inbound rule for SSH (port 22) to allow connections from your Jenkins master server.

Step 4: Launch Instance

Review your configurations and launch the instance. AWS will prompt you to select or create a key pair for connecting to the instance securely. Save the private key file (.pem) as you will need it to connect via SSH.

Step 5: Connect EC2 Instance to Jenkins Master

Once the EC2 instance is running, use SSH to connect to it from your Jenkins master server. Use the private key you saved during the instance launch. The command will look like this:

ssh -i /path/to/private-key.pem ubuntu@ec2-instance-ip        

Step 6: Install Java and Docker

On the EC2 instance, install Java and Docker to ensure compatibility with the Jenkins master server. Ensure that the Java version matches the one installed on the master.

Step 7: Add Agent Node in Jenkins UI

1. In the Jenkins UI, navigate to "Manage Jenkins" > "Manage Nodes and Clouds."
2. Click on "New Node" to add a new agent.
3. Enter a name for the agent, select "Permanent Agent," and click "OK."

Step 8: Configure Agent Node

1. Set the number of executors (parallel job executions) for the agent.
2. In the "Remote root directory" field, specify the working directory on the agent.
3. Under "Launch method," choose "Launch agent via SSH."

Step 9: Add Credentials

In the "Credentials" section, click on "Add" to provide SSH credentials for connecting to the agent. Use the private key associated with the EC2 instance.

Step 10: Save and Connect

Save the agent configuration, and Jenkins will attempt to connect to the agent using the provided SSH credentials. Once connected, the agent status should change to "Online."

Task-02: Running Jobs on the New Agent

Now that the agent is set up, you can run previous jobs on it, optimizing workload distribution. Follow these steps:

Step 1: Use Labels for Agent

1. Edit the Jenkins job you want to run on the new agent.
2. Under the "General" section, check the option "Restrict where this project can be run."
3. Enter the label assigned to the new agent.

Step 2: Trigger Builds on Agent

1. Save the job configuration.
2. Manually trigger a build or wait for the next scheduled build.
3. Check the build console output to confirm that the job is being executed on the new agent.

By completing both tasks, you've successfully set up a Jenkins agent on an AWS EC2 instance, connected it to the master server, and configured jobs to run on the new agent, enhancing the scalability and efficiency of your Jenkins environment.