A short introduction to Jenkins, automation, CI/CD and DevOps
Note: Use Docker and Docker Compose (optional) to start a Jenkins server - don't install Jenkins directly on your machine if you don't have to
- Hello World -
Freestyle
- Setting up Windows agents
- Managing plugins
- Code review with Gerrit
For more information about Git and Gerrit, check out the git - the simple guide and Using Gerrit to enhance your Git video from the Eclipse foundation.
- Install Git
- Open Git Bash
- Configure user name and email
git config --global user.email "<EMAIL>" git config --global user.name "<NAME>"
- Run
ssh-keygen
leaving every input emptyNote: This should never be done in real systems
- Run
cat ~/.ssh/id_rsa.pub
and save the input for later - Go to
localhost:9090
and click Skip Intro to go to the Dashboard - Open Settings ⚙️
- Go to Profile and change the Full Name to
<NAME>
- Go to Email Addresses, type
<EMAIL>
into New email address and click Send VerificationNote: In developer mode, Gerrit will skip this validation
- Go to SSH Keys, paste the output of the
cat
command in the New SSH Key textbox and click Add New SSH Key - Go to Browse > Repositories and create a new repository by clicking Create New
Repository Name:
<REPO_NAME>
Owner:<NAME>
Create initial empty commit:True
Only serve as parent for other repositories:False
- Go back to the Git Bash window and clone the repository
git clone "ssh://admin@localhost:29418/<REPO_NAME>" && scp -p -P 29418 admin@localhost:hooks/commit-msg "<REPO_NAME>/.git/hooks/"
- Make a few changes and push everything to Gerrit
cd <REPO_NAME> echo 'Hello World!' > README.md git add README.md git commit -m "Added README" git push origin HEAD:refs/for/master
- Static analysis with SonarQube
-
Go to
localhost:9000
and log into SonarQubeWhen installing SonarQube, a default user with Administer System permission is created automatically (Login:
admin
, Password:admin
) cf. Instance Administration > Security for additional information -
Go to My Account > Security
-
In Tokens, enter a token name and click Generate - copy the token so we can use it later
-
Go to Plugin Manager and install the SonarQube Scanner plugin
-
Go to Configure System > SonarQube Servers and click Add SonarQube Name:
sonar
Server URL:http://sonarqube:9000
-
In Server authentication token, click Add, select Jenkins and add the sonar token cf. image below for more details
-
Apply and Save
-
Go to Global Tools Configuration > SonarQube Scanner and click Add SonarQube Scanner
-
Apply and Save
-
Create a new Freestyle job
-
In Source Code Management, select
Git
and set Repository URL e.g.https://github.com/Coveros/helloworld
-
In Build, click Add build step and select Executre SonarQube Scanner
-
Set Tasks to run to
-Dsonar.projectKey=helloworld
-
Apply and Save
-
- Unit tests with Maven
- Go to Plugin Manager and install the Maven Integration plugin
- Go to Global Tools Configuration and add a new Maven installation
- Create a new Maven project
- In Source Code Management, select
Git
and add the Repository URL e.g. https://github.com/jenkins-docs/simple-java-maven-app - In Build, set Goals and options to
clean install
- Apply and Save
- Trigger a new build by clicking Build Now
- UI automation with RobotFramework + Selenium
- Setting up your Jenkins machine to execute Robot Framework automation tests
- Install Selenium libraries and drivers
pip3 install selenium robotframework-selenium2library robotframework-seleniumtestability webdrivermanager
- Warning: Beware ice-cream cones 🍦, hourglasses and other test automation anti-patterns
- Running Postman collections with Newman
- Use postmanlabs/newman
newman run examples\sample-collection.json --disable-unicode --reporter-junit-export results.xml --reporters cli,junit
- Use postmanlabs/newman
- Hello World -
Pipeline
- Managing Jenkins with Script Console
- LDAP authentication
Note: use
docker-compose.prod.yml
to start an LDAP service and use phpLDAPadmin to create an organizational unit (OU) forusers
- Go to Manage Jenkins > Security > Configure Global Security
- Scroll down to Security Realm and select
LDAP
Server:ldap://ldap
root DN:ou=users,dc=example,dc=org
User search filter:uid={0}
Manager DN:cn=admin,dc=example,dc=org
Manager Password:secret
Display Name LDAP attribute:cn
Email Address LDAP attribute:mail
- Test LDAP Settings before saving
- Save and Apply
- Updating Jenkins
-
Open a terminal inside master
docker-compose exec -u root master bash
-
Replace
jenkins.war
cd /usr/share/jenkins wget https://updates.jenkins.io/download/war/<NEW_VERSION>/jenkins.war -O jenkins.war
-
Prepare for shutdown, wait for all builds to stop and restart Jenkins
-
- Martin Fowler's website - check out the articles on CI, CD and how to test microservice architectures
- (James Shore) Continuous Integration on a Dollar a Day - oldie but goodie, a funny introduction to CI
- (Google Cloud) DevOps - Google Cloud's take on DevOps, read the chapter on Continuous Testing
- (Google) SRE Books - the Site Reliability Engineering (SRE) team at Google has published some interesting tomes on planetary-scale production Ops and how they've made automation a thing of beauty
- (GitHub) atinfo/awesome-test-automation - a curated list of test automation frameworks, tools and libraries
- (DevOps Cube) Jenkins Architecture Explained – Beginners Guide - concise introduction to Jenkins architectural components
- (Medium) Top Jenkins Tutorials to Start in 2021 - Learn Jenkins online - check out one or two, and get your hands dirty
- (Beyer et al., 2016) Site Reliability Engineering: How Google Runs Production Systems
- (Crispin & Gregory, 2014) Agile Testing: A Practical Guide for Testers and Agile Teams
- (Davis & Daniels, 2016) Effective DevOps: Building a Culture of Collaboration, Affinity, and Tooling at Scale
- (Forsgren, Humble & Kim, 2018) Accelerate: Building and Scaling High Performing Technology Organizations
- (Humble & Farley, 2010) Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation
- (Kim, Behr & Spafford, 2013) The Phoenix Project: A Novel about IT, DevOps, and Helping Your Business Win
- (Kim et al., 2016) The DevOps Handbook: How to Create World-Class Agility, Reliability & Security in Technology Organizations
- (Kim, 2019) The Unicorn Project: A Novel about Developers, Digital Disruption, and Thriving in the Age of Data
- (Laster, 2018) Jenkins 2 Up and Running: Evolve Your Deployment Pipeline for Next Generation Automation
- (Leszko, 2017) Continuous Delivery with Docker and Jenkins: Delivering software at scale
- (Newman, 2015) Building Microservices: Designing Fine-Grained Systems
- (Smart, 2011) Jenkins The Definitive Guide: Continuous Integration for the Masses
- (Whittaker, Arbon & Carollo, 2012) How Google Tests Software
- (Winters, Manshreck & Wright, 2020) Software Engineering at Google: Lessons Learned from Programming Over Time