Jenkin Gerrit JJB

Integration

Before start development, we need to configure Jenkins.

Jenkins Job Builder (JJB) takes simple descriptions of Jenkins jobs in YAML or JSON format and uses them to configure Jenkins. You can keep your job descriptions in human readable text format in a version control system to make changes and auditing easier.
It also has a flexible template system, so creating many similarly configured jobs is easy.

For our Jenkins jobs we will provide the following workflow:

• User creates JenkinsJob (yaml format)
• User commit job into gerrit cicd-jenkins-job repo and send it to review
• Jenkins trigger detects "review-created" event in gerrit and run syntax check job wich check yaml syntax
  • If job success: Voite +1
  • If job fails: Voite -1

• In case of "+1" human can review and add +1
• Merge
• Jenkins trigger detects "merge" event in gerrit and run update job

As result, job defined in yaml and commited to repo will be tested and created in Jenkins.

Create repo in gerrit

Create project, which inherits all permission:

Install, configure and test JJB

Install

• Install pip and libs:

apt-get install python-pip python-yaml

pip install  PyYAML python-jenkins

• Install JJB

git clone https://git.openstack.org/openstack-infra/jenkins-job-builder
cd jenkins-job-builder/
python setup.py install

• Now JJB is installed

# jenkins-jobs

usage: jenkins-jobs [-h] [--conf CONF] [-l LOG_LEVEL] [--ignore-cache]
                    [--flush-cache] [--version] [--allow-empty-variables]
                    {update,test,delete,delete-all} ...
jenkins-jobs: error: too few arguments

Configure JJB

JJB uses config file (defined with --conf option)
For jenkins I'd like to have separate user with separate key, so let's generate rsa key, create user in ldap and add this user to group

• Generate key:

#su - jenkins

$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/var/lib/jenkins/.ssh/id_rsa):
Created directory '/var/lib/jenkins/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/lib/jenkins/.ssh/id_rsa.
Your public key has been saved in /var/lib/jenkins/.ssh/id_rsa.pub.
The key fingerprint is:
99:6c:fb:38:1e:5c:05:c9:ba:d6:cf:9b:f2:db:f0:a6 jenkins@cicd
The key's randomart image is:
+--[ RSA 2048]----+
|         ...     |
|          o.     |
|         .  .    |
|       ..o .     |
|        So.      |
|       ooo.      |
|       .+  o.    |
|        .+. o=.  |
|       .o..oE=o  |
+-----------------+

• Create user in ldap (same password as for all other users):
  • Jenkins user:

dn: cn=Jenkins User,ou=cicd,dc=demo
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: Jenkins User
ou: cicd
uid: jenkins-user
givenName: Jenkins
sn: user1
userPassword: {SSHA}5oLdx/TJdGrRb3Jaz/9JWuFsj59pPoPt
mail: jenkins@jenkins.com

• • Add to group:

dn: cn=cicd  administrators,ou=cicd,dc=demo
changetype: modify
add: memberUid
memberUid: jenkins-user

• • Run commands:

# ldapmodify < jenkins-user
adding new entry "cn=Jenkins User,ou=cicd,dc=demo"

# ldapmodify < add_jenkins_to_group
modifying entry "cn=cicd  administrators,ou=cicd,dc=demo"

• Check in LDAP

• Check Login in Jenkins:

• Log-in to Gerrit and add key

On this step w have user, which can manage jenkins, pull data from gerrit and vote for pathces

Test JJB

• Create config jenkins_jobs.ini

[job_builder]
ignore_cache=True
keep_descriptions=False
include_path=.:scripts:~/git/
recursive=False
exclude=.*:manual:./development
allow_duplicates=False

[jenkins]
user=jenkins-user
password=r00tme
url=http://192.168.56.102:8080
query_plugins_info=True
##### This is deprecated, use job_builder section instead
#ignore_cache=True

• Clone cicd-jenkins-job

$ git clone ssh://jenkins-user@192.168.56.102:29418/cicd-jenkins-job
Cloning into 'cicd-jenkins-job'...
remote: Counting objects: 2, done
remote: Finding sources: 100% (2/2)
remote: Total 2 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (2/2), done.
Checking connectivity... done.

• Create test job doing nothing just show env
  • test.yaml

- job:
    name: 'test'
    project-type: freestyle
    description: |
        Test Job to test JJB dooing nothing
    concurrent: true
    wrappers:
      - timeout:
          timeout: 30
          fail: true
      - timestamps
      - ansicolor:
          colormap: xterm
    logrotate:
      daysToKeep: 1
      numToKeep: -1
      artifactDaysToKeep: -1
      artifactNumToKeep: -1
    properties:
      - heavy-job:
          weight: 1
    parameters:
      - string:
          name: PARAMETER1
          default: master
          description: |
             TEST_PARAMETER
    builders:
      - shell:
          !include-raw './builders/test.sh'
    publishers:
      - archive:
          allow-empty: true
          artifacts: 'artifacts/*.txt'
          latest-only: false

mkdir ./builders

• ./builders/test.sh

#!/bin/bash



mkdir -p artifacts

env > artifacts/env.txt

• Test syntax

jenkins-jobs --conf jenkins_jobs.ini test cicd-jenkins-job/demo/test.yaml

$ jenkins-jobs --conf jenkins_jobs.ini update  cicd-jenkins-job/demo/test.yaml

INFO:root:Updating jobs in ['cicd-jenkins-job/demo/test.yaml'] ([])
/usr/local/lib/python2.7/dist-packages/jenkins/__init__.py:530: DeprecationWarning: get_plugins_info() is deprecated, use get_plugins()
  DeprecationWarning)
WARNING:jenkins_jobs.local_yaml:tag '!include-raw' is deprecated, switch to using '!include-raw:'
INFO:jenkins_jobs.local_yaml:Including file './builders/test.sh' from path '/var/lib/jenkins/tmp/cicd-jenkins-job/demo'
WARNING:root:logrotate is deprecated on jenkins>=1.637, use the property build-discarder on newer jenkins instead
INFO:jenkins_jobs.builder:Number of jobs generated:  1
INFO:jenkins_jobs.builder:Creating jenkins job test
INFO:jenkins_jobs.builder:Cache saved
INFO:root:Number of jobs updated: 1
INFO:jenkins_jobs.builder:Cache saved

• Check Jenkins:

Automate IT!

Next step is to automate jenkins update.

Configure Jenkins