Gerrit

Gerrit

Это часть лекции CI/CD за один день,

• основная статья - http://wiki.sirmax.noname.com.ua/index.php/CI_CD_1_day

Gerrit is a free, web-based collaborative code review tool that integrates with Git. It has been developed at Google by Shawn Pearce (co-author of Git, founder of JGit) for the development of the Android project.

Starting from a set of patches for Rietveld, it became a fork and evolved into a full blown project when ACL patches wouldn't be merged into Rietveld by its author, Guido van Rossum.

Originally written in Python like Rietveld, it is now written in Java (Java EE Java Servlet) with SQL since version 2.

Installation

Gerrit installation and configuration is complicated so it will be described as detailed as it possible.

Prerequisites

By-default gerrit use HB database, but MySQL also supported and we will use it. Also, Java is required (already installed). For demo we install MySQL on the same server but of course it is possible to use external MySQL / Postres /Oracle
Also git is required for gerrit

Install git

apt-get  install git git-man git-review  liberror-perl

Install MySQL

apt-get  install mysql-server-5.6  mysql-common-5.6  mysql-client-core-5.6  mysql-client-5.6  mysql-source-5.6  mysql-testsuite-5.6 

Configure empty root password. (for demo only!)

Check MySQL

# mysql
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 44
Server version: 5.6.28-0ubuntu0.14.04.1 (Ubuntu)

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>

deb package

First step is get deb package for ubuntu and install it.
Deb package is configured in wrong way, so we need only install is, stop gerrit service and configure it before continue.

Download and install

• Download gerrit deb package

wget  http://deb.gerritforge.com/dists/gerrit/contrib/binary-amd64/gerrit-2.11.8-1.noarch.deb

• Install downloaded package.

dpkg -i gerrit-2.11.8-1.noarch.deb

Cleanup data created by post-install

• Remove data from $site_dir.
  

By-default installer use /var/gerrit as $site_dir, but can be changed in /etc/default/gerritcodereview file

cat /etc/default/gerritcodereview
GERRIT_SITE=/var/gerrit

deb installer creates and initialized gerrit with wrong parameter, so we need to delete all folders except bin

cd /var/gerrit

ls -1

bin
cache
data
db
etc
git
index
lib
logs
plugins
static
tmp

Remove dirs we do not need:

rm -rf  ./cache  ./data   ./db  ./git  ./index  ./lib  ./logs  ./static  ./tmp

Configure gerrit

Before initialize we need to prepare gerrit confg files. All config files are stored in /etc/gerrit (which is symbolic link to /var/gerrit/etc)
There are 2 configuration files:

• gerrit.config
• secure.config

gerrit.config

gerrit.config is main gerrit configuration file.

[gerrit]
	basePath = git
	canonicalWebUrl = http://192.168.56.102:8081/r/

[database]
	type = mysql
	database = gerrit
	hostname = localhost
	username = gerrit
	password = gerritpass

[auth]
	type = LDAP

[sendemail]
	smtpServer = localhost

[sshd]
	listenAddress = *:29418

[httpd]
	listenUrl = proxy-http://127.0.0.1:8081/r/

[cache]
	directory = cache

[cache "ldap_groups"]
        maxAge = 1 min

[cache "ldap_usernames"]
        maxAge = 1 min

[cache "accounts"]
        maxAge = 5 min

[cache "accounts_byemail"]
        maxAge = 5 min

[cache "diff"]
        maxAge = 5 min

[cache "groups"]
        maxAge = 5 min

[cache "projects"]
        maxAge = 5 min

[cache "sshkeys"]
        maxAge = 5 min

[receive]
	enableSignedPush = false

[container]
	user = gerrit
	javaHome = /usr/lib/jvm/java-7-openjdk-amd64/jre

[plugins]
    allowRemoteAdmin = true

[ldap]
	server = ldap://127.0.0.1
	username = cn=admin,dc=demo
	password = r00tme
	accountBase = ou=cicd,dc=demo
        accountFullName = ${givenName} ${SN}
	accountEmailAddress = mail
	groupBase = ou=cicd,dc=demo
        groupPattern = (cn=${groupname})
        groupMemberPattern = (memberUid=${username})
        groupQuery = true
        groupsVisibleToAll = true
        localUsernameToLowerCase = true

[index]
	type = LUCENE

[download]
        command = checkout
        command = cherry_pick
        command = pull
        command = format_patch
        scheme = ssh
        scheme = http

• gerrit: General gerrit options
  • canonicalWebUrl: Canonical url. All links in gerrit are related to this URL
• database: Database-related options like host, password, username.
• auth: gerrit auth method (we use LDAP)
• sshd: ssh listener address
• httpd: httpd listener address, proxy-http configured because we will use apache as proxy for gerrit. Using apache is optional.
• cache: Cache options.
• container: Java options
• plugins: Allow remote plugin management.
• ldap: Ldap options
  • username = cn=admin,dc=demo, LDAP admin username
  • password = r00tme LDAP admin password
  • accountBase = ou=cicd,dc=demo Organization unit where to search for accounts
  • groupBase = ou=cicd,dc=demo OU for groups
  • groupMemberPattern = (memberUid=${username}) Group membership query.

Configure MySQL backend

As you can see in config file, we need mysql database configured:

CREATE DATABASE gerrit;
GRANT ALL on gerrit.* to 'gerrit'@'localhost' IDENTIFIED BY 'gerritpass';
GRANT ALL on gerrit.* to 'gerrit'@'%' IDENTIFIED BY 'gerritpass';
FLUSH PRIVILEGES;

Initialize gerrit

Next step is Re-Initialize Gerrit with LDAP and MySQL

java -jar /var/gerrit/bin/gerrit.war  init -d  /var/gerrit

Using secure store: com.google.gerrit.server.securestore.DefaultSecureStore

*** Gerrit Code Review 2.11.8
***


*** Git Repositories
***

Location of Git repositories   [git]:

Set up password (all other parameters are taken from config)

*** SQL Database
***

Database server type           [mysql]:

Gerrit Code Review is not shipped with MySQL Connector/J 5.1.21
**  This library is required for your configuration. **
Download and install it now [Y/n]? Y
Downloading http://repo2.maven.org/maven2/mysql/mysql-connector-java/5.1.21/mysql-connector-java-5.1.21.jar ... OK
Checksum mysql-connector-java-5.1.21.jar OK
Server hostname                [localhost]:
Server port                    [(mysql default)]:
Database name                  [gerrit]:
Database username              [gerrit]:
gerrit's password              :
              confirm password :

Pay attention: Index mast be rebuilt

*** Index
***

Type                           [LUCENE/?]:

The index must be rebuilt before starting Gerrit:
  java -jar gerrit.war reindex -d site_path

Configure LDAP password. All other parameters are taken from config
<PRE>
*** User Authentication
***

Authentication method          [LDAP/?]:
LDAP server                    [ldap://127.0.0.1]:
LDAP username                  [cn=admin,dc=demo]:
cn=admin,dc=demo's password    :
              confirm password :
Account BaseDN                 [ou=cicd,dc=demo]:
Group BaseDN                   [ou=cicd,dc=demo]:

*** Review Labels
***

Install Verified label         [y/N]?

*** Email Delivery
***

SMTP server hostname           [localhost]:
SMTP server port               [(default)]:
SMTP encryption                [NONE/?]:
SMTP username                  :

*** Container Process
***

Run as                         [gerrit]:
Java runtime                   [/usr/lib/jvm/java-7-openjdk-amd64/jre]:

DO NOT update gerrit, latest version my have BUGS or do not have plugins you need.
Use only well-tested gerrit versions!

Upgrade /var/gerrit/bin/gerrit.war [Y/n]? n

Java, http and ssh settins are taken from config.

*** SSH Daemon
***

Listen on address              [*]:
Listen on port                 [29418]:

*** HTTP Daemon
***

Behind reverse proxy           [Y/n]?
Proxy uses SSL (https://)      [y/N]?
Subdirectory on proxy server   [/r/]:
Listen on address              [127.0.0.1]:
Listen on port                 [8081]:
Canonical URL                  [http://192.168.56.102:8081/r/]:

For Demo we need only download-commands plugin. (for prod you can use any plugin you need, or add plugins later )

*** Plugins
***

Installing plugins.
Install plugin singleusergroup version v2.11.8 [y/N]?
Install plugin commit-message-length-validator version v2.11.8 [y/N]?
Install plugin reviewnotes version v2.11.8 [y/N]?
Install plugin replication version v2.11.8 [y/N]?
Install plugin download-commands version v2.11.8 [y/N]? y
Initializing plugins.
No plugins found with init steps.

Initialized /var/gerrit

Fix permissions

chown gerrit:gerrit 

Rebuild indexes

sudo -H -u gerrit java -jar /var/gerrit/bin/gerrit.war  reindex

sudo -H -u gerrit java -jar /var/gerrit/bin/gerrit.war  reindex -d /var/gerrit/
[2016-03-24 17:21:18,244] INFO  com.google.gerrit.server.git.LocalDiskRepositoryManager : Defaulting core.streamFileThreshold to 183m
[2016-03-24 17:21:18,838] INFO  com.google.gerrit.server.cache.h2.H2CacheFactory : Enabling disk cache /var/gerrit/cache
Reindexing changes: done
Reindexed 0 changes in 0.0s (0.0/s)
[2016-03-24 17:21:19,854] WARN  com.google.gerrit.server.cache.h2.H2CacheImpl : Cannot build BloomFilter for jdbc:h2:file:/var/gerrit/cache/diff_intraline: Error opening database: "Sleep interrupted" [8000-174]
[2016-03-24 17:21:19,855] INFO  com.google.gerrit.server.cache.h2.H2CacheFactory : Finishing 4 disk cache updates
root@cicd:/var/gerrit#

Start gerrit

• Start Gerrit Service

/etc/init.d/gerrit  start

• Check process

ps -auxfww | grep 'Gerrit'
gerrit   13050 41.0  4.8 1649372 150576 ?      Sl   17:23   0:09 GerritCodeReview -jar /var/gerrit/bin/gerrit.war daemon -d /var/gerrit --run-id=1458833013.13022

• Check open ports

netstat  -ntpl | grep -i gerrit
tcp6       0      0 127.0.0.1:8081          :::*                    LISTEN      13050/GerritCodeRev
tcp6       0      0 :::29418                :::*                    LISTEN      13050/GerritCodeRev

Configure Apache Proxy

Create gerrit config /etc/apache2/sites-enabled/gerrit.conf, /r/ ith the same URI as we configured in gerrit config

Listen 192.168.56.102:8081
<VirtualHost 192.168.56.102:8081>
      ServerName review.example.com

      ProxyRequests Off
      ProxyVia Off
      ProxyPreserveHost On

      <Proxy *>
        Order deny,allow
        Allow from all
      </Proxy>

      AllowEncodedSlashes On
      ProxyPass /r/ http://127.0.0.1:8081/r/ nocanon
</VirtualHost>

• Enable proxy modules:

sudo a2enmod proxy
sudo a2enmod proxy_balancer
sudo a2enmod proxy_http

• Restart Apache

/etc/init.d/apache2  restart

• Check for open ports (80 port was configured before for phpLdapAdmin)

netstat  -ntpl | grep apache2
tcp        0      0 192.168.56.102:8081     0.0.0.0:*               LISTEN      13217/apache2
tcp6       0      0 :::80                   :::*                    LISTEN      13217/apache2

Configure gerrit

Gerrit was installed and next step is configure Gerrit for CI/CD
For CI/CD we need the following features

• Configure Admin User
• Configure "Verify" Label (+1 .. -1 )
• Create users for developers and configure permissions
• Configure projects

Configure Admin User

First logged-in user becomes admin in Gerrit.

• Open Gerit URL: http://192.168.56.102:8081/r/
• Go to Sign In link

• Sign In with credentials in LDAP (user: mmaxur, password: r00tme)

• Go to Settings page

• See group list, user mmaxur is Administrator.

• Add SSH key for Admin User
  • Generate new key with ssh-keygen
  • Or use existing public key

Exising key:

# cat ~/.ssh/id_rsa_local.pub

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5EIrDnnxNBQavZcxiaHCzt0tjtfW0nNuFAz9f+fs4dL0/3wTbDCWO1l2tahTlupM8r
<SKIP>
WJZdHjFaBJvg1k4zo+WzSA2YtOgFxI0CWHUTIcjLD6d3np534zONNxjxsrUz5MBROPUQYOT9y3m9RDBXJVhdvk7V7lTzFYsrTrsJy+gu0pTCL root@mmaxur-pc

Check ssh connection

Connect to gerrit using ssh and your key:

# ssh mmaxur@192.168.56.102 -p 29418 gerrit stream-events

Connection should be opened, and wait for data. We have no active committers yet so stream is empty.

Configure "Verify" Label

project.config is configurable in a little bit tricky way.

• Clone All-Projects metaproject.

# mkdir tmp
# cd tmp
# git init
# git remote add origin ssh://mmaxur@192.168.56.102:29418/All-Projects
# git fetch origin refs/meta/config:refs/remotes/origin/meta/config
# git checkout meta/config

# git config --global user.email "mmaxur@mirantis.com"
# git config --global user.name "Max Mazur"

git commit -a
git push origin meta/config:meta/config

As you can see project config have no lable section

Create root project

"Root" project is project which contains no data and used only as container for permissions settings.
All other projects we will use will inherit permissions from this project.

• Create LDAP group
• Create LDAP users
• Add users to group
• Create Project
• Add all permissions to group

LDAP settins

Create 3 users (files user1, user2, user3):

• user1:

dn: cn=User1 User1,ou=cicd,dc=demo
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: user1 user1
ou: cicd
uid: user1
givenName: user1
sn: user1
userPassword: {SSHA}5oLdx/TJdGrRb3Jaz/9JWuFsj59pPoPt
mail: user1@user1.com

• user2

dn: cn=User2 User2,ou=cicd,dc=demo
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: user2 user2
ou: cicd
uid: user2
givenName: user2
sn: user2
userPassword: {SSHA}5oLdx/TJdGrRb3Jaz/9JWuFsj59pPoPt
mail: user2@user2.com

• user3

dn: cn=User3 User3,ou=cicd,dc=demo
changetype: add
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
cn: user3 user3
ou: cicd
uid: user3
givenName: user3
sn: user3
userPassword: {SSHA}5oLdx/TJdGrRb3Jaz/9JWuFsj59pPoPt
mail: user3@user3.com

• Add users to LDAP:

# ldapmodify < user1
adding new entry "cn=User1 User1,ou=cicd,dc=demo"

# ldapmodify < user2
adding new entry "cn=User2 User2,ou=cicd,dc=demo"

# ldapmodify < user3
adding new entry "cn=User3 User3,ou=cicd,dc=demo"

• Add users to cicd administrators group

• file add_user_to_group:

dn: cn=cicd  administrators,ou=cicd,dc=demo
changetype: modify
add: memberUid
memberUid: mmaxur
memberUid: user1
memberUid: user2
memberUid: user3

ldapmodify < add_user_to_group
modifying entry "cn=cicd  administrators,ou=cicd,dc=demo"

• Check:

Gerrit Settings

Next Step is Gerrit and Jenkins and JJB integration

Links

• https://gerritcodereview-plugins.storage.googleapis.com/index.html
• gerritcodereview-plugins/plugins/master/download-commands/download-commands.jar
• http://deb.gerritforge.com/dists/gerrit/contrib/binary-amd64/gerrit-2.11.8-1.noarch.deb
• http://blog.bruin.sg/2013/04/how-to-edit-the-project-config-for-all-projects-in-gerrit/
• http://stackoverflow.com/questions/22229536/edit-project-config-in-a-gerrit-project
• https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_Verified
• https://gerrit-review.googlesource.com/Documentation/config-project-config.html
• http://stackoverflow.com/questions/25478344/adding-gerrit-label-but-not-able-to-see-it-properly-in-review-page
• https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_Verified

Other

root@jenkins-demo:~/demo-jenkins-jobs/demo/builders/test-jenkins-jobs# ssh jenkins-demo@192.168.59.103 -p 29418 gerrit stream-events

root@jenkins-demo:~/demo-jenkins-jobs# ssh jenkins-demo@192.168.59.103 -p 29418 'gerrit review 3,4 --message "Build 3 Started nulljob/test-jenkins-jobs/11/ " --label "Verified=0" --code-review 0'

• http://stackoverflow.com/questions/22229536/edit-project-config-in-a-gerrit-project
• https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_Verified
• https://gerrit-review.googlesource.com/Documentation/config-project-config.html
• http://stackoverflow.com/questions/25478344/adding-gerrit-label-but-not-able-to-see-it-properly-in-review-page
• https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_Verified