its-jira is a Gerrit plugin to automatically interact with Jira when actions are taken in Gerrit. It is one of many plugins in a family that essentially does the same but for different issue ticket systems (ITS). There are similar plugins for Bugzilla, Github, IBM RTC, Storyboard etc. Basic actions include:
- Commenting on tickets/stories when new patchsets have been uploaded
- Moving tickets/stories to particular states when patchsets get reviewed/approved/merged etc.
This post will guide you on how to set up its-jira and make it play nicely with both Gerrit and Jira.
Pitfalls (of doom) and handy advice
During working with setting up its-jira for the internal Scionova project ScIoT I found many pitfalls in which you could fall rather deep and spend an enormous time trying to solve issues that could have easily been solved in another way. This section tries to document these pitfalls and give some handy advice on things that will speed up your process.
Administrative rights
Make sure to get administrative rights to Gerrit before starting to work with this, it helps a lot. This may sound basic, but it is actually much more important than you might think. As an example; I tried to do a manual install of its-jira via the Gerrit SSH daemon for a long while until we got administrative rights and realized the Gerrit dev team actually had implemented a plugin handler in the WEB UI which was way much easier. It just wasn’t visible for non-administrators (for obvious reasons). So make sure you have permissions until you start poking around.
The docker daemon needs to run with elevated privileges on your host machine to carry out some of its features and hence we need to access it with elevated privileges as well. There are two approaches here, either issue commands for docker as root or create a group called docker in which your user is part of. I prefer the first solution if possible just due to its simplicity and will extensively use sudo with docker in this post. However, if you feel like using the docker group solution there is some more info on the topic here. Either way, make sure you have permission to poke around before you start.
Persistent storage in docker
Docker is a great technology, but sometimes you can forget to make your data persistent and all work goes poof. Make sure that all files you alter are persistent before modifying them. See more info in the upcoming sections.
Editing files in docker
Docker containers are generally not made to poke around in and hence they rarely include editors by default. You could of course install an editor inside the docker container or mount directories whenever you want to edit something but that sometimes feels like overkill. Instead, I use a home-written script to download the file from the docker container, open an editor and then sync it back again. That way I can reuse it for all docker containers without the need to mount/unmount or install things. The (bash) script goes as follows:
#!/bin/bash
container=$1 # Container name or ID
path=$2 # Path to file in docker
uid=${3:-1000} # Default UID 1000
gid=${4:-1000} # Default GID 1000
tmpfile=$(tempfile) # Get temp file name
# Print information (debug reasons, optional)
echo "Editing docker file:"
echo " Container : $1"
echo " File : $2"
echo " Owner : $3:$4"
echo " Using tempfile $tmpfile"
docker cp $1:$2 $tmpfile # Fetch file from docker
chown $3:$4 $tmpfile # Set correct user rights
nano $tmpfile # Edit file (I use nano editor)
docker cp $tmpfile $1:$2 # Copy file back to docker
rm $tmpfile # Clean up temp file
One handy thing with this is that you can now easily create an alias function to provide some of the common parameters (e.g. container name). In my case I added the following to my .bash_aliases:
function editgerritfile(){
sudo ~/edit_docker.sh dockergerrit_scio_gerrit_1 /var/gerrit/$1 1000 1000
}
This makes it very simple to work with files in docker. To edit the main gerrit config I just need to run
editgerritfile etc/gerrit.config
Installation of its-jira
This guide will assume you are doing the install for Gerrit in a docker container. If this is not the case, just skip the first section about docker.
Prepping docker
This section assumes your docker installation is similar to the official one with docker-compose. This guide makes sure to keep the folders etc, git and db persistent but does not do the same with the plugins folder. The process for fixing this is threefold.
1. Copy all default plugins to your persistent storage folder
Gerrit comes preinstalled with a bunch of rather essential plugins. Before making any changes to our docker-compose file we need to sync those plugin *.jar files to our machine. Do this by running the following command on the host machine (while your docker container is up and running):
sudo docker cp docker-name:/var/gerrit/plugins /your-persistent-storage/plugins
In the case of ScIoT we had our persistent storage in the /external/gerrit/ folder, so we got:
sudo docker cp dockergerrit_scio_gerrit_1:/var/gerrit/plugins /external/gerrit/plugins
2. Make sure permissions are correct
Nice, now we have the files on the host machine. But what about permissions? Docker will not preserve file ownership when copying between container and host machine, instead files are created with root as the owner on the host machine. This is rather problematic due to the fact that the persistent storage functionality tries to preserve ownership rights and will hence use the root user when we sync it back. If we set up persistent storage now and restart our docker container now the gerrit process will get an access denied if it tries to access the plugins folder. So how do we solve this? Inside the docker container the gerrit user owns the plugins folder. In our docker container the gerrit user had UID 1000 and GID 1000 (see Linux UIDs if you are uncertain on UIDs) and hence we want to set the owner of our plugins folder and all of its files on the host machine to the same. In our case this yielded the command
sudo chown -R 1000:1000 /external/gerrit/plugins
Note that the host machine not necessarily needs to have any user that matches the UID/GID combination.
3. Update you docker-compose config
Now we only have to update the docker-compose file to make sure plugin files are persistent. Update the volumes section with the line marked with an arrow. If you used another directory than /external/gerrit/ for persistent storage, make sure to use your path instead.
....
volumes:
- /external/gerrit/etc:/var/gerrit/etc
- /external/gerrit/git:/var/gerrit/git
- /external/gerrit/db:/var/gerrit/db
- /external/gerrit/index:/var/gerrit/index
- /external/gerrit/cache:/var/gerrit/cache
- /external/gerrit/plugins:/var/gerrit/plugins <-----------
....
4. Restart your docker container
Restart your docker container. Everything should start up and get running as before, with the only exception that plugins are now being synced persistently. Make sure you do not get any errors here before proceeding the tutorial.
Security concerns
Before moving on I would like to just say a few words about security here. In Linux you can set read/write/execute rights on a file to a user ID for a user that does not exist. This is very useful, especially in cases like this where the owner ID of a file is the same across two systems. However, this also opens up for security issues.
First off, assume the user ID used in the docker container does not exist on the host system. We still set the file rights and everything works, right? The problem here is that if an attacker can create an account with that specific user ID, he/she has full access to plant new (potentially malicious) plugins in our plugins folder.
Let us also assume the other case; the user ID user in the docker container does exist on the host system. If the attacker can gain access to this account somehow, we end up in the same scenario as in the last paragraph. Hence the importance of strong passwords and firm security measurements/routines for that specific account automatically increases.
Unfortunately docker-compose has, as far as I am aware, no easy way to specify the access rights of the mounted folder, but there exists workarounds. The best solution I have seen is to write a custom script that runs chown inside the docker container itself and lets you keep root as the owner to the plugins folder on the host machine.
Remember to always think of security when doing these small hacks and weigh the ease of the solution against risks of potential attacks. But let us get on with the its-jira installation.
Prepping Jira user
Before we start to configure Gerrit we should take a look at the Jira side and set up some authentication parts for its-jira. Follow these steps:
- Choose what Jira user to use (do either a or b)
- Create a new user (recommended)
- Reuse an existing one (remember that Jira Cloud free only have 10 user slots)
- Log in as the user from step 1
- Generate API token
- Go to Atlassian API Token generator page
- Press Create API Token
- Give the token a name, e.g. its-jira
- Copy the token and make sure to keep it. We will use it soon!
Update Gerrit configs
This section will assume your Gerrit installation is located at /var/gerrit/. If it doesn’t, make sure to keep that in mind. Furthermore, some parts also tell you to edit files. If you are running Gerrit in a docker container and are not used to edit files in docker (which usually comes without any editor installed), please see the previous section Editing files in docker if you skipped it.
Add authentication
First off, we need to add user settings to access Jira. Use the email you usually enter when logging into Jira and use the token generated in the previous section as value for password (see code below).
Edit /var/gerrit/etc/secure.config and add the following lines:
[its-jira]
url = https://your-subdomain.atlassian.net/
username = your-email
password = your-jira-api-token # (generated in previous section)
Add basic config params
We need to configure how its-jira should match for ticket names and access their update pages. Ticket names are assumed to be included in your commit messages in order for its-jira to function properly. Make sure to update below markup with your subdomain and preferred association level before adding it to the configuration file.
Edit /var/gerrit/etc/gerrit.config and add the following lines:
[commentLink "its-jira"]
match = ([A-Z]+-[1-9][0-9]*)
html = "<a href=\"https://your-subdomain.atlassian.net/browse/$1\">$1</a>"
association = SUGGESTED
Available values for the association entry are:
- MANDATORY : blocks commits without an included ticket name
- SUGGESTED : shows a warning for commits without an included ticket name
- OPTIONAL : does not interact with tickets without an included ticket name
Configure actions
its-jira needs to know what you want it to do when something happens. This is where actions come into place. Your actions configuration file should be put in the directory /var/gerrit/etc/its/ (create if it does not already exist) and be called either actions.config or actions-its-jira.config. Settings in actions.config will be shared among all ITS-plugins whilst actions-its-jira.config will be exclusive to Jira. If you do not plan to add integrations to multiple issue tracking systems, I would recommend using actions.config out of simplicity.
The syntax for actions markup is well documented by the parent plugin its-base here. I would recommend reading that page before constructing your actions configuration, it is very efficiently structured and gives you a good overview of what is possible with the its plugin family.
As an example of an action config I post our current configuration here (WIP). As you can see we both comment on issues and move them to different states depending on the received Gerrit event.
[rule "open"]
event-type = patchset-created
action = add-standard-comment
action = In Progress
[rule "resolve"]
event-type = comment-added
approvalCodeReview = 2
action = add-comment An approval was given for this issue
action = In Review
[rule "merged"]
event-type = change-merged
action = add-standard-comment
action = Done
[rule "abandoned"]
event-type = change-abandoned
action = add-standard-comment
action = To Do
[rule "fixneeded"]
event-type = comment-added
approvalCodeReview = -2,-1
action = add-comment A negative code review was received
[rule "positivereview"]
event-type = comment-added
approvalCodeReview = 1
action = add-comment A +1 code review was added. Keep up the good work!
[rule "verifiedsuccess"]
event-type = comment-added
approvalVerified = 1
action = add-comment Latest patchset was Verified +1
[rule "verifiedfail"]
event-type = comment-added
approvalVerified = -1
action = add-comment Latest patchset failed the verification
NOTE: add-standard-comment can be an efficient way to check that something works at all. However, this action is only defined for patchset-created, change-merged, change-abandoned, and change-restored.
Enable its-jira to run
If you haven’t worked with repositories on this Gerrit server before, I would recommend to setup an SSH key with your account before proceeding. Generate an SSH key on your local machine and enter the public key on the following page http://YOUR-GERRIT-SERVER/settings/#SSHKeys. Replace YOUR-GERRIT-SERVER with the IP/domain of your Gerrit server. Your SSH username can be found at the top of your profile settings page on Gerrit. Now you should be able to push/pull/fetch etc. using SSH! (read on for more info)
Enabling its-jira requires doing changes to the project configuration file in the chosen project repository. We have two options:
- Enable it for all projects (apply changes to All-Projects repository)
- Enable it for a specific project (apply changes to the repository of that project)
First off, pull your selected repository and checkout the branch refs/meta/config. If something fails in this step, make sure you actually have permission to do this (requires Administrator or Project Owner in the default settings).
git clone "ssh://username@YOUR-GERRIT-SERVER:29418/PROJECT-NAME" cd PROJECT-NAME git checkout refs/meta/config
Now you should see a file called project.config in your repository base directory. Edit it and add the following lines
[plugin "its-jira"] enabled = true
If you want to limit its-jira to specific branches only you can do this as well. Multiple branches can be specified with either explicit names or regex expressions. This is a basic example of how it might look if you want to run it on the master branch and all branches whose name starts with stable-.
[plugin "its-jira"]
enabled = true
branch = refs/heads/master
branch = ^refs/heads/stable-.*
Once you are done with editing, it is time to commit your changes. If you feel like you really need someone to review the changes, do a normal push. If you feel like this is not necessary, and it is not too much activity on the branch yet, force pushing may be a good option. I spent a good long while trying to get around that we required the Verified flag on all pushes, but no automatic verification process was run on the /refs/meta/config branch (due to reasonable reasons). Either way, make sure your work ends up pushed.
Install plugin
Phew, now the configuration should be done. Make sure you completed all steps before doing the installation, or the install will simply fail (and you will have to start troubleshooting).
Gerrit has a nice simple UI for installing and updating plugins. Note that this retrieves the latest version of the plugin, which may not be what you want if you run an older version of Gerrit. I would recommend to use the Gerrit UI firsthand and then fall back to a manual install if you get errors on git pushes or adding reviews.
Install from Gerrit UI (latest version)
Log in as an administrator account and then navigate to Plugins > Manage from the top menu. On this page, scroll down to its-base and install it. This is a dependency for its-jira so make sure you install it in the correct order. Once its-base is installed, press install for its-jira. If you have succeeded in updating the configurations this should be without errors as well.
Manual install
Open a terminal into your docker installation of Gerrit
sudo docker exec -it CONTAINER-NAME /bin/bash
Move into the /var/gerrit/plugins/ folder and download the correct version of its-jira.jar and its-base.jar matching your Gerrit version from here. For example, we were running Gerrit version 3.1.2 and hence I downloaded the artifacts from plugin-its-base-bazel-stable-3.1 and plugin-its-jira-bazel-stable-3.1. The commands for this version would be as follows
cd /var/gerrit/plugins wget https://gerrit-ci.gerritforge.com/job/plugin-its-base-bazel-stable-3.1/lastSuccessfulBuild/artifact/bazel-bin/plugins/its-base/its-base.jar wget https://gerrit-ci.gerritforge.com/job/plugin-its-jira-bazel-stable-3.1/lastSuccessfulBuild/artifact/bazel-bin/plugins/its-jira/its-jira.jar
Now either restart your docker container or restart Gerrit inside of the docker container to reload the plugins. Make sure to check startup logs to see that the its plugins load correctly. The correct output from its-jira (at least for stable version 3.1) looks like this:
....
INFO com.googlesource.gerrit.plugins.its.jira.JiraModule : JIRA is configured as ITS
INFO com.googlesource.gerrit.plugins.its.jira.JiraItsStartupHealthcheck : Connection to Jira (https://yourdomain.atlassian.net/) succeeded: {"status"="ok","system"="Jira","version"="1001.0.0-SNAPSHOT","ur ...
INFO com.google.gerrit.server.plugins.PluginLoader : Loaded plugin its-jira, version e67eeeea39
....
Test it out!
Now is time to test your installation. Create a ticket/story and an accompanying branch and make sure everything works as you expect it too. Add patchsets and reviews to see that its-jira can react to the changes. If it doesn’t work as it should, go check out your gerrit logs. The logs are often rather verbose on what went wrong and helped me more times than I dare to admit.
Does it work now? Nice! ?
Helpful resources
ITS-JIRA official documentation
ITS-BASE configuration documentation
Making Gerrit play nice with Jira (2018)
Matz Larsson, Software Developer
