Setting up Sonarqube using Docker and integrating in Jenkins

This blog post will show how to setup Sonarqube based on Docker. The system consists of the Sonarqube server and a Postgres database. Subsequently, it is shown how to integrate Sonarqube in the Jenkins build.

First of all, we need to pull both the official Docker images for Sonarqube and Postgres. Additionally, we need the ubuntu image as this image will be used to create a volume container.

docker pull sonarqube
docker pull postgres
docker pull ubuntu

As we want to keep database data over the lifecycle of the Postgres container, we have to options. Either we create a volume container or a bind-mount volume. Volume containers are more portable and this is the reason why we are taking this option here. Creating a volume container for Postgres data is done as follows:

docker run -itd --name vc-sonarqube-postgres -v /var/lib/postgresql/data ubuntu

Next, the Postgres container can be created with the volume container as input. Additionally, the arbitrary port 5555 is exposed. This is needed in order to connect via PgAdmin to the database.

docker run --name sonarqube-postgres --volumes-from vc-sonarqube-postgres -p 5555:5432 -e POSTGRES_PASSWORD=mysecretpassword -d postgres

Once, both containers are started and running, a connection to the database can be established via psql:

docker exec -it -u postgres sonarqube-postgres psql

By pressing \q and Enter, psql can be exited again.

A connection via PgAdmin can be established as follows:

psotgres_connection

Once connected, we need to manually create a database called sonar. Next, we need to run a docker inspect sonarqube-postgres to get the IP of this container. At last, the Sonarqube image can be started with the corresponding Postgres connection information:

docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 -e SONARQUBE_JDBC_USERNAME=postgres -e SONARQUBE_JDBC_PASSWORD=mysecretpassword -e SONARQUBE_JDBC_URL=jdbc:postgresql://172.17.0.3/sonar sonarqube

By starting Sonarqube, the schema with all tables is automatically created. Sonarqube can then be accessed via

http://locahost:9000

That’s all concerning the Docker part to setup a Sonarqube system. All this single steps could be consolidated in a docker-compose.yml. This would be even more convenient. The whole system could then be started with a single command, namely docker-compose up -d.

Next, we are going to look how to configure Jenkins in order to work with Sonarqube. First, we need to install the Sonarqube plugin. Once done, the Sonarqube server has to be configured via Manage Jenkins/Configure System . Get the Sonarqube Docker IP again via docker inspect sonarqube.

sonarqube_server

Subsequently, a build step for the Sonarqube scanner in the corresponding Jenkins job, can be configured. Note that there are some required analysis properties.

sonarqube_build

That’s all. Whenever the build job is started, a Sonarqube analysis is conducted. The results are saved in the Postgres database and displayed on the Sonarqube server.

Understanding Docker volumes

Docker volumes was one of the last things I understood of Docker. As you can get started with Docker quite conveniently without using any volumes I never took the time to really understand them. But using volumes gives you the opportunity to work with data in Docker. Primarily, volumes offer persistent storage and shared state. So it’s high time to have a look at Docker volumes.

Volumes are used to persist data like database files, log files or any other container-independent data outside of a Docker container. Volumes map directories in a container to a defined directory on the host.

There exist two types of Docker volumes, bind-mount volume and managed volume. The former maps any user-specified directory or file on the host operating system. The latter uses internal Docker locations that are managed by Docker itself.

A bind-mount volume is created via the v-flag in the docker run command as follows:

docker run -it -v ~/shared:/tmp/data ubuntu /bin/bash

This will map the /tmp/data directory inside the container to the shared directory inside the local users directory on the host (~ is the shortcut for the local users directory in windows). All locations must be specified with absolute paths. This can be checked via:

docker inspect --format='{{json .Mounts}}'

Once the container is started, you can navigate to the /tmp/data directory and all files contained in the ~/shared directory on the host will be displayed. In the same way, single files can be mapped as well. Read-only volumes can be specified by appending :ro to the docker volume.

Bind-mount volumes have the disadvantage that they make the container not portable to any other host and volumes of different containers can conflict with each other. Thus, its best to avoid these volumes in generalized platforms and use Docker-managed volumes instead. Managed volumes are created via the v-flag as well, but without specifying a directory inside the container:

docker run -it -v ~/shared: /bin/bash

Besides persistent storage, volumes are used to share data between different containers. This can be achieved in two ways. Either each container has a bind-mount volume to the same location or managed volumes are shared between them. The former is quite straightforward whereas for the latter the —–volumes-from flag is used. This flag is used to copy a managed volume by a so called volume container into each container. A volume container is a single container that only defines a volume. This looks as follows:

docker run -d --volumes-from vc_shared ubuntu

Docker can’t delete bind-mount volumes because this is outside of its scope. However, care must be taken to always remove managed volumed when a container is deleted. This is done via the v-flag:

docker rm -v ubuntu

Using this v-flag option is really important. If deleting a container without this option, the managed volume becomes orphan and subsequently requires any manual steps to be removed.

Docker in Docker for Jenkins

If  Jenkins is running inside a Docker container and the CI build is setup to create Docker images, you have to find a way how to use Docker inside Docker. Indeed, there exists such a way to run Docker-in-Docker as described here. However, the primary purpose of this mechanism was the help with development of Docker itself. Although this Docker-in-Docker mechanism generates many problems which are listed in this blog, it is often abused to run a CI inside a container that creates Docker images.

A better way to miles is to run Docker inside Docker by bind-mounting the Docker socket of the Docker host into the Docker Jenkins container. This can be achieved by installing Docker binaries into the Jenkins container and then mapping the Docker socket as volume from the Docker host to the Jenkins container.

First of all, we need to have a Jenkins Docker image. Therefore, I created the Dockerfile shown below. It comprises a couple of useful tools and Jenkins itself. Furthermore, it installs the Docker binaries, docker-compose and docker-machine. The latter two are not really needed here, but I added them for completeness.


FROM renewinkler/ubuntu-oraclejdk:8
ENV DOCKER_VERSION 1.12.0
ENV DOCKER_COMPOSE_VERSION 1.8.0
ENV DOCKER_MACHINE_VERSION 0.8.0
# tools
RUN apt-get update -qq && apt-get install -qq curl wget git subversion nano nodejs npm iputils-ping && apt-get clean
# Maven
RUN curl -sf -o /opt/apache-maven-bin.tar.gz http://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz; \
tar xzf /opt/apache-maven-bin.tar.gz -C /opt/; \
rm /opt/apache-maven-bin.tar.gz; \
ln -s /opt/apache-maven-3.3.9 /opt/maven
ENV MAVEN_HOME /opt/maven
#Docker bins
WORKDIR /home/toolbox/
RUN curl -L -o /tmp/docker-latest.tgz https://get.docker.com/builds/Linux/x86_64/docker-${DOCKER_VERSION}.tgz && \
tar -xvzf /tmp/docker-latest.tgz && \
mv docker/* /usr/bin/
#Docker compose
RUN curl -L https://github.com/docker/compose/releases/download/1.8.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose && \
chmod +x /usr/local/bin/docker-compose
#Docker machine
RUN curl -L https://github.com/docker/machine/releases/download/v${DOCKER_MACHINE_VERSION}/docker-machine-`uname -s`-`uname -m` > /usr/local/bin/docker-machine && \
chmod +x /usr/local/bin/docker-machine
# Jenkins
ENV JENKINS_HOME /opt/jenkins
ENV JENKINS_MIRROR http://mirrors.jenkins-ci.org
RUN mkdir -p $JENKINS_HOME
RUN curl -sf -o /opt/jenkins/jenkins.war -L $JENKINS_MIRROR/war/latest/jenkins.war
RUN mkdir -p $JENKINS_HOME/plugins; for plugin in greenballs; \
do curl -sf -o $JENKINS_HOME/plugins/${plugin}.hpi -L $JENKINS_MIRROR/plugins/${plugin}/latest/${plugin}.hpi; done
VOLUME $JENKINS_HOME/data
WORKDIR $JENKINS_HOME
EXPOSE 8080
CMD [ "java", "-jar", "jenkins.war" ]

view raw

Dockerfile

hosted with ❤ by GitHub

Once the image is built, it can be started. In this step the Docker socket of the Docker host has to be bind-mounted by the -v flag via /var/run/docker.sock:/var/run/docker.sock. The volume instruction from within a Dockerfile doesn’t allow to do a host mount. We just can do this from a Docker run command. This is the reason why this volume mapping is not done in the Dockerfile directly. Simply put, when you start the container, start it as follows:

docker run -itd —-name jenkins -p 8080:8080 -v /var/run/docker.sock:/var/run/docker.sock renewinkler/jenkins-oraclejdk

That’s basically all. If you enter the container and type e.g. ‚docker images‘ you should see all images of your Docker host.

Lastly, I want to demonstrate a way how to create a Docker image in a Jenkins build. For this purpose, there exist several maven plugins. One of the better is the maven-plugin-docker from Spotify. There also exist the maven Docker plugin of fabric8 which is great too. I configured the former plugin in my maven build as follows:


<plugin>
<groupId>com.spotify</groupId>
<artifactId>docker-maven-plugin</artifactId>
<version>0.4.11</version>
<executions>
<execution>
<id>build-image</id>
<phase>package</phase>
<goals>
<goal>build</goal>
</goals>
</execution>
<execution>
<id>push-image</id>
<phase>deploy</phase>
<goals>
<goal>push</goal>
</goals>
<configuration>
<imageName>registry.example.com/app_name:${project.version}</imageName>
</configuration>
</execution>
</executions>
<configuration>
<imageName>app_name</imageName>
<imageTags>
<imageTag>${project.version}</imageTag>
<imageTag>latest</imageTag>
</imageTags>
<baseImage>java:8</baseImage>
<entryPoint>["java", "-jar", "/${project.build.finalName}.jar"]</entryPoint>
<!– copy the service's jar file from target into the root directory of the image –>
<resources>
<resource>
<targetPath>/</targetPath>
<directory>${project.build.directory}</directory>
<include>${project.build.finalName}.jar</include>
</resource>
</resources>
</configuration>
</plugin>

view raw

pom.xml

hosted with ❤ by GitHub

The plugin is bind to the package phase which builds the image as well as to the deploy phase which pushes the image to the configured registry. The built images is based on the java:8 base image and two tags latest and current project version are created. The generated jar file of my application will be the entrypoint of the image. As soon as the build mvn clean install is finished, a newly generated Docker image should be visible in the local Docker host registry.

Docker shortcuts

After a while working with Docker, you get tired of always typing the relatively long command ‚docker‘ or of remembering more complicated commands. Thus, I was wondering how I can setup some aliases to make me more productive. I found a good references on github from where I was inspired and took or derived my shortcuts. The shortcuts consist of aliases and functions.


# Docker
alias d='docker'
# Get container process
alias dps='docker ps'
# Get process including stopped containers
alias dpsa='docker ps -a'
# Get images
alias di='docker images'
# Start container
dstart() { docker start $1; }
# Stop container
dstop() { docker stop $1; }
# Stop all running containers
dstopa() { docker stop $(docker ps); }
# Remove container
alias drm='docker rm'
# Remove image
alias drmi='docker rmi'
# Removing all stopped containers
drma() { docker rm $(docker ps -a -q); }
# Removing all images
drmia() { docker rmi $(docker images -q); }
# Getting the IP address
alias dip="docker inspect –format '{{ .NetworkSettings.IPAddress }}'"
# Build from dockerfile
db() { docker build -t=$1 .; }
# Enter into a running container with bash
dbash() { docker exec -it $1 /bin/bash; }

view raw

.bash_profile

hosted with ❤ by GitHub

All these shortcuts have to be placed in the .bash_profile file. This file has to be created in the local user’s home if it  does not exist yet. In order to find out which is the user’s home type in cd ~ in the Docker client which will jump to this directory. Whenever the Docker client is launched, the file is loaded and all commands are executed. After that, all these shortcuts are available in the Docker client.

Creating and running a Docker Jenkins image

In this blog post, I’m going to quickly summarize how to create a Docker Jenkins image and how to make it working. I’m going to demonstrate how to create two images, the first with OpenJDK 8 and the second with Oracle JDK 8 installed. So let’s get started.

The following Dockerfile creates a Docker Jenkins image based on the OpenJDK image. First, it updates existing packages, installs curl, git and nano as a text editor. Next, Maven and Jenkins are downloaded and installed. The only Jenkins plugin that is installed, is the greenball plugin. All other required plugins are easier to install via the Jenkins GUI console as this will also resolve plugin dependencies. Lastly, port 8080 is exposed and Jenkins is run.


FROM openjdk:8
RUN apt-get update -qq && apt-get install -qq curl git nano && apt-get clean
RUN curl -sf -o /opt/apache-maven-bin.tar.gz http://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz; \
tar xzf /opt/apache-maven-bin.tar.gz -C /opt/; \
rm /opt/apache-maven-bin.tar.gz; \
ln -s /opt/apache-maven-3.3.9 /opt/maven
ENV MAVEN_HOME /opt/maven
ENV JENKINS_HOME /opt/jenkins
ENV JENKINS_MIRROR http://mirrors.jenkins-ci.org
RUN mkdir -p $JENKINS_HOME
RUN curl -sf -o /opt/jenkins/jenkins.war -L $JENKINS_MIRROR/war/latest/jenkins.war
RUN mkdir -p $JENKINS_HOME/plugins; for plugin in greenballs; \
do curl -sf -o $JENKINS_HOME/plugins/${plugin}.hpi -L $JENKINS_MIRROR/plugins/${plugin}/latest/${plugin}.hpi; done
VOLUME $JENKINS_HOME/data
WORKDIR $JENKINS_HOME
EXPOSE 8080
CMD [ "java", "-jar", "jenkins.war" ]

view raw

Dockerfile

hosted with ❤ by GitHub

Creating a Oracle JDK image is a bit more costly. First of all, an Ubuntu image with Oracle JDK has to be created. Launchpad offers a Oracle Java installer which automatically downloads and installs Java.


FROM ubuntu:latest
RUN apt-get update -qq && apt-get install -qq software-properties-common && apt-get clean
RUN \
echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections && \
add-apt-repository -y ppa:webupd8team/java && \
apt-get update && \
apt-get install -y oracle-java8-installer && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /var/cache/oracle-jdk8-installer

view raw

Dockerfile

hosted with ❤ by GitHub

In order to indicate that this image contains Oracle JDK 8, it can be tagged (on github) with version 8. This image is the base image for the following Jenkins image. which exactly looks the same as the one with OpenJDK configured.


FROM renewinkler/ubuntu-oraclejdk:8
RUN apt-get update -qq && apt-get install -qq curl git nano && apt-get clean
RUN curl -sf -o /opt/apache-maven-bin.tar.gz http://archive.apache.org/dist/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz; \
tar xzf /opt/apache-maven-bin.tar.gz -C /opt/; \
rm /opt/apache-maven-bin.tar.gz; \
ln -s /opt/apache-maven-3.3.9 /opt/maven
ENV MAVEN_HOME /opt/maven
ENV JENKINS_HOME /opt/jenkins
ENV JENKINS_MIRROR http://mirrors.jenkins-ci.org
RUN mkdir -p $JENKINS_HOME
RUN curl -sf -o /opt/jenkins/jenkins.war -L $JENKINS_MIRROR/war/latest/jenkins.war
RUN mkdir -p $JENKINS_HOME/plugins; for plugin in greenballs; \
do curl -sf -o $JENKINS_HOME/plugins/${plugin}.hpi -L $JENKINS_MIRROR/plugins/${plugin}/latest/${plugin}.hpi; done
VOLUME $JENKINS_HOME/data
WORKDIR $JENKINS_HOME
EXPOSE 8080
CMD [ "java", "-jar", "jenkins.war" ]

view raw

Dockerfile

hosted with ❤ by GitHub

Of course, it would have been possible to configure this image with only one Dockerfile. However, that would have less corresponded to the modular idea. In this way, one is flexible and the base image can be reused.

The most eleagnt way to create Docker images out of dockerfiles is to configure an automated build on Dockerhub. I’m not going in these details here, but I show you how to build the image manually. The docker image can be build

docker build . -t jenkins-oraclejdk

and then run as follows

docker run -itd -p 8080:8080 --name jenkins jenkins-oraclejdk

Next, open a web browser and type in http://192.168.99.100:8080/ Once Jenkins is started, an input will appear to create a first user:

firstuser

Next, Jenkins has to be unlocked.

unlockJenkins.png

In order to this, one have to connect to the docker container and get the initial admin password. The following docker command, let you connect to the container.

docker exec -it jenkins /bin/bash

Navigate to the corresponding folder and open the intialAdminPassword file with nano.

nano intialAdminPassword

Double click the password and press enter to copy the password and leave the editor via Ctrl+X. Exit the container by typing in ‚exit‘ and paste the password to the corresponding input field to unlock Jenkins. That’s all. At last, you have the opportunity to customize Jenkins, e.g. by installing additional plugins.