Building GitHub projects on Jenkins slaves on OpenShift

This post showed how to build GitHub projects with Jenkins, Maven and SonarQube 4 on OpenShift. For starters, it used the Jenkins master node for running build jobs. However, when running on a small gear, the master node might run out of memory pretty fast, resulting in a reboot of the node during builds.

In order to resolve this issue, there are two options:

  • limitting the memory of the build or
  • running the build on a slave node.

As spawning additional nodes is easy in a PaaS context such as OpenShift and provides a better performance than running builds with small memory, the slave solution seems to be the better approach.

This post shows how.

  1. Create new DYI app as a slave node (a how-to can be found here), name the node e.g. slave
  2. Create node in Jenkins
    1. Go to Jenkins web UI and create new node:
      https://jenkins-<yourAccount>.rhcloud.com/computer/new
    2. Set the following values:
      Remote FS root:/app-root/data folder on slave. Typically this is /var/lib/openshift/<slave's UID>/app-root/data/jenkins, you can find out by SSHing to the slave node and calling

      echo $OPENSHIFT_DATA_DIR/app-root/data/jenkins

      Labels: Some label to use within builds to refer the node, e.g. OS Slave #1
      Host: the slave’s hostname, e.g. slave-<youraccount>.rhcloud.com

    3. Add Credentials
      username: <slave's UID>
      Private Key File:Path to a private key file that is authorized for your OpenShift account. In the first post this path was used: /var/lib/openshift/<UID of your jenkins>/app-root/data/git-ssh/id_rsa. Note: $OPENSHIFT_DATA_DIR seems not to work here.
      BTW: You can change the credentials any time later via this URL

      https://jenkins-<yourAccount>.rhcloud.com/credentials/
  3. Prepare slave node: Create same environment as on master in the first post
    1. Create folder structure
      mkdir $OPENSHIFT_DATA_DIR/jenkins
      mkdir $OPENSHIFT_DATA_DIR/.m2
      echo -e "&lt;settings&gt;&lt;localRepository&gt;$OPENSHIFT_DATA_DIR/.m2&lt;/localRepository&gt;&lt;/settings&gt;" &gt; $OPENSHIFT_DATA_DIR/.m2/settings.xml
      
    2. Copy SSH directory from master to same directory on slave, e.g.
      scp -rp -i $OPENSHIFT_DATA_DIR/.ssh $OPENSHIFT_DATA_DIR/.ssh &lt;slave's UID&gt;@slave-&lt;your account&gt;.rhcloud.com:app-root/data/.ssh
    3. As the different cartridges (jenkins and DIY) have different environment variables for their local IP addresses ($OPENSHIFT_JENKINS_IP vs $OPENSHIFT_DIY_IP) we’ll have to improvise at this point. There are two options: Either
      1. Replace all occurrences of $OPENSHIFT_JENKINS_IP
        In all builds and in

        https://jenkins-<yourAccount>.rhcloud.com/configure

        Sonar | Sonar installations
        Database URL: jdbc:postgresql://$OPENSHIFT_DIY_IP:15555/sonarqube
        or

      2. Create an $OPENSHIFT_JENKINS_IP environment variable on your slave machine
        rhc env set OPENSHIFT_JENKINS_IP=&lt;value of  $OPENSHIFT_DIY_IP&gt; -a slave

        You can find out the value of $OPENSHIFT_DIY_IP by SSHing to the slave and execute

        echo $OPENSHIFT_DIY_IP 
      3. I’d love to hear suggesstions that do better 😉
  4. Adapt Build
    Easiest way is to not use the master at all.
    To do so, go to

    https://jenkins-<yourAccount>.rhcloud.com/configure

    and set # of executors to 0.
    Hit Apply

  5. Limit memory usage.
    Make sure the slave does not run out of memory (which leads to a restart of the node):
    Global properties | Environment variables
    name: MAVEN_OPTS
    value: -Xmx512m
    Hit Save.
  6. Now run a build. It should run on the slave and hopefully succeed 🙂

See also Libor Krzyzanek’s Blog: Jenkins on Openshift wi… | JBoss Developer