Android Logging for Java Professionals – SLF4J and Logback in Android

Posted by schnatterer on September 9, 2017

One of my articles was published in Java Magazin 9.17. I wrote it while working on the nusic android app, about how to use SLF4J in Android using logback-android. It also features an example and a small library for android.

Triology GmbH provides an English version of this article, and also acquired the original article PDF (in German), which can be found here: Android Logging für Java-Profis – SLF4J und Logback in Android. I’d like to thank my colleagues there for their support.

Android 7 (Nougat) on a 5-year-old phone

Posted by schnatterer on June 10, 2017

A 5-year-old phone is not going to the kindergarden, it is more likely to be found on the phone graveryard. One important reason for this is that its manufacturer stopped caring for it at least three years ago. Fortunately, there are other caretakers – people that maintain the latest android version also for five-year-old phone grannies – in form of custom ROMs.
So, is it possible to run the latest Android version on a device that’s 5 years old? Hell, yes! This article shows how.

My old HTC One S (aka ville, the older S4 processor not the “C2” with S3 processor), launched April 2012 runs terribly laggy on its latest stock anroid 4 firmware, last update 2013 – four years ago! That’s why it was retired a long time ago. At the beginning of 2017, when the first unofficial builds of lineage OS turned up, I was surprised to see that HTC One S was among them. I was wondering, how can such an old device run the latest android version? I had to see for myself.

TLDR; Android Nougat works surprisingly well, way better than the latest Stock version but HTC makes it a hell of a way to get there. Lets see what has to be done.

Note that all files on your device will be deleted during the process!

Unlock Bootloader

To be able to flash software to our HTC devices, HTC forces us create an account on htcdev.com and register our device there. Then we receive an unlock code as zip file which we can flash on our device.


adb reboot bootloader

fastboot flash unlocktoken Unlock_code.bin

For all Windows 8 (and also Windows 10, apparently) users who have problems getting the fastboot driver to work, whereas ADB works, this little registry edit did the trick for me.

In the bootloader make sure to remember your HBoot version, it will be required during the process.

HBoot 2.16 – we’re good!

Flash TWRP

Download from here to your PC and flash as follows


fastboot flash recovery twrp.img

fastboot reboot

Check HBoot version

I had to find out the hard way by flashing lineage 14.1 that there are requirements regarding the HBoot (HTC Bootloader) version: HBoot 2.16 is required.

So better check in advance. If you’re on HBoot 2.16 flash lineage and enjoy. If not, let the games beginn.

S-OFF & root

Upgrading HBoot needs S-OFF (Security off, enables writing to /system directory). On top of unlocking our bootloader at htcdev.com we need to risk bricking our phone to enalbe writing in the /system directory of our device.
Those two things are the reasons why I most likely won’t ever buy an HTC device again.
Anyway, with the One S, we’re lucky! S-OFF takes a bit of time but not much effort and comes at almost no risk, thanks to rumrunner (a good overview of other S-OFF Mechanisms is described here). Rumrunner makes it easy to get S-OFF, but it needs root access.

What? Why are we doing this again? We want to Upgrade HBoot, that’s why we need S-OFF, in order to get S-OFF, we need root. That’s the last unpleasant surpise, promise!
Lucky again, root access can be gained easily on HTC One S. I used Superboot: Just Download, execute Script on PC while Phone phone is in bootloader mode. Root done. More detailed description here.

Then just download rumrunner follow the instructions, wait for about 10 minutes (don’t worry, your phone is restarting about one million times) and you’re S-OFF!

Upgrade HBoot

We’re getting closer. Now, with S-OFF we can flash HBoot 2.16. I followed these instructions, and found the firmware here.

Caution: After flashing HBoot 2.16, the internal memory is really small, only 50 MB left. Obviously Android 4 and Hboot 2.16 are not good friends. Don’t worry, this is solved once Android 7 is flashed.

Flash Android 7.1.2

It’s finally gonna happen. We’ll flash Lineage 14.1, Android 7.1.2 (maintained by moozon, thanks so much!). But how, the internal memory is only 50MB, but the image is almost 300MB? I copied the image to an USB thumb drive, connected it to the phone via an USB OTG adapter and flashed via TWRP Recovery. TWRP also offers ADB sideloading, which might be an alternative, it you don’t have an USB OTG Adapter at hand.

Once flashed, you finally can enjoy the latest android version on your “antique” phone. 🥂

Optional finishing and more info

Root is built-in but disabled by default. If you want root access, you can enable it in developer options.

If you need google services, get ARM | 7.1 | pico (my recommendation) from opengapps.org and flash via TWRP.

If you need more info, you might get started here: [LINEAGE OS] HTC One S Lineage OS 14.1, Android Nougat 7.1 ROM

Final thoughts

The One S with Android 7.1.2 almost works better than my HTC One M8 with Android 6! Why must HTC make it so hard to get there? And why are manufacturers not able to support their devices longer than 2 years, while the community or even single indivudals are able to do so for more than 5 years??

[EDIT 2017/06/13: Google just published the guaranteed updates for its devices: two years of upgrades and another year of security updates. While this is at least some formal guarantee (other manufacturer just don’t guarantee anything) it still is far from the five years provided by the community for my HTC One S… ]

I’m still waiting for a manufacturer that guarantees support for its devices for many years. That would really be a unique selling point! I can’t believe no manufacturer uses this USP on the highly competitive market for mobile devices. In addition, this would be so much more sustainable, and contribute to a green(er) IT. While the whole world complaints about planned obsolescence, why is there no manufacturer that uses this fact for positive marketing?

[EDIT 2017/07/01: HMD Global (owner of the Nokia brand) just announced that they intend to provide plain vanilla android phones with support “even after two years“. I looking forward to their next anouncements. My calls might just have been heard 👂]

[EDIT 2017/10/05: Google talks about four android version upgrades being possible from Android O and releases Pixel 2 with three years of support. See More sutainable Android Software with Project Treble and 6-y LTS Kernels with Android O?]

What is Google doing, by the way? They introduce Safety Net, providing the oportunity for developers to hide their apps on Google Play from specific devices, such as the ones running custom ROMs. So now, no more Netflix, etc. for custom ROMs. Effectively, this is another punch for the custom ROM scene after the death of CyanogenMod.

Lets just hope custom ROM developers will not be discouraged by these facts and continue their imporant work, making success stories such as the one of my HTC One granny possible.

Thank you android community, XDA forums, custom ROM developers, etc. for sharing your work and never giving up! 👍

Automatic checks for vulnerabilities in Java project dependencies

Posted by schnatterer on January 17, 2017

Java aktuell published an article I wrote on a topic at work for TRIOLOGY GmbH.

You can find an English version on the TRIOLOGY Blog: Automatic checks for vulnerabilities in Java project dependencies. The article shows an approach to keeping your Java project dependencies free of known vulnarabilities (e.g. CVEs) using the OWASP Dependency check with Jenkins and Maven. There also is an example project on GitHub.

The original article PDF (in German) is available for download here: Automatisierte Überprüfung von Sicherheitslücken in Abhängigkeiten von Java-Projekten.

TRIOLOGY also published a short Q&A on the article, which can be found here.

Maven: Create a more sophisticated build number

Posted by schnatterer on November 8, 2016

Earlier this year, while working on a project for TRIOLOGY GmbH, I once again used maven to write a version name into an application, using the mechanism described in my post Maven: Create a simple build number. As a more sophisticated version name was required for this project, we expanded it by a time stamp, SCM information (branch and commit), build number and a also created a special name for releases. You can find a how-to here – Version names with Maven: Creating the version name – which is the first part of a small series of blog posts on this topic.

The second part shows how the version name can be read from within the application. While writing the examples for the post, I wondered how many times I must have implemeted reading a version name from a file in Java. Way too often! So I decided that this would be the very last time I had to do it, and extracted the logic into a small library: versionName, availble on GitHub. What it does and how to use it is described in the second part of the post: Version names with Maven: Reading the version name.

Hopefully, this will be useful for someone else. Funny enough, in the new project I’m on, I’m about to reuse it once again. I’m glad I don’t have to write it again. Here’s to reusability 🍺

Building GitHub projects with Jenkins, Maven and SonarQube 5.2 on OpenShift

Posted by schnatterer on November 17, 2015

Time for an update of the post Building GitHub projects with Jenkins, Maven and SonarQube 4.1.1 on OpenShift, because SonarQube 5.2 is out: It’s the first version since 4.1.1 that can be run on OpenShift. That is, it’s the first version of SonarQube 5 and the first one that contains Elasticsearch and many other features that are now available on OpenShift!
Interested? Then let’s see how to set up SonarQube on OpenShift.

If you’re starting from scratch, just skip to this section.
If you got a running instance of SonarQube
- make sure to back up you instance before you continue:
  rhc snapshot save -a <application name>--filepath <backup destination>
  or
  ssh <UID>@<application name>-<yourAccount>.rhcloud.com 'snapshot' > sonar.tar.gz
- Then pull the git repository just like in step 2,
- wait until the app has started and visit
```
https://sonar-<yourAccount>.rhcloud.com/setup
```
  SonarQube will update it’s database during the process.
- If you followed this post to set up your SonarQube instance and therefore use an SSH tunnel to access the SonarQube database, note that you can now get rid of this workaround. From SonarQube 5.2 the analyses can be run without direct contact to the database.
  That is, you can also remove the database connection from your the configuration of the SonarQube plugin in jenkins.

Install new SonarQube instance

To install SonarQube 5.2, execute the following steps on your machine:

```
rhc app create sonar diy-0.1 postgresql-9.2
```
Make sure to remember the login and passwords!

git rm -r diy .openshift misc README.md
git remote add upstream -m master https://github.com/schnatterer/openshift-sonarqube.git
git pull -s recursive -X theirs upstream master
git push

Login to your SonarQube instance at
```
http://sonar-<yourAccount>.rhcloud.com/
```
Note that the initial setup may take some minutes. So be patient.
The default login and passwords are admin / admin.
You might want to change the password right away!

Basic installation Jenkins

Basically, the following is an updated (and a lot simpler) version of my post about SonarQube 4.1.1.

Create Jenkins app
```
rhc app create jenkins jenkins-1
```
Install Plugins
Browse to Update Center
```
https://jenkins-<yourAccount>.rhcloud.com/pluginManager/advanced
```
and hit Check Now (as described here).
Then go to the Available tab and install
1. Sonar Plugin,
2. GitHub plugin,
3. embeddable-build-status (if you’d like to include those nifty build badges in you README.md).
Then hit Install without restart or Download and install after restart. If necessary, you can restart your app anytime like so
```
rhc app restart -a jenkins
```

Set up maven settings.xml to a writable location.

SSH to Jenkins

mkdir $OPENSHIFT_DATA_DIR/.m2
echo -e "&amp;amp;lt;settings&amp;amp;gt;&amp;amp;lt;localRepository&amp;amp;gt;$OPENSHIFT_DATA_DIR/.m2&amp;amp;lt;/localRepository&amp;amp;gt;&amp;amp;lt;/settings&amp;amp;gt;" &amp;amp;gt; $OPENSHIFT_DATA_DIR/.m2/settings.xml

Browse to Configure System
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
Default settings provider: Settings file in file system
File path=$OPENSHIFT_DATA_DIR/.m2/settings.xml

Either see my post on how to introduce a dedicated slave node to this setup or
set up the Jenkins master to run its own builds as follows (not recommended on small gears, as you might run out of memory pretty fast during builds):
Go to Configure System
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
and set
# of executors: 1
Setup sonar plugin (the following bases on SonarQube Plugin 2.3 for Jenkins)
On the Jenkins frontend, go to Configure System
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
- Global properties,
  tick Environment variables
  Click Add
  name=SONAR_USER_HOME
  value=$OPENSHIFT_DATA_DIR
  See here for more information.
- Setup the Runner:
  Navigate to SonarQube Runner
  Click Add SonarQube Runner
  Name=<be creative>
- Then set up the plugin itself
  Navigate to SonarQube
  tick Enable injection of SonarQube server configuration as build environment variables
  and set the following
  Name=<be creative>
  Server URL:
```
http://sonar-<yourAccount>.rhcloud.com/
```
  Sonar account login: admin
  Sonar account password: <your pw> (default: admin)
- Hit Save

Configure build for a repository

Now lets set up our first build.

Go to
```
https://jenkins-<yourAccount>.rhcloud.com/view/All/newJob
```
Item name: <your Project name>
(Unfortunately, Maven projects do not work due to OpenShift’s restrictions.)
Hit OK

On the next Screen
GitHub project:

https://github.com/<your user>/<your repo>/

Source Code Management:

https://github.com/<your user>/<your repo>.git

Branch Specifier (blank for 'any'): origin/master
Build Triggers: Tick Build when a change is pushed to GitHub
Build Environment: Tick Prepare SonarQube Scanner environment
Build | Execute Shell

cd $WORKSPACE
# Start the actual build
mvn clean package  $SONAR_MAVEN_GOAL --settings $OPENSHIFT_DATA_DIR/.m2/settings.xml -Dsonar.host.url=$SONAR_HOST_URL

I’d also recommend the following actions
Post-build Actions| Add post-build action| Publish JUnit test result report Test report XMLs=target/surefire-reports/TEST-.xml* Post-build Actions| Add post-build action| E-mail Notification Recipients=<your email address>
Hit Apply.
Finally, press Save and start you first build. Check Jenkins console output for errors. If everything succeeds you should see the result of the project’s analysis on SonarQube’s dashboard.

Using Custom Maven / JDK version when building with Jenkins on OpenShift

Posted by schnatterer on September 9, 2015

[EDIT (2016-06-02): OpenShift now provides different JDK “alternatives”, e.g.

/etc/alternatives/java_sdk_1.8.0

So you might want to skip the steps bellow, regarding a custom JDK. The steps described for using a custom maven still apply, however.

]

In previous posts I pointed out how to build GitHub projects with Jenkins, Maven and SonarQube and how to run these builds on dedicated Jenkins slaves. The following shows how to replace the “stock” versions of maven and JDK that are provided by OpenShift.

At the time of writing OpenShift features Maven 3.0.4 and OpenJDK Server 1.7.0_85. Why would you want to change those? Best example is a Java8 project to be build on Jenkins. Can we just advise Jenkins to download the newest Oracle JDK and we’re good to go? Nope, it’s not that simple on OpenShift! Jenkins does download the new JDK, sets the JAVA_HOME variable and the correct PATH, but maven is always going to use the stock JDK. Why? Running this command provides the answer

$ cat `which mvn`
#!/bin/sh
prog=$(basename $0)
export JAVA_HOME=/usr/lib/jvm/java
export JAVACMD=$JAVA_HOME/bin/java
export M2_HOME=/usr/share/java/apache-maven-3.0.4
exec $M2_HOME/bin/$prog &amp;quot;$@&amp;quot;

The stock maven is setting its own environment variables that cannot be overridden by Jenkins!

So, in order to exchange the JDK, we need to exchange maven first.

SSH to the machines where your builds are executed (e.g. your slave node). The following example show what to do for maven 3.3.3:

cd $OPENSHIFT_DATA_DIR
mkdir maven
cd maven
wget http://apache.lauf-forum.at/maven/maven-3/3.3.3/binaries/apache-maven-3.3.3-bin.tar.gz
tar -xvf apache-maven-3.3.3-bin.tar.gz
rm apache-maven-3.3.3-bin.tar.gz

Edit maven config
```
vi $OPENSHIFT_DATA_DIR/maven/apache-maven-3.3.3/conf/settings.xml
```
Add the following to the <settings> tag (replace <UID> by your OpenShift UID first)
```
<localRepository>/var/lib/openshift/<UID>/app-root/data/.m2</localRepository>
```
(press i button for edit mode, insert, then press esc button, enter :wq, finally press return button)
Browse to
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
Set Environment variables
PATH=$OPENSHIFT_DATA_DIR/maven/apache-maven-3.3.3/bin:$PATH
M2_HOME=$OPENSHIFT_DATA_DIR/maven/apache-maven-3.3.3
And that’s it, your builds are now running on the custom maven!
This allows for using a specific JDK in Jenkins. You could just choose a specific JDK via Jenkins console. This is comfortable, but has one disadvantage: It takes a lot of memory (approx. 600MB per JDK), because the JDK is stored twice – compressed in cache to be sent to slave and again uncompressed to be used on the master. If you got enough memory, you’re done here.

However, In case you’re running a small gear with only 1GB of memory, you might want to save a bit of your precious memory. The following example shows how to do so for JDK 8 update 51 build 16.
On SSH:
```
cd $OPENSHIFT_DATA_DIR
mkdir jdk
cd jdk
wget --no-check-certificate --no-cookies --header &amp;quot;Cookie: oraclelicense=accept-securebackup-cookie&amp;quot; http://download.oracle.com/otn-pub/java/jdk/8u51-b16/jdk-8u51-linux-x64.tar.gz
tar -xvf jdk-8u51-linux-x64.tar.gz
rm jdk-8u51-linux-x64.tar.gz
```
Then go to Jenkins
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
JDK installations | JDK
Name=SlaveOnly-Custom-JDK8u51
JAVA_HOME=$OPENSHIFT_DATA_DIR/jdk/jdk-8u51-linux-x64

Building GitHub projects on Jenkins slaves on OpenShift

Posted by schnatterer on August 9, 2015

This post showed how to build GitHub projects with Jenkins, Maven and SonarQube 4 on OpenShift. For starters, it used the Jenkins master node for running build jobs. However, when running on a small gear, the master node might run out of memory pretty fast, resulting in a reboot of the node during builds.

In order to resolve this issue, there are two options:

limitting the memory of the build or
running the build on a slave node.

As spawning additional nodes is easy in a PaaS context such as OpenShift and provides a better performance than running builds with small memory, the slave solution seems to be the better approach.

This post shows how.

Create new DYI app as a slave node (a how-to can be found here), name the node e.g. slave
Create node in Jenkins
1. Go to Jenkins web UI and create new node:
```
https://jenkins-<yourAccount>.rhcloud.com/computer/new
```
2. Set the following values:
  Remote FS root:/app-root/data folder on slave. Typically this is /var/lib/openshift/<slave's UID>/app-root/data/jenkins, you can find out by SSHing to the slave node and calling
```
echo $OPENSHIFT_DATA_DIR/app-root/data/jenkins
```
  Labels: Some label to use within builds to refer the node, e.g. OS Slave #1
  Host: the slave’s hostname, e.g. slave-<youraccount>.rhcloud.com
3. Add Credentials
  username: <slave's UID>
  Private Key File:Path to a private key file that is authorized for your OpenShift account. In the first post this path was used: /var/lib/openshift/<UID of your jenkins>/app-root/data/git-ssh/id_rsa. Note: $OPENSHIFT_DATA_DIR seems not to work here.
  BTW: You can change the credentials any time later via this URL
```
https://jenkins-<yourAccount>.rhcloud.com/credentials/
```
Prepare slave node: Create same environment as on master in the first post
1. Create folder structure
```
mkdir $OPENSHIFT_DATA_DIR/jenkins
mkdir $OPENSHIFT_DATA_DIR/.m2
echo -e "&lt;settings&gt;&lt;localRepository&gt;$OPENSHIFT_DATA_DIR/.m2&lt;/localRepository&gt;&lt;/settings&gt;" &gt; $OPENSHIFT_DATA_DIR/.m2/settings.xml
```
2. Copy SSH directory from master to same directory on slave, e.g.
```
scp -rp -i $OPENSHIFT_DATA_DIR/.ssh $OPENSHIFT_DATA_DIR/.ssh &lt;slave's UID&gt;@slave-&lt;your account&gt;.rhcloud.com:app-root/data/.ssh
```
3. As the different cartridges (jenkins and DIY) have different environment variables for their local IP addresses ($OPENSHIFT_JENKINS_IP vs $OPENSHIFT_DIY_IP) we’ll have to improvise at this point. There are two options: Either
  1. Replace all occurrences of $OPENSHIFT_JENKINS_IP
    In all builds and in
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
    Sonar | Sonar installations
    Database URL: jdbc:postgresql://$OPENSHIFT_DIY_IP:15555/sonarqube
    or
  2. Create an $OPENSHIFT_JENKINS_IP environment variable on your slave machine
```
rhc env set OPENSHIFT_JENKINS_IP=&lt;value of  $OPENSHIFT_DIY_IP&gt; -a slave
```
    You can find out the value of $OPENSHIFT_DIY_IP by SSHing to the slave and execute
```
echo $OPENSHIFT_DIY_IP 
```
  3. I’d love to hear suggesstions that do better 😉
Adapt Build
Easiest way is to not use the master at all.
To do so, go to
```
https://jenkins-<yourAccount>.rhcloud.com/configure
```
and set # of executors to 0.
Hit Apply
Limit memory usage.
Make sure the slave does not run out of memory (which leads to a restart of the node):
Global properties | Environment variables
name: MAVEN_OPTS
value: -Xmx512m
Hit Save.
Now run a build. It should run on the slave and hopefully succeed 🙂

IT affinity!

The Ultimate Question of Life, the Universe, and Everything is answered somwhere else. This is just about IT.