Unit tests (Part 2)
In part 1 I described the rationale behind why I personally think unit tests can help save you money and time when building software for your clients. Part 2 is what started as a google search, turned into a dive into the deep-end of CI.
Continuous Integration (CI)
CI is a trick subject either you instantly get what it’s for or you don’t. I guess it’s a case of you’ll know when you need it.
So what is it? Well it’s a normally a server application that allows you to monitor your builds and allow you to track broken builds as well as the state of your unit tests that you’ve been writing. This becomes incredibly useful if you’re working with others (not limited too though, it’s also useful if you write alone). At any point in time you would be able to see what the main state of your software is at a very high level. A great side effect for me is that – I can point clients, developers and testers to one location and they can grab a build (known as an artifact in the CI world) for them to use. Everything is transparent, to you, your boss, your clients, everybody and this is the benefit. The more transparent you are the better expectations of the software people will have. It’s amazingly simple, hiding things away will come and bite you later on.
Which software? I spent a only a few hours trying to find a CI application, because the way I see it is, try something first and if it doesn’t fit the bill, just migrate to the next thing. I ended up with Cruise Control, it’s definitely not the prettiest application in the world and it feels very disjointed from a UX point of view, but does the job well. I did originally want to use Hudson, but it doesn’t support OS X.
Hardware? Ideally I would love to use ubuntu as the OS of choice, but could only get my hands on a old PowerPC G5 (with 6GB ram!!). This should be easily sufficient for running the builds.
Getting started.
Because Cruise Control is a java application, it default runs on the Jetty server. It’s a great server out of the box and needs almost no configuration to get the server running. The only one for me is hooking up to the log4j to something other than the console.
The next thing was setting up a project that was the “turnkey” example. This turned out to be a mixed experience… to set up a very basic example is extremely easy. The configuration file syntax is not to dissimilar to ant, which makes life extremely easy. I just pulled a branch (so I didn’t break anything from the master) from my git repository to the projects folder and hooked up my build file to the configuration file. I also utilised the clean to remove all the artifacts from the previous build so we always start with a clean build. That way if something goes wrong, it doesn’t affect the next build. I also used the configuration file to do a “git pull” at the bootstrap phase, so I didn’t have to do a pull manually.
That’s it, I had the simplest setup CI, the massive issue I had was getting logging to work for unit tests. There really needs to be more documentation about this as to me or new comers it feels like magic how it works.
So to all new people, you have to output all you’re log files to xml, it has to be in xml (you can do other file formats, but this is the easiest way) and it has to conform to junit xml format. I put my log files in the project and made a log folder where the logs could be dumped into when building. On clear command I would recreate the log folder. Then using the configuration tell CI where to look to find out if it was a successful build. This part took me half a day to figure out, it didn’t help when you have two programs called “cruisecontrol”, one being this and another being for ruby. I don’t know which came first, but one needs to change their name!
Web interface
I’m not going to talk too much about this, but to be honest it feels horrible to use from a UX and graphical point of view. It needs someone to go through the design and just tie it together. It’s a real pity as there is so much information in the interface, but you really have to go looking. There is a lot of information about types of builds, failures and configurations it’s just not presented well. It could easily become amazing as it has all the right features, but just doesn’t compete.
Conclusion
All in all, it wasn’t a bad experience, but it just feels like the whole thing is held together with magic – especially with the logging system for builds and unit tests. It’s hard to have 100% faith in something that you would like to do CI with. The great thing is that this could be easily sorted with some more documentation and some fixes to the configuration site.
Part 3 : ASUnit vs FlexUnit