Hosted continuous integration - delivering "Infrastructure++"

Posted by m.jackson on 9 August 2012 - 12:05pm

By Mike Jackson.

At the Digital Social Research workshop, we asked researchers about what would help them to produce good code. One suggestion was that while repositories, such as SourceForgeGitHub and Launchpad, offer a rich infrastructure, they lack support for automated build-and-test or continuous integration. As one attendee put it, they wanted "SourceForge++".

Using a continuous integration server that is hosted by a third-party is more convenient than acquiring a server and setting up the software yourself (it can be cheaper too). This convenience should encourage more people to systematically test their software, and that's something that we're in favour of, because well tested software is more sustainable software.

A web search reveals that there are a number of providers that, thanks to the capabilities of cloud computing, now offer continuous integration services for projects. To help you choose a provide, I've listed the main details of the providers below.

Name Source code repositories Languages Build tools Databases Operating system Hardware Continuous integration Cost
Travis CI GitHub Python, Ruby, Java, JavaScript, Perl, PHP, Clojure, others Maven, Gradle, ANT, make MySQL, PostgreSQL, SQLite3, MongoDB, CouchDB, Redis, Riak, Memcachd 32-bit Ubuntu Linux 11.10   Travis open source framework Free to GitHub-hosted open source projects. "Travis CI pro" for private builds is under development. See Love Travis CI.
BuildHive (part of CloudBees DEV@Cloud platform-as-a-service environment) GitHub Java Maven, Gradle, ANT   Fedora 13 (i686) Amazon EC2 m1.small and m1.large Jenkins Free to GitHub-hosted open source projects. Various pricing plans are available e.g. FOSS for free open source software.
Shining Panda Git, Mercurial, CVS, Subversion, Bazaar Java, Python, C++ Maven, ANT MySQL, PostgreSQL, MongoDB Linux, Windows VM - Intel Xeon core, 2GB RAM, 5GB hard disk Jenkins Free open source software projects can get access on a first-come-first-served basis. Other pricing plans are available.
tddium Git Ruby (Python, PHP coming soon)   MySQL, PostgreSQL, SQLite, MongoDB, Redis, Riak, Sphinx Ubuntu 11 32 or 64-bit x86 Linux VM Custom Pricing from $15/month.
CodeBetter CI Git, Subversion   Maven, MSBuild       TeamCity Free for open source projects (TeamCity funds it)
AppHarbor Git/GitHub, Mercurial, Subversion, BitBucket ASP.NET applications   Shared or dedicated MySQL, SQL Server, MongoDB, depending on subscription. Windows Amazon EC2 Custom. Also deploys applications for users on their servers. Pricing from $0/month for a single worker.
Bamboo CVS, Subversion, Git/GitHub, Mercurial, BitBucket Java, JavaScript, Python, Ruby ANT, Make, Maven, .NET     Amazon EC2 Custom Free 30 day trial. "On Demand" (they manage deployment and use your EC2 VMs) from $10/month (plus your EC2 subscriptions)
CloudForge Subversion, Git Any Any   Any   Custom Free for a single user. Pricing from $3.5/month otherwise.
Semaphore GitHub Ruby 1.9   MySQL, PostgreSQL, SQLite, Redis, MongoDB     Custom Pricing from $14/month for a single project (public or private)
TestPilot (beta and most web pages show "we're still writing this page") GitHub Ruby         Custom Free for open source projects.
CircleCi (beta) GitHub Ruby, Java, JavaScript, Python, PHP, Clojure, Scala, Erlang, C++ Maven, ANT, Make MySQL, PostgreSQL, MongoDB, CouchDB, Redis, Riak, Memcached, Sphinx, Cassandra Ubuntu 11.10   Custom 14 day free trial
hostedci (beta)         iOS, Mac   Jenkins 20 hours/month build free for open source projects. Other pricing plans available.
continuous.io (seems to be an alpha which has stalled, last update was 07/10/11) GitHub         Amazon EC2 micro Custom Free for open source projects (you need to use your own EC2 VMs). Other pricing plans are available.
Fazend.com (may have stalled, last update was 30/08/11) Subversion (theirs or 3rd party), Git, Mercurial Java, Python, C++, PHP, Ruby, Perl Maven, ANT, Make MySQL, SQLite Linux   Hudson Free. E-mail for requirements review.

Please remember that when choosing to rely on a third-party to host a continuous integration server, the reliability, longevity, cost, usability and sustainability of the provider's service should be considered. Our advice on choosing a repository for your software project is also relevant to choosing a continuous integration server.