Posted by r.silva on 5 April 2016 - 4:01pm

By Laurence Billingham, British Geological Survey, Steven Lamerton, Science and Technology Facilities Council, Nick Rees, Square Kilometre Array Organisation, Mike Croucher, University of Sheffield, Richard Domander, Royal Veterinary College, and Carl Wilson, Open Preservation Foundation.

A speed blog from the Collaborations Workshop 2016 (CW16).

Researchers, we need to talk about software.

The research community has a problem, some still see talking about it as taboo. Hiding from the problem and hoping it goes away will not make it go away. We are going to have to deal with it sometime, we can only put off dealing with it, and do so by borrowing time from our future selves.

This post assumes little prior exposure to good software development practice and is quite detailed. However, the take home message can be summarised in the one paragraph.

If you have code that seems like a big ball of spaghetti: improve it. First, put in version control, then test it with a known good input and output. Isolate some functionality you do understand, explain what needs to happen to another person, turn that explanation into code tests. Write an isolated function to do that thing well, passing all the tests, check it into version control at each incremental step. Now explain how your new code works to the other person. Finally integrate your code into the big ball of spaghetti. Congratulations, you just made you future life an easier place to exist. Now automate as many of the above steps as you can.

As is usual with these things the first step is acceptance: it may also be the hardest step.

Since that acceptance step can be really, really hard, it might be easier to start with a hypothetical example.

Imagine just as your latest paper was on the verge of being accepted… Only one of the reviewers wanted a seemingly trivial change to Figure 4. Well, it should have been a tiny change… only the code that made Figure 4 has a position within your group like the runes passed down from the ancients.  This distilled genius of generations of renowned forbears is actually a big mess. Every post-doc and grad student for a decade has nailed new pieces of code to the outside willy-nilly; adding abstruse functionality and murdering comprehensibility.  Nobody on Earth understands quite exactly how any of this code works; but, with the right magic incantations you could bring forth the plots vital to the sustenance of a publication record. But, when you needed it most the code let you down. Not only could you not reproduce Figure 4, the only possible products were obviously garbage… Out of the panic- and adrenalin-fuelled all-night hacking sessions you eventually conjured up the figure for the reviewer (maybe it was that time the thirteenth strike of the head-upon-keyboard happened at precisely four am). You know there must be a better way, listen to the sick feeling in the pi of your stomach. You must, and can stop this happening ever again. And this time you really mean it.

But how….?

What follows is a series of concrete actionable steps you can take towards better quality code.

1. Admit that you have a problem

Hello I am Laurence and I have written shitty software. We all have it was hard for us to admit it too; but denial caused by a sense of shame never fixed anything. Your code is not perfect, in fact it will never be perfect, but making it better is a valuable activity: you are investing a little time now to save your future self from a world of panic and frustration. You may have to change the way you think about some of the things you do. Software, like the prose in a paper is something that can be high or low quality. Spending effort on improving that quality, the code equivalent of spell-checking and proof reading, will pay dividends later. The best news is that the open source software community has built, and is using, a whole bunch of tools and workflows to help.

2. Introduce a version control system

Without version control you can never really know when your code was changed or why. Without version control any change you make is a leap of faith. A leap of unknown direction and magnitude. Quite possibly a leap into a sea of chaos. Version control tells you where you’ve been, you can look back and see that you are making things better. Think of it like an infinite undo button for your code editor: one where each iteration of undo comes with comments explaining what the changes were for.

The de facto version control software standard is Git. If your computer runs Linux or OS X, then the git command line tool is probably already installed; it may be out of date though, so it pays to be like the Windows running folk and check the website for version and installation instructions. Git can run in isolation on your local machine, but it is also easy to synchronise changes to a server. You might have heard of GitHub, it is a huge online service, hosting many people’s version control repositories. You can put your code on GitHub for free, and get to it from anywhere with an internet connection, if you are able to make it open source. Other version control hosting options exist if you can’t go the open source route, some of them free for academic use. Your institution’s IT service or computer science department can probably help.

3. There is no reason to be commitment-phobic (commit small things to version control, commit often)

Is hitting save in your document processor or code editor almost a reflex; something you do after every independent thought? Committing to version control should approach something like that. Committing to version control actually shouldn’t be all that much of a commitment. It is not some arcane rite only to be performed on high days and holidays, it isn’t the opening night of the play of your life.

Commit small things frequently, you will build up a much richer, more valuable, more searchable history of changes. You can use the pull request model so that bug fixes and new features are committed in a branch before being integrated into the main code. Using this, individual commits do not even have to work.

If you are doing something hard (would it need researching if everyone knew how to do it?) you will not get it right first time. Everyone accidentally commits bugs at some point.  Committing small and often allows you to record your mistakes and iteratively improve on them. You’ll also likely feel better about progress if it is easy to see, near continuously, what a difference you are making.

Leaving your keyboard to get a coffee? Commit your changes so far.

4. Get your code to build and run on another machine

You should be able to compile, build and run your code on a whim pretty much anywhere (the running is the important part, you can obviously skip some steps if you are using an interpreted language like python). Too many of us have blessed machines which are the only ones that can run ‘you know that one thing’. You want your work to be reproducible right? This may be an apt time to remind you that reproducibility is at the core of the scientific method. In order to do this you will have to figure out exactly what you code depends on being installed: this may be an old compiler, a specific version of a library, or the exactly number of milliliters of unicorn tears. Whatever it depends on, document it and put it in version control alongside your code. There are tools to help, search the interwebs for dependency checking + your operating system.

5. Get your code to build and run in a single step

How much fun is it going through the steps between source code and runnable product? How long does it take? Would you not rather spend that time doing some awesome research (or looking at the latest developments in internet kitten cuteness)? There are a variety of build and deployment automation tools that are available. Like the rest of these steps an hour invested now will save you aeons integrated over your future builds.

6. Test the promises you make

There is a whole movement in software engineering called test driven development. It is actually similar to how we’ve seen a lot of research software get written. At first we have a test in mind: does degrees_to_radians(180) == pi? Then we write code to that makes that test work. Most researchers then throw their test away. Even when keeping it around costs essentially nothing. Keep the tests, wire them up to your code so that they are trivial to run (there are ‘testing frameworks’ available to help for most languages). You can then change stuff and trivially re-run the test you made earlier to make sure the new changes didn’t break anything.

If you have a big spaghetti-like ball of code you don’t fully understand; test the overall functionality. ‘Bless’ some sets of input and output values, treating everything in between as a black box. This set of inputs and outputs become a promise you make to the world. You can break the promise, but only for very good reasons. For example if you find degrees_to_radians() multiplied its input by 57.296… rather than dividing by it, then that is a good reason to ‘bless’ a new set of reference outputs.

Once you have the test in place for your big ball of spaghetti, isolate something that it does (loading a file, checking inputs are within valid ranges, plotting a chart) and make it a standalone function.

7. Modularise

Be as simple as you can. Build software from many simple interacting components rather than in one monolithic lump. If a single function or method does one thing it will be far simpler to verify that it does it correctly. It is also significantly easier to reuse.

Make small pieces of code that do one thing and do them well. Make that quadratic root solver, accurate enough, robust enough, generic enough, and well-known enough, that it is the single place to which your group goes when it needs that piece of functionality.

How much better is a single, well-tested day_of_year(year, month, day) function rather than everyone and their kitten writing it from scratch as lines 40-60 in a 20,000 line monolith?

8. Show someone what you’ve done, with many eyes all bugs are shallow

Show your code to others and offer to read their code. Being the only person to know how X works makes you the wrong kind of indispensable: the kind that has you debugging code over the phone on your honeymoon. Nobody should be that kind of indispensable.

You’ve been writing prose since you were a kid; but you still get your grant proposals and papers proof read. How much easier is to make a mistake with consequences in code rather than prose?

A fresh set of eyes will not only pick up that place you loaded the wrong file, but will also spread good practice around your group. Both reviewer and reviewee will learn things. It will probably work best with an informal chat rather than some formal review process.

Writing code with a human audience makes it more comprehensible, including to future you, and encourages writing in small, understandable units. Writing down a version of what you needed to tell someone about what the software does (which is different from how it does it) is a great place to start with documentation.

Many source hosting sites like GitHub include code review tools for free.

9. Automate all the things

We already discussed automating your build process (Section 5). There are tools available to help automate many of the steps above. Continuous integration services like Travis CI automagically detect changes to your code repository and can then set a whole chain of tools running. You can: create a fresh environment, check out your code, check what is written against a style guide, build the code, run the tests, and tell you how much of the functionality of your code is actually getting tested.

Once these steps are automated, their incremental cost is free. You get rapid feedback after making any change; you know exactly all the things you can trust about your project to do; you can show the evidence on which that trust is based at the click of a hyperlink and; any bugs that do get found can be isolated or squashed easily, without breaking anything else.

This is current good practice for most open source software projects. The open source community have a huge number of tools to help improve software quality, and they are available, and fairly easy, to use.

Epilogue

This post was inspired by the Joel Spolsky’s The Joel Test: 12 Steps to Better Code.

Joel, like the AA, needed 12 steps. Getting to a place where you can improve your code base is easier than that, we can make do with  fewer, but like the AA there are bunch of people who want you to succeed and are willing to help.

You are not alone, there are many people out there interested in making research software better. The Software Sustainability Institute and the UK Community of Research Software Engineers are passionate about making better research through improving software: joining them, learn and implement new practices to decrease the worry about what your code maybe hiding and free up time to do more cool research stuff!