Software and research: the Institute's Blog

How to do data science, big data, and IoT easily - with cloud computing

By Kenji Takeda, Microsoft Research

When people talk about big data, data science and streaming data from devices, it can seem pretty scary. It conjures up images of complex IT infrastructure, many different systems to be stitched together, and requiring expertise beyond most researchers’ comfort zone. You certainly need to think about what you’re trying to do, but with cloud computing you can create what you need easily through a web portal, script or program. For example, researchers at the University of Oxford have taken their machine learning prototype from the lab, processing data from smart water pumps, and are deploying it across thousands of pumps in Kenya using the cloud. The best way to find out if this will work for you is to try it out…

You can get your hands dirty by joining us at one of our free, cloud computing training courses in the UK and across Europe, custom-designed for you by Microsoft Research. Whether it's big data, machine learning, big compute (HPC), or analysing data streaming from devices for an IoT project, you’ll discover how easy it is using the open Microsoft Azure cloud platform to speed up your research.

Open source tomography training: Swansea & Leicester 2016

Tomography machine. Image by Martin Abegglen. https://flic.kr/p/688v1HBy Russell Garwood, Lecturer at the University of Manchester.

2015 Software Sustainability Institute fellow Russell Garwood has completed his fellowship by giving two institute-sponsored training courses in using open source software, showing how to analyse and visualise tomographic datasets. By learning the basics of Drishti, SPIERS and Blender, attendees have many of the tools needed to conduct research using tomography data, avoiding expensive proprietary software.

From zero to a responsible software developer in a week

University of Oslo by Alexander Ottesen (under CC-BY).By Iza Romanowska, Institute Fellow and PhD Student at University of Southampton.

Learning a new computational technique, be it simulation, specific type of data analysis or even lab-based methods, can be a daunting task. You could start by reading up on all the previous applications and methodological papers but it can leave you frustrated with the technical nitty-gritty which is virtually impenetrable without a good knowledge of the tools that were used. So perhaps, it is better to start from the other end and learn how to use the software first? Sounds like a reasonable plan until we are reminded of the legions of early career researchers trawling through literature looking for a nail they could hit with their shiny new hammer.

Steps To Start Liberating Your Science

Handcuff. Image by Naiane Mello. https://flic.kr/p/34MDfiBy Robert Davey, The Genome Analysis Centre (TGAC), Ross Mounce, University of Cambridge, Larisa Blazic, University of Westminster, Anelda van der Walt, Talarify, and Raniere Silva, Software Sustainability Institute.

A speed blog from the Collaborations Workshop 2016 (CW16).

The Open Science movement is facing a challenge - how do we convince our peers to liberate their science? During the Collaborations Workshop 2016, we developed these 9 steps to help anyone that is unsure what Open Science is, or who are looking to make their science more open.

Evolutions in the Discussion of RSE Career Paths

Path. Image by Miguel Carvalho. https://www.flickr.com/photos/miguelvirkkunen/4382242057/in/photolist-7Ff8Tx-4FaRUB-dJbcDz-4FpuLy-aswoX-8bRHjP-nMuH4k-4mkXSn-ngi9Rx-pMtkir-ajGwps-5cSUri-4nUhX1-6VGkWF-jwfTNF-fbv68g-e9Hst4-fdRKVP-4yirjk-aVRq58-bn3VPt-5UMJwb-saqPyz-4D1DEu-jwfuzM-i3RcZu-owH6UW-67F1wU-dcpvSR-4nrvzk-892xuZ-6U6HrJ-sjp8TD-5meAyX-e4h7uv-bvNZhf-fx476s-81zcov-aox41f-8oqksv-nNKakF-aAyhDv-7s9yhu-9G4XJu-it8wBd-5zobAE-biUXUF-9FkLjc-8YGjfD-57KRg2By Mark Stillwell, Cisco Meraki, Caroline Jay, University of Manchester,  Robert Haines, University of Manchester, Louise Brown, University of Nottingham, Jeremy Cohen, Imperial College London, Alys Brett, Culham Centre for Fusion Energy, Shih-Chen Chao, University of Manchester, Raquel Alegre, UCL, James Davenport, University of Bath, and James Hetherington,UCL.

speed blog from the Collaborations Workshop 2016 (CW16).

Huge progress has been made in recognising research software engineering as a profession since initial discussions about this role began at the Collaborations Workshop in 2012. The topic still gets a lot of coverage at Software Sustainability Institute and UK Research Software Engineer (RSE) events, and with good reason. Many of the basic problems that led to the initial discussions continue to exist: in particular, a lack of academic credit for software contributions, and lower pay in relation to similar industry roles. While these problems remain unsolved and important, the fact of the matter is that people are now carving out career paths as RSEs or managers of RSEs, and new issues and concerns are starting to arise.

NULL, not void

'Girls carrying hay bundles' by Inhabitat (CC-BY-NC-ND)By M.H. Beals, Loughborough University, J. H, Nielsen, UCL, B. A. Laken, UCL and M. Antonioletti, University of Edinburgh.

A speed blog from the Collaborations Workshop 2016 (CW16).

The importance and credit associated with publishing negative results.

As researchers, the majority our experiments and explorations do not always pan out. When this occurs, pressure prompts us to move on to the next idea, looking for that big result that will make our name and build our reputation. What are the knock-on effects of doing this? By not reporting our failures, are we cursing others to repeat them? Does our tendency to curate our results slow our progress and, if so, can we change this?

Collaborate and don't die trying

Image by CASTLE ROCK INNOVATIONS.By David Perez-Suarez, University College London, Phil Bradbury, University of Manchester, Aleksandra Nenadic, University of Manchester, Laurent Gatto, Cambridge University, and Niall Beard, University of Manchester.

A speed blog from the Collaborations Workshop 2016 (CW16).

Remote collaboration: challenges in Human-Computer-Human interactions.

Tools that were mentioned during the discussion: GitHub, BitBucket, GitHub issue tracker, Skype, Google Hangouts (but max participants in Skype/Google Hangouts), Google Docs, spreadsheets, Jira, todo lists, time sheets, DropBox, … but are tools really the problem?

How do you teach Sustainable Software Practices 101?

By Oliva Guest, University of Oxford, Robin Wilson, University of Southampton, Martin Jones, Python for biologists and Craig MacLachlan, Met Office Hadley Centre.

A speed blog from the Collaborations Workshop 2016 (CW16).

Why are sustainable software practices difficult to teach?

Programming is a difficult thing to learn for students who have not been exposed to it before. However, for general programming there are at least some factors that help to make it easier. Feedback is generally very rapid; after writing and running a piece of code, students can see the result straight away. This isn't true for e.g. automated testing; the payoff for writing a test suite comes long after the fact, when it helps to catch a bug. The same goes for version control — until students have encountered one of the problems that version control is designed to solve, it seems like an unnecessary extra step in development.

What do you tell your PhD student Three Years Before They Leave …

By Vince Knight, Cardiff University, Olivia Wilson, University of Southampton, Shoaib Sufi, Software Sustainability Institute, Steve Crouch, Software Sustainability Institute, and Ian Gent, University of St Andrews.

A speed blog from the Collaborations Workshop 2016 (CW16).

"Congratulations Dr Smith"

The words every PhD student dreams of hearing, at least if your name is Smith. First from your examiners, and then soon afterwards from your supervisor. And then those words every PhD student dreads of hearing from your supervisor…

"Just before you go to your super-rich quant futures job on Wall Street, could you just …

…. hand over your code to my new PhD student please?"

The inevitable abyss: find mentors who will help you get out

A view through my glasses. Image by UnknownNet Photography. https://flic.kr/p/rFgD9rBy Heather Ford, University of Leeds, Jonah Duckles, Software Carpentry Foundation, Angus Maidment, STFC, Martin Callaghan, University of Leeds, Terhi Nurmikko-Fuller, University of Oxford e-Research Centre, and Amy Beeston, University of Sheffield.

A speed blog from the Collaborations Workshop 2016 (CW16).

We are all continually entering and exiting the constant cycle of the hero’s journey. It’s the story of learning.

The question of best practices for the mentoring of a range of people from various different backgrounds is multi-faceted and complex. There are three main aspects that proved particularly fruitful for conversation: