Posted by m.jackson on 16 December 2014 - 10:00am

By Mike Jackson, Software Architect, Andrew Turner ARCHER Computational Science and Engineering Support Team Leader, and Clair Barrass, ARCHER training administrator.

In 2013, the DiRAC consortium rolled out the DiRAC driving licence, a software skills aptitude test for researchers wanting to use DiRAC's high performance computing resources.

Now, ARCHER, the UK National Supercomputing Service, is to roll out an ARCHER driving test. Despite their similar names, these tests differ in nature, intent, scale and reward. In this post we compare and contrast these two supercomputer tests.

DiRAC Driving Licence

DiRAC is the UK's integrated supercomputing facility for theoretical modelling and HPC-based research in particle physics, astronomy and cosmology. The DiRAC driving licence was originally developed in 2012 by Mike, Greg Wilson of Software Carpentry, and, from DiRAC, Andy, James Hetherington of the Research Software Group at University College London, DiRAC project director Jeremy Yates, and project manager Harpreet Dhanoa.

The driving licence covers software development skills including the shell and automation, version control, testing, code review and the use of public/private keypairs and secure shell. DiRAC users sit an open book test - a sample test is also available for them to look at in advance - which takes 1-1.5 hours, under the supervision of a trainer. The trainer marks each user's answers, providing 1-on-1 feedback via e-mail or Skype.

The licence is mandatory for all UK-based DiRAC users who are studying for advanced degrees, be they Postgraduate or Doctoral, and for those employed as post-doctoral research assistants or associates with less than 3 years’ experience. Other users are also encouraged to take the test. Upon successful completion, participants are awarded a certificate of competency. Those who fail are directed to courses to fill in the gaps in their knowledge, before sitting the test again at a later date.

The driving licence is intended to encourage users to undertake training which will help to improve their scientific computing skills and so to deliver research that makes a greater impact. It also helps DiRAC ensure that their resources are used as efficiently as possible.

The driving licence was rolled out across the DiRAC consortium in 2013, and has been run numerous times at seven sites, including Edinburgh, across the UK. The driving licence will continue to run throughout 2015.

ARCHER Driving Test

ARCHER is the UK National Supercomputing Service. ARCHER provides a capability resource to allow researchers to run simulations and calculations that require large numbers of processing cores working in a tightly-coupled, parallel fashion.

The ARCHER driving test has been developed by the ARCHER Computational Science and Engineering (CSE) team at EPCC. The test consists of questions that cover both ARCHER's resources and how day-to-day development is done within ARCHER's infrastructure. The test consists of multiple-choice questions which either require users to choose a single answer or a set of answers. The test is taken online and feedback is provided from canned responses that provide more information about both correct and incorrect answers, with links to further information. The questions, and supporting commentary, were crowd-sourced from within the CSE team.

Users who pass the test are given time on ARCHER as a reward. The test is optional and is intended to provide another, lightweight, avenue by which users can get access to ARCHER's resources, in particular those with experience of other HPC systems who can demonstrate they can make effective use of ARCHER. The test can also be taken by attendees at ARCHER’s Hands-on Introduction to High Performance Computing courses to help them to identify what they have learned and what they might need to revise. It is hoped that rewarding success with time on ARCHER will encourage attendees to apply some of their new learning immediately, while still fresh in their minds.

The ARCHER driving test will be rolled out in 2015.

Scalability versus personalisation

The ARCHER driving test is less time-intensive and more scalable than the DiRAC driving licence, but at the expense of personalised feedback on each user's skills and training requirements. However, such comparisons need to be viewed in light of each consortium’s respective user communities and training services.

DiRAC has approximately 500 registered users. Most are permanent academics or PDRAs with more than three years’ experience and so are exempt from taking the driving licence. DiRAC provides no training courses of its own, rather it recommends a portfolio of online material that DiRAC users are encouraged to study. The driving licence serves as a motivator for users to undertake this. Personalised face-to-face feedback from trainers provides users with the opportunity to ask questions and resolve problems that would, in other contexts, be resolved on training courses.

In contrast, ARCHER has around 3000 registered users. To deploy a DiRAC-style driving licence across such a large community with personalised marking and feedback would be un-scalable. However, ARCHER runs a large number - 32 in 2014 alone - of training courses at introductory, intermediate and advanced levels at sites across the UK, complemented by virtual tutorials, which offers scope for users to ask questions and resolve problems specific to them.

Conclusion

Though the DiRAC driving licence and ARCHER driving test differ in both nature and delivery they share one important quality – they both encourage the users of these HPC resources to improve their understanding of these resources and how best to use them. As a consequence, they both help to contribute to the development of users' skills in software development, and also in the more effective and efficient usage of HPC resources which are in much demand.