Posted by m.jackson on 16 May 2013 - 2:22pm 

By Mike Jackson.

Deciding upon a computing infrastructure for your project can be a daunting task. Do you want to use your institution’s existing infrastructure, buy a new one, or go for a third-party? Would an in-house cluster be best or a commercial cloud platform? Regardless of whether you seek a cluster, grid, cloud, HPC facility or volunteer computing infrastructure, there are a number of things you should consider.

Here are our top tips for choosing a computing infrastructure…

1. Understand your application, understand your requirements

Make sure you understand the CPU, memory, disk and network requirements of your application as these place requirements on any infrastructure you may choose. Will the infrastructure have the ability to handle your application? You may not know for certain, not least because your application might not yet exist, but you can determine if it’s theoretically feasible.

If you already have your application, you could run some performance tests to identify your application’s demands. If you do, consider the performance of your application at its extremes, like running on the largest, or most complex, data sets that you are likely to want to use. It can also help if you know how to configure and use your application such that its requirements can be optimised with respect to the available resources provided by the infrastructure. For example, can you run your application in parallel, or partition its inputs in a different way?

2. Assess the dependability of your candidate infrastructure

You don’t want to go to the effort of migrating to an infrastructure only to discover it will cease to be supported a year before the end of your project, that its performance is erratic, or that it takes days for support requests to be answered or new packages installed.  For any infrastructure you consider, find out whether it will be there for as long as you need, and what you can expect in terms of resource availability, help and support. Are there any service level agreements offered? For commercial infrastructures, especially clouds, what happens if you don’t pay your bill? Is your content deleted? Are you warned first? It’s also important to consider backups. Are these done for you? If so, how frequently? If not, then is there a way for you to do your own backups?

3. Think about legal and ethical issues

If you are dealing with sensitive data, such as medical data, there may be severe restrictions on how and where and in what form this data can be transferred and hosted. You may be able to host the data on a third-party infrastructure but only if the data is suitably anonymised, or you may have to ensure that your software or data is adequately encrypted in transit. Familiarise yourself with policies relating to the security and privacy of your data, and discuss these with your stakeholders, to ensure that you'll be in compliance with them.

Whether the software you use is proprietary or open source, free or paid for, there may be terms and conditions attached that affect where it can be hosted and run. For example, deploying software on a cloud could be considered as supplying it to a third-party and violate the terms of a licence that prohibits distribution to third-parties. OSS-Watch provides consultancy in the use of open source software and JISCLegal can provide advice on legal issues relating to data protection and storing data on third-party resources.

4. Consider financial issues

If considering a commercial infrastructure, or purchasing and running your own, you’ll have to assess whether this is within your financial abilities to do so. You need to be aware of your application’s demands (see 1. above) and the period for which you’ll need your infrastructure, particularly if you intend to deploy a long-running service. You should also check that such expenditure is acceptable to your funders.

Consider, also, how you’ll actually pay for your usage. Commercial services, for example clouds, may require you to register a credit card. Are you prepared to use your own credit card or does your department or institution have a credit card that you could use? Would they be happy for you to use it? And, what happens if you exceed your usage? Are you warned, or just billed at a higher rate? It’s important to understand any pricing terms and conditions carefully so that you are not hit by unforeseen and excessive bills.

5. Choose your infrastructure

Hopefully, you’ll now be ready to make your choice…but, you should also have a contingency plan just in case your infrastructure becomes unavailable. Is there another infrastructure you could use? Would you have the time, money and effort to migrate your content?

Good luck!