Software evaluation results

This software evaluation report is for your software called: softwarename. It is a list of recommendations that are based on the survey questions to which you answered "no".

How to get hold of this report

The URL of this page is personalised to your answers, so if you want to see this report again, simply copy the URL.

What happens if there's no more text?

If no text appears below this paragraph, it means you must already be following all of the recommendations made in our short evaluation. That's fantastic! We'd love to hear from you, because your project would make a perfect case study. Please get in touch!

Q1 - What does your software do?

Question 1.1: Does your website, READMEs and documentation provide a clear, high-level description of your software?

The fundamental question about your software is what does it do?. A potential user should be able to easily find a one- or two-sentence description of your software in every user resource: the website, all documentation and any READMEs.

It can be difficult to quickly summarise your software, especially when you’ve spent months coding hundreds of exciting and powerful features. However, unless you catch the attention of a potential user very quickly, you risk losing them as a real user!

Question 1.2: Does your website, READMEs and documentation clearly describe the type of user who should use your software?

Very little software is designed for users from all backgrounds. If your software is only designed for people with specific types of expertise, you must be upfront about this in all your documentation. If you are not, you risk attracting the wrong type of users who will find the software unsuitable for their needs and may complain about this fact.

Question 1.3: Do you make case studies available to show how your software has been used by other users?

A great way of showing off your software is to write case studies about the people who have used it. This helps potential users learn about the software but, more to the point, it is a great advert for your software. If you can show happy users benefiting from your software, you are likely to gain more users.

Q2 - Support

Question 2.1: Does your website describe how to get help with your software?

The level of support that a user can expect to receive is often a vital element in a user’s choice of software. This means that the support you provide - whether it’s a guaranteed response in twenty-four hours, or a potential response on a best effort basis – needs to be made clear on your website and on any support resources you provide, e.g. guides and README files. It is also important to describe how a user can submit a support request: email, telephone, issue tracker, forum or other means.

Providing this information is simple expectation management. A user will always want their issue to be solved as quickly as possible, and may become disgruntled (and might even stop being a user) if this is not the case. If you are clear and honest about the level of support you can provide, then you are likely to keep your users happy.

Question 2.2: Does your software distribution describe how a user can gain help with the software?

There is one certainty in support: when a user discovers a problem, they’ll be sitting at their computer using your software. This means that the first place they will look to try and find a solution is in any help packaged with your software distribution. For this reason, you should always have a “help” menu or an equivalent. Such information could include support email addresses, website URLs, etc. for where to get help. If so, make sure the resources you state are static for the anticipated lifetime of the software!

Question 2.3: Does your project have an email address that is dedicated to support requests from users?

Email is purpose-made for resolving support problems. The user can provide a good description of the problem and can even attach screenshots or other supporting evidence. This means that you should always try to have an email for support queries.

It’s best if the support email is clearly labelled as such: support@mysoftware.com. This makes it easy to identify the email address on your website and it helps separate your support queries from all of the other email you receive. However, a personal address is better than nothing if you don’t have the facility to provide a dedicated support address.

Question 2.4: Are emails to your support email address received by more than one person?

It’s easy to forget about an email (especially one that’s asking difficult questions), so your support email should always be received by more than one person. This also means that your support queries will be registered even if you are on holiday, ill or otherwise indisposed. However, it is best to ensure that there is always one person who is primarily responsible for the list and for responding to queries.

Question 2.5: Does your project have a ticketing system for recording support requests?

Dealing with one or two support queries is straightforward enough, but as the number of queries grows and their complexity increases, it gets easier to make mistakes. Nothing will annoy a user more than their support queries dropping into a black hole.

The best way to organise support queries in a scalable way is with a ticketing system. Not only does this provide an easy method to remember who asked what, it also allows you to store additional information about the query and prioritise queries so that you can work on the most important first. A ticketing system is an absolute requirement if you have more than one person working on the queries – it prevents effort duplication and allows you to easily keep up to date with progress.

Ticketing systems are also very handy at providing statistics. You can find out the components of your software that produce the most problems, then improve them or better document their use. You can also see how quickly you are dealing with issues, and check that you are meeting the level of support that you have advertised.

Examples of ticketing systems include JIRA, Bugzilla and Trac. If you are already using something like SourceForge or GitHub to host your software, these already have issue trackers built in!

Question 2.6: Is your project's ticketing system publicly accessible so that your users can view the support requests you have received and your response to them?

An open ticketing system has a number of advantages. Primarily, your users can see that you deal with your support queries, which gives them confidence in your software and makes them more likely to use it. Secondly, it provides your users with a means to identify an encountered issue as a known issue that is already registered, and allow them to check progress on it.

Question 2.7: Does your project have a blog that is regularly updated (approximately a post per fortnight)?

Blogs are one of the best ways of showing that your project is active. If users see frequent posts, especially if they talk about new features and resolved problems, they know that the project is working.

A blog also gives you a way of getting in touch with your users and asking for their input or help with problems. Don’t know whether to implement a new feature? Ask your users whether they think it would be useful.

Q3 - Documentation

Question 3.1: Is your documentation easy to locate? Can a user quickly access the documentation on your website or from links in your software distribution?

Documentation is one of the major ways of helping your users get to grips with your software, but it can’t help if your users can’t find it.

There are two places that a user will expect to find documentation: packaged with the software and on your website.

Question 3.2: Do you provide a "quick start" guide (a guide that provides a quick overview of the software with some basic examples of use)?

Quick start guides provide the bare minimum amount of information needed to install software and try out some of the basic features. If you provide this documentation, your users will be able to test your software without investing a significant amount of their time. This helps more people try out your software, and that leads to more users.

 

Question 3.3: Does your documentation provide clear, step-by-step instructions for installation and use of your software?

You get so familiar with software during its development that it can be difficult to put yourself in the mind of a novice user. If you write documentation from an expert point of view, you can overlook the basic steps that novices need to know. This kind of documentation will soon alienate a novice user, and could potentially lose you a user.

When documenting software, go through every step of its installation and use and try to see through the eyes of someone completely new to the software. Describe everything you do, and err on the side of caution. It’s better to slightly over-simplify documentation than under-simplify it.

Question 3.4: Does your documentation describe the different support options that are available for your software?

If a user is looking at your documentation, there's a good chance they're looking to solve a problem, so it makes sense to describe the support you provide in the documentation.

Question 3.5: Does your documentation clearly state who is intended to use your software (e.g. any users, or only users with specific knowledge or from a specific background)?

Your software might be written for users with any level of expertise, or it might only be intended for users with specific experience. If you are only targeting a specific type of user, you should state this clearly before they invest time in your software

Question 3.6: Do you provide a reference guide your the software (i.e. a comprehensive guide to all software functions and options)?

If you want users to make use of every feature of your software, you must describe each feature and how to use it in a reference guide.

Question 3.7: Do you provide a troubleshooting guide: a guide that describes the symptoms and step-by-step solutions for problems and error messages?

A troubleshooting guide helps users quickly solve problems. It’s a useful addition to your documentation that can help reduce the number of support queries that you have to deal with.

Question 3.8: Do you provide comprehensive API documentation?

If your software includes an Application Programming Interface (API), including comprehensive technical documentation about the functions of the software is important if you want people to use it. Even with code examples, it may not be obvious how to use the API in their own code. From structured comments in the code, generating complete, structured API documentation can be done automatically with, for example, Javadoc or Doxygen.

Question 3.9: If your software's source code is available, do you store the software's documentation under version control with the source code?

As your software changes, your documentation will too. A lot of confusion can result from a user applying one version of the documentation to another version of the software. The way to solve this problem is to clearly label the documentation with the version of the software it should be used with, and store the documentation with the software under version control.

Question 3.10: Does the documentation for your source code describe the licence and copyright for the code?

Users need to know about the copyright and licensing provisions of your software (and of any third-party software bundled with it), because there might be conditions and obligations on its use and redistribution. Developers need to know the conditions under which they can change or extend your software, and any restrictions on their extensions. It is also useful for users and developers to know the provenance of any third-party software bundled in a release should they wish to create releases that contain this third-party software.

If you don't have a licence, then see our guide on Adopting an open source licence.

Question 3.11: Does the documentation for the binary distribution describe the licence and copyright for the code

Users need to know about the copyright and licensing provisions of your software (and of any third-party software bundled with it), because there might be conditions and obligations on its use and redistribution. It is also useful for users and developers to know the provenance of any third-party software bundled in a release should they wish to create releases that contain this third-party software.

If you don't have a licence, then see our guide on Adopting an open source licence.

Q4 - Plans for the future

Question 4.1: Does your website include a project roadmap (i.e. a list of project and development milestones)?

A roadmap allows users to see when new features will be added and plan their project accordingly. It also has an important secondary benefit: one of the most important factors that will influence a user’s choice of software is the likelihood of that software still being around - and supported - in the future. There are many ways in which a project can persuade a user of its longevity: an active website, active support provision, information about funding and its plans for the future - a roadmap.

Question 4.2: Does your website describe how your project is funded, and the period over which funding is guaranteed?

Especially on academic projects, users will link the likelihood of future support for the software to the duration of the software project’s funding. If you want to persuade users that your software will be around in the future, it is a good idea to state your funding model and the duration over which funding is assured.

Q5 - Availability of your software

Question 5.1: Is the software available as a pre-compiled binary distribution?

A pre-compiled binary distribution saves users the time and effort of building your software. This is especially important if your software takes more than a few minutes to build or if the users are not software developers. Ideally, you'll test that your software builds on the platforms it is meant to support, which means you will already have created binaries that can be distributed to your users.

See our guide on Ready for release? A checklist for developers, and if you're interested in the consequences of ignoring your users' needs, see How to frustrate your users, annoy other developers and please lawyers.

Question 5.2: Is the binary distribution available for free?

A free binary distribution means that there is no financial cost for users to downloading your software and "give it a go". It may also prove more appealing than competing software that has similar functionality, but which charges a fee.

Question 5.3: Is the source code for your software available to download?

Making your source code available can give users confidence that if you disappear they at least have the means to be able to fix, improve or extend your software themselves. It also gives them the potential to make such changes when they need them, rather than having to wait for you to do it on their behalf. If you adopt a suitable open-source licence, and are willing to accept contributions from your users, then this can serve as a source of free effort for bug fixes or functionality that you don't have time to develop yourself.

OSSWatch describe the benefits of open source and its appeal to users. The Ask Steve! blog has discussions on making the move to developing software in an open way (see part 1 and part 2 of the discussion).

Question 5.4: Is your software, and the resources for your project, hosted in an established, third-party repository like SourceForge, LaunchPad or GitHub?

Hosting resources in an established, third-party repository gives users confidence that if you disappear then at least they will continue to have access to your source code, to be able fix, improve or extend your software themselves. There is another significant benefit: many third-party repositories offer free project hosting, which relieves you and your organisation of the cost and effort of software and systems maintenance associated with hosting the software.

See our guide on Choosing a repository for your software project.

Q6 - Source code structure

Question 6.1: Does your project recommend a coding standard?

A coding standard is a specification on how your code should look: how blocks are indented, what comments should be provided and the format for these, and how files, modules, classes, functions, methods and variables are named. As well as ensuring that your code is readable, coding standards help to provide a sense of identity and consistency across your software, even if you have a large team of developers. See our guide on Writing readable source code

Question 6.2: Is your source code structured into modules or packages?

Modularity is a fundamental software design approach centred on the creation of self-contained functional units, or modules, which serve specific purposes (e.g. file I/O, authorisation, logging, gene splicing, text analysis).

Modularity has many benefits. It's easier to reuse modules in other software, which prevents replication of development effort. Each module is self-contained, so it can be changed or updated without affecting the other modules in the code, and it can also be tested is isolation, which is useful when developing as part of a team. Modular designs are also easier to understand.

Given that source code is the final realisation of a design, it too should be modular and structured in a way that makes the modules clear.

For more on modularity, see our guide on Developing maintainable software and the modular design article on Wikipedia.

Question 6.3: Is your source code stored in a repository under revision control?

If you are developing open-source software then revision control is essential, especially if you plan on migrating to open development.

Almost everyone uses some form of revision control. Even just keeping a back-up of some code - while doing bug fixes for example - is a rough type of version control. The naive approach is to just dump everything into directories with names like my-product-16012012-works. A repository with revision control provides a number of benefits:

  • Retrieve any version of your software, or any file within those versions, from any point in time.
  • Support multiple developers working on the same software, allowing them to share their changes without any risk of a file being overwritten and its previous contents being lost forever.
  • Record who changed what, when and why, and keep a complete audit trail of the evolution of your software for you and, in an open-source world, for your users.

 

For more on revision (or version) control see What is version control? from OSSWatch and the Revision control article on Wikipedia. Revision control is an important aspect of developing maintainable software.

Question 6.4: Is each source-code release a snapshot of the repository?

Some source distributions take source code from the source-code repository and use it to create a release with a directory structure which is markedly different to the structure of the repository. This can lead to a lot of confusion.

It is much better to treat your source-code release as a snapshot of your repository, i.e. make your repository structure identical to your release structure. This makes it far easier to understand the mapping between the structures, which means given a file in one structure, it is easier to find the same file in the other. In addition, the scripts that you use to build your source code in the repository can also be used as-is by users to build binary releases from source-code releases. This reduces the number of release-specific scripts you need to write and maintain.

Question 6.5: Is your source code well laid out and indented?

Indentation helps make your code readable. It makes a clear connection between blocks of code and the classes, functions or loops to which they belong. If a statement is longer than a single line on screen, indentation helps a developer understand where the statement begins and ends. Whitespace makes the code appear less cluttered and allows the grouping together of logically-related elements (like constants or local variable declarations). See our guide on Writing readable source code.

Question 6.6: Is your source code well commented?

Source code tells a developer what the code does and how it does it. Comments allow you to provide a developer with additional information on why it was written that way - they can capture the reasons the code is written as it is, which can be important if you've implemented a function in a non-intuitive way.

A developer should be able to understand a single function or method from its code and its comments, and should not have to look elsewhere in the code for clarification. It's easy to get lost in code, and a developer who reads your code will not have the same knowledge of it as you do. See our guide on Writing readable source code.

Question 6.7: Does your source code use sensible class, package and variable names?

Cryptic names of components, modules, classes, functions, arguments, exceptions and variables can lead to confusion about the role that these components play. A developer may get the wrong idea about the role of the component or, in the worst case, have no idea at all about what the component does.

Good naming is fundamental to good software design - your source code represents the most detailed version of your design. Good naming also helps make your code readable.

Q7 - Open standards

Question 7.1: Does your software use open standards?

Open standards enable different software packages to interoperate. If data are held in a format complying to an open standard, they can be accessed by any other software that can read data in that format. This also prevents a user from being locked in to a particular software package, because they can use any compliant software to access their data. Even more appealingly, it's easy for users to swap between software that uses open standards, because they don't need to convert their data. This means that users of other software may switch to your software, if your software is more efficient, robust, scalable or functional than that of your competitors'.

Question 7.2: Are the standards you use mature and ratified?

Standards can go through many iterations, because they evolve as ideas are proposed and debated and the scope, remit and intent of the standards are agreed. If a standard changes, then any software that uses the standard needs to be changed to keep up to date. Most of the big changes occur early in the lifetime of a standard. This means that mature and ratified standards are less likely to change significantly or frequently, which reduces the risk of you having to modify your software in response.

Of course, if your software is specifically designed to implement and demonstrate an evolving standard then you have no choice but to accept the risk!

Question 7.3: Does your software provide tests to demonstrate compliance with open standards?

Anyone can claim compliance with an open standard, but tests allow this claim to be proven. By inspecting your tests, potential users can assess whether your software does indeed comply to the standard.

Compliance tests can also be used to assess whether other software complies to the standard. For example, the OGF Database Access and Integration Services Working Group used a single set of tests to assess whether two implementations of one of their standards complied with the standard.

Q8 - Building from source

Question 8.1: Do you provide instructions for building your software from the source-code distribution?

If there are no instructions for building your software, how will your users build it? At best, you'll end up dealing with lots of enquiries about building your software. At worst, you'll get no enquiries, nor any users.

Documenting the build helps you to acquire users.

Question 8.2: Do you build your software using an automated build (e.g. Make, ANT)?

Typing in lots of instructions is both time-consuming and prone to error. An automated build reduces the pain for your user, and reduces the risk of errors. Automated builds are also useful for developers: it allows them to easily compile your code and rebuild your software - after recoding to fix a bug, for example. An automated build is an important aspect of developing maintainable software.

Question 8.3: Does your source distribution list all third-party dependencies?

If your software has third-party dependencies, such as compilers or third-party software, then you must tell your user about them. You should also state which versions of the dependencies are needed. Nothing can be more irritating than starting a build only for it to fail - after an hour or so - due to a missing third-party package.

Question 8.4: Does your source distribution list the web address, suitable version number and licences for all third-party dependencies and state whether the dependencies are mandatory or optional?

Users don't want to have to search the websites of your third-party dependencies to find the information they need to run your software. You already know all the information that your users will need about suitable versions, licences and suchlike, so you should make it available to your users.

Licence information is very important, because users need to understand the terms and conditions of third-party dependencies so that they can determine whether they are legally permitted to use your software.

Question 8.5: Is dependency management used to download dependencies (e.g. ANT, Ivy, Maven)?

Bundling all third-party dependencies with your software means that your users don't need to download the dependencies themselves. However, it can lead to a very big release bundle and, in some cases, you will not be able to bundle a dependency, because its licence prevents it. Dependency management tools such as Maven or Ivy provide automated frameworks to download and install third-party dependencies at build time - reducing the size of release bundles, avoiding some licensing issues and saving users from having to download dependencies themselves.

Question 8.6: Do you provide tests to show whether the build has been successful?

Automated tests give a user confidence that your software has built correctly on their platform. If a test fails, the nature of the fail can help identify why. Automated tests can contribute to a fail-fast environment, which allows the rapid identification of failures introduced by changes to the code (e.g. a bug fix).

Automated tests are an important aspect of maintainable software. There are many tools available for the development of automated tests in a range of languages, such as JUnit for Java, CPPUnit for C++ or fUnit for Fortran.

Q9 - Installing the binary

Question 9.1: Is your binary distribution provided with installation instructions?

If a user can't install your software, how can they use it? Unless your software is a standalone EXE file or a single Linux/UNIX executable, then you need to provide installation instructions.

Question 9.2: Is dependency management used to automatically download dependencies (e.g. ANT, Ivy, Maven)?

Bundling all third-party dependencies with your software means that your users don't need to download the dependencies themselves. However, it can lead to a very big release bundle and, in some cases, you will not be able to bundle a dependency because its licence prevents it. Dependency management tools such as Maven or Ivy provide automated frameworks to download and install third-party dependencies at build time - reducing the size of release bundles, avoiding some licensing issues and saving users from having to download dependencies themselves.

Question 9.3: Does your binary distribution list all third-party dependencies?

It is useful to list the third-party dependencies distributed in your binary distribution. This primarily applies to cases where your binary distribution is a collection of your own and third-party dependencies (rather than a standalone executable). Listing the dependencies allows your users to upgrade them as newer versions become available, which means they can benefit from efficiency improvements or bug fixes.

Question 9.4: Do you provide tests to show that the install of the binary has been successful?

Automated tests give a user confidence that your software has installed correctly on their platform. If a test fails, the nature of the fail can help identify why. Automated tests are an important aspect of maintainable software. There are many tools available for the development of automated tests in a range of languages e.g. JUnit for Java, CPPUnit for C++ or fUnit for Fortran.

Alternatively, you could provide a list of steps that a user can take to validate the installation of the software. For deployment of a web-based application, this might just be checking that it the application is accessible via a browser.

Q10 - Testing

Question 10.1: Do you provide a unit test suite as part of your software distribution?

After changing your code and rebuilding it, a developer will want to check that their changes or fixes have not broken anything. Providing developers with a fail-fast environment allows for the rapid identification of failures introduced by changes to the code. Not having such an environment can dissuade developers from fixing, extending or improving your software.

Including a unit test suite allows them to do this. A test might verify an individual function or method, a class or module, related modules or components or the software as a whole. Tests can ensure that the correct results are returned from a function, that an operation changes the state of a system as expected, or that the code behaves as expected when things go wrong. Examples of unit test tools are JUnit for Java, CPPUnit for C++ or fUnit for Fortran.

Automating the run of a test suite means the entire set of unit tests can be run in one go, making life easier for the developer. Having an automated build system is a very valuable precursor to providing a test suite, and having an automated build and test system is a valuable resource in any software project.

For more on testing software see our guide on Testing your software.

Question 10.2: Is your software provided with an integration test suite as part of its distribution?

At a higher level than unit testing, an integration test suite works by automatically building and installing the entire software, and then running a set of high-level functional tests against that installation. This can prove very valuable in determining how new modifications affect the system as a whole while in full operation (albeit in a test environment). They also test how components of your software work together and can help in identifying problems with APIs.

For more on testing software see our guide on Testing your software.

Question 10.3: Do you support continuous integration by automatically running tests whenever changes are made to your source code?

Having an automated build and test system is a solid foundation for continuous integration testing. You can do this by performing an independent, automated build and test on a nightly basis, perhaps by scheduling a cron (or other automated) job on a non-critical machine.

For large-scale testing, you can also consider using a build and test infrastructure, based around many of the technologies available. Inca, for example, harnesses a number of machines through a central server to distribute a wide variety of tests in a parallel and scalable way.

Going further, this can also be done automatically whenever the source code repository changes. This means your team (and others if you publish the results more widely) obtain very rapid feedback on the impact of changes.

For more on testing software see our guide on Testing your software.

Question 10.4: Are the continuous integration test results publicly visible?

Publishing test results publicly from frequently run tests (e.g. nightly build and test runs) can give a community assurance about the software's testing procedures. It also means that feedback on recently contributed modifications can be verified on the core development team's infrastructure as not break existing functionality. This could be especially valuable if tests are run automatically in a nightly or continuous fashion.

You can consider automatically publishing test runs to your project website, or even have the test run results mailed to a developer's mailing list if the community deems it of value.

An example of a project that publishes their test runs is the particle physics project MAUS.

Q11 - Portability

Question 11.1: Is your software compatible with Windows, Unix/Linux AND Mac OS X?

Choosing which platforms to support is a balancing act. With each extra platform that you support, you will widen the pool of potential users for your software. Of course, the more platforms you support, the more platforms you'll have to maintain, develop, test and user-support your software against! Decide which platforms represent the most community value given your development and maintenance effort. And always target your community's platforms, don't make them move to yours.

Microsoft Windows should at least be considered for support. It is the world's most widely used operating system, and even in the Linux-heavy world of technical computing it may be an important platform for your software's user community. Even if your software has a Linux-specific server component, perhaps a Windows-based client would be a good idea. In addition, developing using languages and products designed for, or runnable under, Windows may yield more efficient and easier to install software than insisting on the pre-requisite deployment of a Linux emulator, for example.

 

Where possible, try to think about the technical expertise of your users given that Linux systems typically require a greater set of skills, especially when it comes to building and deploying software. On the other hand, keep in mind that Linux has a strong presence in the technical computing arena, e.g. on clusters, HPC platforms and clouds. Generally speaking, when considering portability you should try to understand how widely used Linux might be in your software's user community.

In most cases, Mac OS X might not be considered a core platform to support, given its relative low uptake compared to Windows and Linux based systems. But perhaps the Mac 'cool' factor has permeated your user community, and your software would only require small modifications to support it? Counting the number of Macs versus PCs at your community's favourite conferences should give you some pointers!

Question 11.2: Is your browser application compatible with Internet Explorer, Firefox, Chrome AND Safari?

No single browser has an overwhelming market share over the others, so if you want to maximise the uptake of you browser application, you should aim to support all of the major browsers. Your choice of browsers to support should be driven by what your community tends to use (and not what you favour!)

Windows platforms have Internet Explorer by default - that's a lot of browsers! IE has around a 34% market share, so it should probably be high on the browser support wish list. Mozilla Firefox has around a 23% market share and Google Chrome has around a 20% market share so both of these browsers should be supported too. Commonly used on Mac OS X platforms, but available on others, Safari may be viewed as less important given it only has around 11% market share, but 11% of the world browser market represents a lot of people.

Q12 - Community

Question 12.1: Does your website state how many users/developers/members are associated with your project?

Where you have an active set of users and/or developers, advertising their existence is not just good for promoting the success and high activity of your project. It also offers assurance to potential new users that the software fulfils its intended purpose, and that there's a reasonable chance of user and developer support should it be needed.

Question 12.2: Do you provide success stories on your website?

A great way of showing off your software is to write case studies about the people who've used it and how they've used it. This helps potential users learn about the software but, more to the point, is a great advert for your software. If you can show happy users benefiting from your software, you are likely to gain more users.

Question 12.3: Do you list your important partners and collaborators on your website?

Providing a list of active important partners and collaborators gives potential new software adopters valuable assurance that the software has a future. Also, the higher the scientific, academic or industrial reputation of those partners, the higher the perceived reputation of the software will be.

Publicly recognising partners' efforts in improving or working with the software also increases the likelihood they will work with your software in the future. Credit where credit is due!

Question 12.4: Do you list your project's publications on your website?

Listing your related software publications not only provides an academic perspective on the value of your software, it can also help users to understand, in detail, the context of your software in relation to others, or perhaps how your software has been used to solve complex scientific problems. It also gives researchers something to cite when they write their own papers where your software has been used, which is of value to them and also increases your citation count for your papers, which helps you demonstrate your impact!

Question 12.5: On your website, do you list third-party publications that cite your software?

Providing a list of any related third-party publications can show, academically, how the software is used by others, as well as promoting their efforts and successes. It also gives potential users ideas for how they may choose to use the software, as well as providing assurance that the software can be used by people other than its original developers to achieve something. Having such a list also means you can cite these publications in your own papers, funding proposals and reports to show or justify its value and the impact you have made!

As a matter of routine, you should always ask people to cite your software if they’ve used it in their research for these reasons, and to inform you if they have included such a reference in one of their papers.

Question 12.6: Do you make timely announcements of the deprecation of components, APIs, etc.?

It's never a good idea to remove components or features without informing your user community first. It could be there are users who are dependent on particular feature(s). So announcing such planned deprecations well in advance means users and developers can respond if a given feature is important to them.

If a feature is due to be superseded by a newer, better feature or component, including both for a suitable period within the software can allow the community to transition comfortably from the older version to the new version.

You could also consider developing and publicising a deprecation policy, stating how and when features or components in general are deprecated. This gives your community assurance that features will not be summarily removed without warning. see, for example the Eclipse API deprecation policy.

Q13 - Contributor policy

Question 13.1: Do you allow people to contribute to your software even though they are not part of your project?

If external developers, even technical writers, can contribute to your project, you can take advantage of the code or documentation fixes they have written themselves and wish to contribute. It also allows you to include code that introduces new features that may be desired and/or particularly useful. If managed correctly, this can yield improvements in your software with a minimum of overhead, providing a free source of effort for your software development and helping to evolve a community around your software.

For information on how to manage contributions, see OSS Watch's Contributor Licence Agreements.

Question 13.2: Is your contributions' policy publicly available?

Having a clearly stated contribution policy allows contributors to determine easily how they can send you code, the mechanisms and methods to do so, and to understand any coding or documentation conventions to which they should adhere.

For information on how to manage contributions, see OSS Watch's Contributor Licence Agreements.

Question 13.3: Do contributors to your project retain the copyright/IP of their contributions?

Asking contributors to sign over their copyright and intellectual property to your project or organisation can prove a deterrent. This, in effect, is asking them to give away ownership of something that may be novel and which may represent a key aspect of their research. Allowing contributors to keep their own copyright and intellectual property removes this barrier, thereby making contribution a more attractive option. It also contributes to promoting a community round the software - everyone sharing their products rather than handing it over to a select group of individuals.

For information on how to manage contributions, see OSS Watch's Contributor Licence Agreements.

Question 13.4: Are changes in the source code repository e-mailed to a mailing list?

Allowing developers to stay informed of changes to your code is important. If they're actively developing using your software, keeping them up to date with intended or on-going structural or other source-code changes enables them to factor this into their own development plans. It also helps to avoid any unintended, uninformed and unwanted changes negatively impacting their own efforts, providing the opportunity for community discussion and feedback about planned changes.

If the e-mails actually include the changes made (e.g. excerpts of the source code showing additions, removals and alterations) then this may promote more rapid identification of errors as any subscriber to the list may notice an issue in the changes made.

Question 13.5: Can anyone subscribe to the mailing list to see when changes have been made to your source code?

Keeping the source code changes mailing list as open as possible is always a good idea, and gives the positive impression of being inclusive. Keeping the process of joining as automated as possible also reduces, or even removes, your manual overhead for administering the list and means that potential subscribers aren't subject to long delays waiting for their membership to be approved.

Q14 - Identity

Question 14.1: Does your project/software have its own domain name?

It is now less important to have a domain name that mimics the name of your main product, because most people will use a web search to find a website rather than manually entering a URL. However, it is important to check that the domain that contains the name of your main product isn’t owned by another group, because this will confuse your users.

Question 14.2: Is the name of your project/software unique within its application area?

It’s fairly obvious that you shouldn’t choose a project or software name that is shared by another group - especially if they are a competitor - but few people spend enough time researching the uniqueness of their chosen name.

For guidance on choosing a good name, see our guide on Choosing project and product names.

Question 14.3: Is your project/software name free from trademark violations?

When you choose a name, it’s important to check that someone else hasn’t got there first and protected the name with a trademark. If they have, you run the risk of being sued for trademark infringement.

Running a quick trademark search is free and fairly straightforward. Details are provided in our guide on Choosing project and product names.

Q15 - Copyright

Question 15.1: Does your website clearly state the owner of the copyright for your software?

A clear statement of copyright on your website for your software provides a clear message to anyone using the software that you believe the software to be your property. As such, anyone visiting your website will know exactly who to contact if they have a question about adapting or licensing your software.

Question 15.2: Does each of your source-code files include a copyright statement?

It’s easy to distribute source-code across a team of collaborators, and this separates the code from any copyright statement that might have been displayed when the code was downloaded. To cover this eventuality, and remove any ambiguity about ownership, it's good practice to include a copyright statement with each of your source-code files.

Q16 - Licences

Question 16.1: Is your software (source and binaries) distributed under a licence?

Users need to know the licensing conditions of your software (and also any third-party software bundled with it) since this may impose constraints and obligations on how they can use or redistribute it. Developers need to know the conditions under which they can change or extend your software and any restrictions on their extensions. It is also useful for users and developers to know the provenance of any third-party software bundled in a release should they wish to create releases of their components that contain this third-party software.

If you don't have a licence, and an open-source licence is right for you, then see our guide on adopting an open source licence.

Question 16.2: Is the name of the licence under which your software is distributed clearly stated on your website?

Being able to view the licence for the software on your website, without having to download and unpack the software's source or binaries, means users can quickly determine if the licence is suitable for how they intend to use and/or develop the software.

Question 16.3: Is your software distributed under an open-source licence?

An open-source licence allows software provided in source form to be studied, used, modified, and redistributed to the extent specified by the licence. Open-source licences can be generic (e.g. GPL, BSD or Apache 2) or product-specific.

The selection of an appropriate open-source licence is important as the type of licence adopted may determine whether or not a potential user decides to use your software. See our guide on adopting an open source licence.

Question 16.4: Does each of your source-code files include a licence header?

Including a licence header at the top of source-code files acts as a convenient reminder for developers as to what they are permitted to do whilst modifying the code. It also serves to avoid confusion between source files that may have different licences, particularly if there are a number of third-party products in the distribution. Also, if a source code file becomes detached from the source distribution, the licence travels with it.