Testing is an integral part of the software development process. We want to catch mistakes early, before they go on to affect our results.

There are a lot of different types of software testing that exist. Most commonly, for scientific codes, we hear about:

• Unit testing : Tests that a single function does what it was designed to do

• Integration testing : Tests whether the individual pieces work together as intended. Sometimes done one piece at a time (iteratively)

• Regression testing : Checks whether code changes have changed answers

• Verification & Validation (from the science perspective)

  • Verification: are we solving the equations correctly?

  • Validation: are we solving the correct equations?

The best testing is automated. Github provides a continuous integration service that can be run on pull requests. You write a short definition (a Github workflow) that tells Github how to run your tests and then any time there is a change, the tests are run.

• When to write tests?

  • Some people advocate writing a unit test for a specification before you write the functions they will test

    • This is called Test-driven development (TDD): https://en.wikipedia.org/wiki/Test-driven_development

  • This helps you understand the interface, return values, side-effects, etc. of what you intend to write

• Often we already have code, so we can start by writing tests to cover some core functionality

  • Add new tests when you encounter a bug, precisely to ensure that this bug doesn’t arise again

• Tests should be short and simple

  • You want to be able to run them frequently

  • The more granular your tests are, the easier it will be to track down bugs