Testing mcasopt
===============

In order to ensure that this package functions correctly, we will build a
suite of test cases.

The first step will be to extract the job-spawning logic into a single,
self-contained class, and define a simple interface for the function that is
being optimised.

We can then implement this interface for a range of functions and types of
computations, in order from simplest to most complex:

1. A simple mathematical function implemented in Python. For example:

   .. code:: python

      lambda x, y: abs((x - 2)**3) + (y - 4)**2

2. A simple mathematical function (e.g., as above) but implemented as a Python
   **script** that will be run in a separate process.

3. As per 2, but where the script delays the computation for, say, 2 seconds.

4. As per 2, but where we terminate the optimiser after every :math:`N`
   function evaluations, to simulate a long-running slurm job being
   terminated, and to test that the cache works correctly.

5. A simple mathematical function, implemented as a Python script, that reads
   parameter values from an MCAS input file and writes an MCAS output file,
   where for :math:`N` parameters it introduces an error in each of the first
   :math:`N` compound states.
   This will ensure that the error metric is being evaluated correctly.