Parameter optimisation with mcasopt
===================================

Introduce the concept of mcasopt_:

* Defining solution criteria;
* Defining an error metric for MCAS outputs (the **objective function**);
* Using optimisation methods to search for improved parameter estimates; and
* How mcasopt_ ties this all together.

The mcasopt_ package uses a numerical optimisation method to find an approximately-optimal model solution over some subset of the model parameters.

This requires four things to be defined by the user:

* An input file that defines values for all of the model parameters;

* A set of compound states;

* A function that takes a vector of free parameters and updates the corresponding model parameters (i.e., determines the contents of a new MCAS input file); and

* A vector of initial values for these free parameters.

Optimisation methods
--------------------

The `SciPy <https://scipy.org/>`__ library provides a range of optimisation methods (see the `reference documentation <https://docs.scipy.org/doc/scipy/reference/optimize.html>`__ and `optimisation tutorial <https://docs.scipy.org/doc/scipy/tutorial/optimize.html>`__ for details).
Some of these methods only require function evaluations, some require calculating or estimating the objective function's Jacobian, and some require calculating the objective function's Hessian.
Depending on the method, bounds constraints, linear constraints, and/or non-linear constraints may also be provided.

Promising optimisation methods include the `Nelder-Mead simplex algorithm <https://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method>`__ and  `Powell's method <https://en.wikipedia.org/wiki/Powell%27s_method>`__.

.. note::

   Experimentation will likely be required to identify the optimal minimisation method.
   Because the :ref:`error metric <mcas-metric>` is discontinuous, we should avoid using optimisation methods that use the objective function's Jacobian and/or Hessian.

   We may also need to explore the **global optimisation methods** provided by SciPy, such as basin-hopping, differential evolution, simplicial homology global optimisation (SHGO), dual annealing, dividing rectangles (DIRECT), and even brute force (worst-case).

.. note::

   Also consider **genetic optimisation methods** provided by `pymoo <https://pymoo.org/>`__ and/or `pygad <https://pygad.readthedocs.io/en/latest/>`__.

.. note::

   Also consider `nevergrad <https://facebookresearch.github.io/nevergrad/>`__, a gradient-free optimiser from Facebook Research.