7. MADS Assisted Calibration
This is an new and evolving approach to calibraton. The process described here generally follows the conceptual outline described in the Calibration Process section, but with some portions being done iteratively and or in an automated fashion.
Note
The MADS Assisted Calibration process has been a group effort led by Helene Genet and Elchin Jafarov with contributions and testing from the team at Woodwell Climate Research Center as well as the UAF Arctic Ecosystem Modeling Group, including:
Tobey Carman
Ruth Rutter
Aiza Kabeer
Andrew Mullen
Ben Maglio
Joy Clein
Valeria Briones
Eugenie Euskirchen
The description in the Calibration Process section is a guideline from the most susccessful of the manual calibration efforts. The new part of the approach presented here involves using the following flow diagram to carry out each step of the process.
There are two distinct parts of the workflow: the Sensitivity Analysis (SA; orange boxes, left side) and the MADS Assist (MADS; blue boxes, right side). In many cases it is reccomened to use Sensitivity Analysis before MADS to better constrain the search space. Conducting a full calibration of a new site will involve going through this flow chart many times with different config files and settings that increase the ecosystem complexity - following the steps in Calibration Process.
7.1. Sensitivity Analysis (SA)
The first part of the SA is to setup your config file. The config file is where you will choose which parameters to run the analysis for and which targets (outputs) you will need for comparison. Additionally in the config file you will specify the PFTs that will be used for each of the PFT specific parameters (and targets).
There are other settings available to you as well concerning the number of years to run each stages , the site to run, and the working directory to you would like the runs to take place in.
Note
We are working on defining a reccomended directory structure for organizing your work.
Once you have adjusted the config file to your liking, then next step is using
the SA_setup_and_run.py tool. There are several command line options with
this script allowing you to control the number of samples and the sampling
method used to generate the parameter samples.
When the SA_setup_and_run.py tool has completed, you will have the following
files:
sample_matrix.csv
targets.csv
param_props.csv
results.csv
With the above files in hand the user can conduct an equlibrium check and any
other pos hoc analysis using the SA_post_hoc_analysis.py tool. The tool
provides a number of plotting functions that will help the user assess whether
or not the modeled output is within acceptable range of the targets.
See here for examples.
7.2. MADS Assisted Calibration (CA)
Once you have completed the SA portion, you can move on to the CA (calibration) portion of the workflow. In this step you use a minimization algrorithm to find an optimal set of parameters such that the discrepancy between model outputs and target (observed) values is minimized.
The actual minimization alrorightm is provided by the Mads Library.
Simiar to the SA part of the workflow you will provide your settings in a config file and then feed that file to a script that will carry out the optimization. At the end you will have an optimal set of parameters as well as some metadata about the optimizaiton process that you can use to reason about the quality of the optimization.