1. CESM-ECT (CESM Ensemble Consistency Test):

CESM-ECT is a suite of tests to determine whether a new simulation set up (new machine, compiler, etc.) is statistically distinguishable from an accepted ensemble. The verification tools in the CESM-ECT suite are:

CAM-ECT - detects issues in CAM and CLM (12 month runs) UF-CAM-ECT - detects issues in CAM and CLM (9 time step runs) POP-ECT - detects issues in POP and CICE (12 month runs)

The ECT process involves comparing runs generated with the new scenario ( 3 for CAM-ECT and UF-CAM-ECT, and 1 for POP-ECT) to an ensemble built on a trusted machine (currently cheyenne). The python ECT tools are located in the pyCECT subdirectory or https://github.com/NCAR/PyCECT/releases.

-OR-

We now provide a web server for CAM-ECT and UF-CAM-ECT, where you can upload the (3) generated runs for comparison to our ensemble. Please see the webpage at http://www.cesm.ucar.edu/models/cesm2/verification/ for further instructions.

1.1. Creating or obtaining a summary file:

Before the test can be run, a summary file is needed of the ensemble runs to which the comparison will be made. Ensemble summary files (NetCDF) for existing tags for CAM-ECT, UF-CAM-ECT, and POP-ECT that were created by CSEG are located (respectively) in the CESM input data directories:

$CESMDATAROOT/inputdata/validation/ensembles $CESMDATAROOT/inputdata/validation/uf_ensembles $CESMDATAROOT/inputdata/validation/pop_ensembles

If none of our ensembles are suitable for your needs, then you may create your own ensemble (and summary file) using the following instructions:

(1) To create a new ensemble, use the ensemble.py script in this directory. This script creates and compiles a case, then creates clones of the original case, where the initial temperature perturbation is slightly modified for each ensemble member. At this time, cime includes functionality to create ensembles for CAM-ECT, UF-CAM-ECT, and POP-ECT.

(2) Use –ect <pop,cam> to specify whether ensemble is for CAM or POP. (See ‘python ensemble.py -h’ for additional details).

(3) Use –ensemble <size> to specify the ensemble size. Recommended ensemble sizes: CAM-ECT: 151 UF-CAM-ECT: 350 POP-ECT 40

  1. Examples:

CAM-ECT:

python ensemble.py –case /glade/scratch/cesm_user/cesm_tag/ensemble/ensemble.cesm_tag.000 –mach cheyenne –ensemble 151 –ect cam –project P99999999

UF-CAM-ECT:

python ensemble.py –case /glade/scratch/cesm_user/cesm_tag/uf_ensemble/ensemble.cesm_tag.uf.000 –mach cheyenne –ensemble 350 –uf –ect cam –project P99999999

POP-ECT:

python ensemble.py –case /glade/scratch/cesm_user/cesm_tag/uf_ensemble/ensemble.cesm_tag.000 –mach cheyenne –ensemble 40 –ect pop –project P99999999

Notes:

  1. ensemble.py accepts (most of) the argumenets of create_newcase

  2. case name must end in “.000” and include the full path

  3. ensemble size must be specified, and suggested defaults are listed above. Note that for CAM-ECT and UF-CAM-ECT, the ensemble size needs to be larger than the number of variables that ECT will evaluate.

(5) Once all ensemble simulations have run successfully, copy every cam history file (.cam.h0.) for CAM-ECT and UF-CAM-ECT) or monthly pop history file (.pop.h.) for POP-ECT from each ensemble run directory into a separate directory. Next create the ensemble summary using the pyCECT tool pyEnsSum.py (for CAM-ECT and UF-CAM-ECT) or pyEnsSumPop.py (for POP-ECT). For details see README_pyEnsSum.rst and README_pyEnsSumPop.rst with the pyCECT tools.

1.2. Creating test runs:

(1) Once an ensemble summary file has been created or chosen to use from $CESMDATAROOT/inputdata/validation, the simulation run(s) to be verified by ECT must be created via script ensemble.py.

NOTE: It is important that the same resolution and compset be used in the individual runs as in the ensemble. The NetCDF ensemble summary file global attributes give this information.

  1. For example, for CAM-ECT:

python ensemble.py –case /glade/scratch/cesm_user/cesm_tag/camcase.cesm_tag.000 –ect cam –mach cheyenne –project P99999999 –compset F2000climo –res f19_f19 For example, for UF-CAM-ECT:

python ensemble.py –case /glade/scratch/cesm_user/cesm_tag/uf.camcase.cesm_tag.000 –ect cam –uf –mach cheyenne –project P99999999 –compset F2000climo –res f19_f19

For example, for POP-ECT:

python ensemble.py –case /glade/scratch/cesm_user/cesm_tag/popcase.cesm_tag.000 –ect pop –mach cheyenne –project P99999999 –compset G –res T62_g17

(3) Next verify the new simulation(s) with the pyCECT tool pyCECT.py (see README_pyCECT.rst with the pyCECT tools).