compare_test_resultsΒΆ
compare_test_results is a script in CIMEROOT/scripts/Tools.
$ ./compare_test_results --help
usage:
compare_test_results [-r <TESTROOT>] [-b <BRANCH> -c <COMPILER>] [-t <TESTID>] [<TEST> <TEST> ...] [--verbose]
OR
compare_test_results --help
[1mEXAMPLES:[0m
[1;32m# From most recent run, compare all changes [0m
> compare_test_results
[1;32m# From most recent run, compare only changes for test foo and bar only [0m
> compare_test_results foo bar
[1;32m# For an old run where you know test-id, compare only changes for test foo and bar only [0m
> compare_test_results foo bar -t mytestid
[1;32m# From most recent run of jenkins, compare history changes for next [0m
> compare_test_results -r /home/jenkins/acme/scratch/jenkins -b next
[1;32m# For typical CESM workflow, where baselines are named with tags [0m
> compare_test_results -t TESTID -b BASELINE_TAG
Analyze results from a test root area, comparing non-BFB changes. Purpose is,
instead of re-running tests in compare mode, which is very slow, allow for
very fast analysis of diffs. Outputs results for each test to stdout (one line
per test); possible status codes are: PASS, FAIL, SKIP. (A SKIP denotes a test
that did not make it to the run phase or a test for which the run phase did
not pass: we skip baseline comparisons in this case.) In addition, creates
files named compare.log.BASELINE_NAME.TIMESTAMP in each test directory, which
contain more detailed output. Also creates *.cprnc.out.BASELINE_NAME.TIMESTAMP
files in each run directory. Returns a 0 exit status if all tests are bit-for-
bit, and a non-zero exit status (TESTS_FAILED_ERR_CODE) if any tests differed
from the baseline. You may need to load modules for cprnc to work.
positional arguments:
compare_tests When comparing, limit the comparison to tests matching
these regex (default: None)
optional arguments:
-h, --help show this help message and exit
-d, --debug Print debug information (very verbose) to file
/glade/work/turuncu/APPS/ufs-mrweather-
app/cime/scripts/Tools/compare_test_results.log
(default: False)
-v, --verbose Add additional context (time and file) to log messages
(default: False)
-s, --silent Print only warnings and error messages (default:
False)
-n, --namelists-only Only analyze namelists. (default: False)
--hist-only Only analyze history files. (default: False)
-b BASELINE_NAME, --baseline-name BASELINE_NAME
Name of baselines to use. Default will use
BASELINE_NAME_CMP first if possible, otherwise branch
name. (default: None)
--baseline-root BASELINE_ROOT
Root of baselines. Default will use BASELINE_ROOT from
the case. (default: None)
-c COMPILER, --compiler COMPILER
Compiler of run you want to compare (default: intel)
-r TEST_ROOT, --test-root TEST_ROOT
Path to test results that are being compared (default:
/glade/scratch/turuncu)
-t TEST_ID, --test-id TEST_ID
Limit processes to case dirs matching this test-id.
Can be useful if mutiple runs dumped into the same
dir. (default: None)