CIME.XML package
Submodules
CIME.XML.archive module
Interface to the archive.xml file. This class inherits from GenericXML.py
CIME.XML.archive_base module
Base class for archive files. This class inherits from generic_xml.py
- class CIME.XML.archive_base.ArchiveBase(infile=None, schema=None, root_name_override=None, root_attrib_override=None, read_only=True)[source]
Bases:
GenericXML
- get_all_hist_files(casename, model, from_dir, suffix='', ref_case=None)[source]
gets all history files in directory from_dir with suffix (if provided) ignores files with ref_case in the name if ref_case is provided
- get_entry_value(name, archive_entry)[source]
get the xml text associated with name under root archive_entry returns None if no entry is found, expects only one entry
- get_hist_file_ext_regexes(archive_entry)[source]
get the xml text associated with each of the hist_file_ext_regex entries based at root archive_entry (root is based on component name) returns a list of text entries or an empty list if no entries are found
- get_hist_file_extensions(archive_entry)[source]
get the xml text associated with each of the hist_file_extensions based at root archive_entry (root is based on component name) returns a list of text entries or an empty list if no entries are found
CIME.XML.batch module
Interface to the config_batch.xml file. This class inherits from GenericXML.py
The batch_system type=”foo” blocks define most things. Machine-specific overrides can be defined by providing a batch_system MACH=”mach” block.
- class CIME.XML.batch.Batch(batch_system=None, machine=None, infile=None, files=None, extra_machines_dir=None)[source]
Bases:
GenericXML
- get_batch_jobs()[source]
Return a list of jobs with the first element the name of the case script and the second a dict of qualifiers for the job
- get_optional_batch_node(nodename, attributes=None)[source]
Return data on a node for a batch system
CIME.XML.component module
Interface to the config_component.xml files. This class inherits from EntryID.py
- class CIME.XML.component.Component(infile, comp_class)[source]
Bases:
EntryID
- get_valid_model_components()[source]
return a list of all possible valid generic (e.g. atm, clm, …) model components from the entries in the model CONFIG_CPL_FILE
- get_value(name, attribute=None, resolved=False, subgroup=None)[source]
Get a value for entry with id attribute vid. or from the values field if the attribute argument is provided and matches
- return_values()[source]
return a list of hashes from target config_component.xml file This routine is used by external tools in https://github.com/NCAR/CESM_xml2html
CIME.XML.compsets module
Common interface to XML files which follow the compsets format,
- class CIME.XML.compsets.Compsets(infile=None, files=None)[source]
Bases:
GenericXML
- get_compset_match(name)[source]
science support is used in cesm to determine if this compset and grid is scientifically supported. science_support is returned as an array of grids for this compset
- get_compset_var_settings(compset, grid)[source]
Variables can be set in config_compsets.xml in entry id settings with compset and grid attributes find and return id value pairs here
CIME.XML.entry_id module
Common interface to XML files which follow the entry id format, this is an abstract class and is expected to be used by other XML interface modules and not directly.
- class CIME.XML.entry_id.EntryID(infile=None, schema=None, read_only=True)[source]
Bases:
GenericXML
- add_elements_by_group(srcobj, attributes=None, infile=None)[source]
Add elements from srcobj to self under the appropriate group element, entries to be added must have a child element <file> with value “infile”
- get_default_value(node, attributes=None)[source]
Set the value of an entry to the default value for that entry
- get_value(vid, attribute=None, resolved=True, subgroup=None)[source]
Get a value for entry with id attribute vid. or from the values field if the attribute argument is provided and matches
- get_value_match(vid, attributes=None, exact_match=False, entry_node=None, replacement_for_none=None)[source]
Handle this case: <entry id …>
- <values>
<value A=”a1”>X</value> <value A=”a2”>Y</value> <value A=”a3” B=”b1”>Z</value>
</values>
</entry>
If replacement_for_none is provided, then: if the found text value would give a None value, instead replace it with the value given by the replacement_for_none argument. (However, still return None if no match is found.) This may or may not be needed, but is in place to maintain some old logic.
CIME.XML.env_archive module
Interface to the env_archive.xml file. This class inherits from EnvBase
CIME.XML.env_base module
Base class for env files. This class inherits from EntryID.py
- class CIME.XML.env_base.EnvBase(case_root, infile, schema=None, read_only=False)[source]
Bases:
EntryID
- get_children(name=None, attributes=None, root=None)[source]
This is the critical function, its interface and performance are crucial.
You can specify attributes={key:None} if you want to select children with the key attribute but you don’t care what its value is.
CIME.XML.env_batch module
Interface to the env_batch.xml file. This class inherits from EnvBase
- class CIME.XML.env_batch.EnvBatch(case_root=None, infile='env_batch.xml', read_only=False)[source]
Bases:
EnvBase
- get_children(name=None, attributes=None, root=None)[source]
This is the critical function, its interface and performance are crucial.
You can specify attributes={key:None} if you want to select children with the key attribute but you don’t care what its value is.
- get_queue_specs(qnode)[source]
Get queue specifications from node.
Returns (nodemin, nodemax, jobname, walltimemax, jobmin, jobmax, is_strict)
- get_value(item, attribute=None, resolved=True, subgroup=None)[source]
Must default subgroup to something in order to provide single return value
- set_value(item, value, subgroup=None, ignore_type=False)[source]
Override the entry_id set_value function with some special cases for this class
- submit_jobs(case, no_batch=False, job=None, user_prereq=None, skip_pnl=False, allow_fail=False, resubmit_immediate=False, mail_user=None, mail_type=None, batch_args=None, dry_run=False, workflow=True)[source]
no_batch indicates that the jobs should be run directly rather that submitted to a queueing system job is the first job in the workflow sequence to start user_prereq is a batch system prerequisite as requested by the user skip_pnl indicates that the preview_namelist should not be run by this job allow_fail indicates that the prereq job need only complete not nessasarily successfully to start the next job resubmit_immediate indicates that all jobs indicated by the RESUBMIT option should be submitted at the same time instead of
waiting to resubmit at the end of the first sequence
workflow is a logical indicating whether only “job” is submitted or the workflow sequence starting with “job” is submitted
- CIME.XML.env_batch.get_job_deps(dependency, depid, prev_job=None, user_prereq=None)[source]
Gather list of job batch ids that a job depends on.
Parameters
- dependencystr
List of dependent job names.
- depiddict
Lookup where keys are job names and values are the batch id.
- user_prereqstr
User requested dependency.
Returns
- list
List of batch ids that job depends on.
CIME.XML.env_build module
Interface to the env_build.xml file. This class inherits from EnvBase
CIME.XML.env_case module
Interface to the env_case.xml file. This class inherits from EnvBase
CIME.XML.env_mach_pes module
Interface to the env_mach_pes.xml file. This class inherits from EntryID
- class CIME.XML.env_mach_pes.EnvMachPes(case_root=None, infile='env_mach_pes.xml', components=None, read_only=False, comp_interface='mct')[source]
Bases:
EnvBase
- get_max_thread_count(comp_classes)[source]
Find the maximum number of openmp threads for any component in the case
CIME.XML.env_mach_specific module
Interface to the env_mach_specific.xml file. This class inherits from EnvBase
- class CIME.XML.env_mach_specific.EnvMachSpecific(caseroot=None, infile='env_mach_specific.xml', components=None, unit_testing=False, read_only=False, standalone_configure=False, comp_interface=None)[source]
Bases:
EnvBase
- allow_error()[source]
Return True if stderr output from module commands should be assumed to be an error. Default False. This is necessary since implementations of environment modules are highlty variable and some systems produce stderr output even when things are working fine.
- get_mpirun(case, attribs, job, exe_only=False, overrides=None)[source]
Find best match, return (executable, {arg_name : text})
- load_env(case, force_method=None, job=None, verbose=False)[source]
Should only be called by case.load_env
- make_env_mach_specific_file(shell, case, output_dir='')[source]
Writes .env_mach_specific.sh or .env_mach_specific.csh
Args: shell: string - ‘sh’ or ‘csh’ case: case object output_dir: string - path to output directory (if empty string, uses current directory)
CIME.XML.env_run module
Interface to the env_run.xml file. This class inherits from EnvBase
- class CIME.XML.env_run.EnvRun(case_root=None, infile='env_run.xml', components=None, read_only=False)[source]
Bases:
EnvBase
CIME.XML.env_test module
Interface to the env_test.xml file. This class inherits from EnvBase
- class CIME.XML.env_test.EnvTest(case_root=None, infile='env_test.xml', components=None, read_only=False)[source]
Bases:
EnvBase
- get_value(vid, attribute=None, resolved=True, subgroup=None)[source]
Get a value for entry with id attribute vid. or from the values field if the attribute argument is provided and matches
- set_initial_values(case)[source]
The values to initialize a test are defined in env_test.xml copy them to the appropriate case env files to initialize a test ignore fields set in the BUILD and RUN clauses, they are set in the appropriate build and run phases.
CIME.XML.env_workflow module
Interface to the env_workflow.xml file. This class inherits from EnvBase
- class CIME.XML.env_workflow.EnvWorkflow(case_root=None, infile='env_workflow.xml', read_only=False)[source]
Bases:
EnvBase
- get_children(name=None, attributes=None, root=None)[source]
This is the critical function, its interface and performance are crucial.
You can specify attributes={key:None} if you want to select children with the key attribute but you don’t care what its value is.
- get_value(item, attribute=None, resolved=True, subgroup='PRIMARY')[source]
Must default subgroup to something in order to provide single return value
CIME.XML.expected_fails_file module
Interface to an expected failure xml file
Here is an example:
<?xml version= “1.0”?>
- <expectedFails version=”1.1”>
- <test name=”ERP_D_Ld10_P36x2.f10_f10_musgs.IHistClm50BgcCrop.cheyenne_intel.clm-ciso_decStart”>
- <phase name=”RUN”>
<status>FAIL</status> <issue>#404</issue>
</phase> <phase name=”COMPARE_base_rest”>
<status>PEND</status> <issue>#404</issue> <comment>Because of the RUN failure, this phase is listed as PEND</comment>
</phase>
</test> <test name=”PFS_Ld20.f09_g17.I2000Clm50BgcCrop.cheyenne_intel”>
- <phase name=”GENERATE”>
<status>FAIL</status> <issue>ESMCI/cime#2917</issue>
</phase> <phase name=”BASELINE”>
<status>FAIL</status> <issue>ESMCI/cime#2917</issue>
</phase>
</test>
</expectedFails>
However, many of the above elements are optional, for human consumption only (i.e., not parsed here). The only required elements are given by this example:
<?xml version= “1.0”?>
- <expectedFails version=”1.1”>
- <test name=”…”>
- <phase name=”…”>
<status>…</status>
</phase>
</test>
</expectedFails>
- class CIME.XML.expected_fails_file.ExpectedFailsFile(infile)[source]
Bases:
GenericXML
CIME.XML.files module
Interface to the config_files.xml file. This class inherits from EntryID.py
CIME.XML.generic_xml module
Common interface to XML files, this is an abstract class and is expected to be used by other XML interface modules and not directly.
- class CIME.XML.generic_xml.GenericXML(infile=None, schema=None, root_name_override=None, root_attrib_override=None, read_only=True)[source]
Bases:
object
- class CacheEntry(tree, root, modtime)
Bases:
tuple
- modtime
Alias for field number 2
- root
Alias for field number 1
- tree
Alias for field number 0
- DISABLE_CACHING = False
- get_children(name=None, attributes=None, root=None)[source]
This is the critical function, its interface and performance are crucial.
You can specify attributes={key:None} if you want to select children with the key attribute but you don’t care what its value is.
- get_resolved_value(raw_value, allow_unresolved_envvars=False, subgroup=None)[source]
A value in the xml file may contain references to other xml variables or to environment variables. These are refered to in the perl style with $name and $ENV{name}.
>>> obj = GenericXML() >>> os.environ["FOO"] = "BAR" >>> os.environ["BAZ"] = "BARF" >>> obj.get_resolved_value("one $ENV{FOO} two $ENV{BAZ} three") 'one BAR two BARF three' >>> obj.get_resolved_value("2 + 3 - 1") '4' >>> obj.get_resolved_value("0001-01-01") '0001-01-01' >>> obj.get_resolved_value("$SHELL{echo hi}") == 'hi' True
- get_value(item, attribute=None, resolved=True, subgroup=None)[source]
get_value is expected to be defined by the derived classes, if you get here the value was not found in the class.
- lock()[source]
A subclass is doing caching, we need to lock the tree structure in order to avoid invalidating cache.
- read(infile, schema=None)[source]
Read and parse an xml file into the object. The schema variable can either be a path to an xsd schema file or a dictionary of paths to files by version.
- scan_child(nodename, attributes=None, root=None)[source]
Get an xml element matching nodename with optional attributes.
Error unless exactly one match.
- scan_optional_child(nodename, attributes=None, root=None)[source]
Get an xml element matching nodename with optional attributes.
Return None if no match.
- set_value(vid, value, subgroup=None, ignore_type=True)[source]
ignore_type is not used in this flavor
CIME.XML.grids module
Common interface to XML files which follow the grids format, This is not an abstract class - but inherits from the abstact class GenericXML
- class CIME.XML.grids.Grids(infile=None, files=None, comp_interface=None)[source]
Bases:
GenericXML
CIME.XML.headers module
Interface to the config_headers.xml file. This class inherits from EntryID.py
- class CIME.XML.headers.Headers(infile=None)[source]
Bases:
GenericXML
CIME.XML.inputdata module
Interface to the config_inputdata.xml file. This class inherits from GenericXML.py
- class CIME.XML.inputdata.Inputdata(infile=None, files=None)[source]
Bases:
GenericXML
CIME.XML.machines module
Interface to the config_machines.xml file. This class inherits from GenericXML.py
- class CIME.XML.machines.Machines(infile=None, files=None, machine=None, extra_machines_dir=None, read_only=True)[source]
Bases:
GenericXML
- get_field_from_list(listname, reqval=None, attributes=None)[source]
Some of the fields have lists of valid values in the xml, parse these lists and return the first value if reqval is not provided and reqval if it is a valid setting for the machine
- get_first_child_nodes(nodename)[source]
Return the names of all the child nodes for the target machine
- get_value(name, attributes=None, resolved=True, subgroup=None)[source]
Get Value of fields in the config_machines.xml file
- probe_machine_name(warn=True)[source]
Find a matching regular expression for hostname in the NODENAME_REGEX field in the file. First match wins.
- return_values()[source]
return a dictionary of machine info This routine is used by external tools in https://github.com/NCAR/CESM_xml2html
CIME.XML.namelist_definition module
Interface to namelist_definition.xml.
This module contains only one class, NamelistDefinition, inheriting from EntryID.
- class CIME.XML.namelist_definition.CaseInsensitiveDict(data)[source]
Bases:
dict
Basic case insensitive dict with strings only keys. From https://stackoverflow.com/a/27890005
- proxy = {}
- class CIME.XML.namelist_definition.NamelistDefinition(infile, files=None)[source]
Bases:
EntryID
Class representing variable definitions for a namelist. This class inherits from EntryID, and supports most inherited methods; however, set_value is unsupported.
Additional public methods: - dict_to_namelist. - is_valid_value - validate
- dict_to_namelist(dict_, filename=None)[source]
Converts a dictionary of name-value pairs to a Namelist.
The input is assumed to be similar to the output of parse when groupless=True is set. This function uses the namelist definition file to look up the namelist group associated with each variable, and uses this information to create a true Namelist object.
The optional filename argument can be used to assist in error reporting when the namelist comes from a specific, known file.
- get_default_value(item, attribute=None)[source]
Return the default value for the variable named item.
The return value is a list of strings corresponding to the comma-separated list of entries for the value (length 1 for scalars). If there is no default value in the file, this returns None.
- get_value_match(vid, attributes=None, exact_match=True, entry_node=None)[source]
Return the default value for the variable named vid.
The return value is a list of strings corresponding to the comma-separated list of entries for the value (length 1 for scalars). If there is no default value in the file, this returns None.
- is_valid_value(name, value)[source]
Determine whether a value is valid for the named variable.
The value argument must be a list of strings formatted as they would appear in the namelist (even for scalar variables, in which case the length of the list is always 1).
- set_nodes(skip_groups=None)[source]
populates the object data types for all nodes that are not part of the skip_groups array returns nodes that do not have attributes of skip_default_entry or per_stream_entry
- split_type_string(name)[source]
Split a ‘type’ attribute string into its component parts.
The name argument is the variable name. This is used for error reporting purposes.
The return value is a tuple consisting of the type itself, a length (which is an integer for character variables, otherwise None), and the size of the array (which is 1 for scalar variables).
CIME.XML.pes module
Interface to the config_pes.xml file. This class inherits from GenericXML.py
- class CIME.XML.pes.Pes(infile, files=None)[source]
Bases:
GenericXML
CIME.XML.pio module
Class for config_pio files . This class inherits from EntryID.py
CIME.XML.standard_module_setup module
CIME.XML.stream module
Interface to the streams.xml style files. This class inherits from GenericXML.py
stream files predate cime and so do not conform to entry id format
- class CIME.XML.stream.Stream(infile=None, files=None)[source]
Bases:
GenericXML
CIME.XML.test_reporter module
Interface to the testreporter xml. This class inherits from GenericXML.py
CIME.XML.testlist module
Interface to the config_files.xml file. This class inherits from generic_xml.py It supports version 2.0 of the testlist.xml file
In version 2 of the file options can be specified to further refine a test or set of tests. They can be specified either at the top level, in which case they apply to all machines/compilers for this test:
- <test …>
- <options>
<option name=”wallclock”>00:20</option>
</options> …
</test>
or at the level of a particular machine/compiler:
- <test …>
- <machines>
- <machine …>
- <options>
<option name=”wallclock”>00:20</option>
</options>
</machine>
</machines>
</test>
Currently supported options are:
walltime: sets the wallclock limit in the queuing system
memleak_tolerance: specifies the relative memory growth expected for this test
comment: has no effect, but is written out when printing the test list
workflow: adds a workflow to the test
- class CIME.XML.testlist.Testlist(infile, files=None)[source]
Bases:
GenericXML
CIME.XML.tests module
Interface to the config_tests.xml file. This class inherits from GenericEntry
- class CIME.XML.tests.Tests(infile=None, files=None)[source]
Bases:
GenericXML
CIME.XML.testspec module
Interface to the testspec.xml file. This class inherits from generic_xml.py
CIME.XML.workflow module
Interface to the config_workflow.xml file. This class inherits from GenericXML.py
- class CIME.XML.workflow.Workflow(infile=None, files=None)[source]
Bases:
GenericXML