|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
PrepIFS GUI client |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Graphical user interface functionality.Description of features for defining configurations.
Version 1.0 Revision History
Table of contents
IntroductionThis document is intended as an introduction for a model developer in how to use PrepIFS to configure his model. It explains the hierarchy of experiments at ECMWF and how the tool can help the modeller and the user to compose a correct setup. TerminologyThe general hierarchy of an experiment is as follows:
Experiment
!
!--- ExperimentType X (forecast, analysis etc)
!
!--- ExperimentType Y
!
!--- ExperimentType Z
!
!
!---NameList A (Archiving, post-processing etc)
!
!---NameList B
!
!---NameList C
!
!
!---------Configurable variable I (Archive name, days2run, etc)
!
!---------Configurable variable J
!
!---------Configurable variable K
!
!---------Configurable variable L
The word ``variable'' refers to an object in the GUI applications data model representing any configurable
variable in the experiment. Also referred to as making up the fact base
The word ``namelist'' refers to a collection of variables with related functionality. The word ``experimenttype'' refers to a collection of name lists representing a configuration of a type of experiment such as a forecast model. A ``check'' and ``checking'' means the interpretation of a series of rules by the Verifier engine returning the result as 0 or 1, true or false. The ``Verifier'' is the software component responsible for translating and executing the rules of the check, the rules engine. By ``defaults'' we mean the set of configurable variables created by the modeller/author for his model and the values they are given initially. These values together with a rule definition file makes up a valid configuration, i.e. one that will deploy correctly. The experiment hierarchy is subject to change in the PRISM project. Input to the system
The PrepIFS application reads files specifying the variables to configure(A) and
the values the user specified for these variables(B).
The PrepIFS application writes files with the values specified by the user (B) and
files needed for deployment of the model (C).
In the document the person that creates the files (A) specifying the variables to configure is the author and the person who configures the variables,i.e. the user, is termed the composer.
The file A contains information on every variable to configure. The information is declared as elements in the XML file. A special element guiattribute is optional and can add a specific behaviour for the variable. In the GUI each variable is treated as an object and the properties are the object attributes. An example from a defaults file is given below:
<?xml version = "1.0"?>
<?xml-stylesheet href="./namelist-html.xsl" type="text/xsl"?>
<?cocoon-process type="xslt"?>
<?xml-stylesheet type="text/css" href="../namelist.css"?>
<modelnamelist
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xhtml="http://www.w3.org/1999/xhtml"
xmlns="http://www.ecmwf.int/services/prepifs/"
xsi:schemaLocation="http://www.ecmwf.int/services/prepifs/ ../NameListSchema.xsd"
name= "Archiving">
<!-- -->
<description>Archive parameters
</description>
<!-- -->
<dependency><xhtml:a xhtml:href="VariationalAnalysis.xml"> Variational Analysis<
/xhtml:a>,<xhtml:a xhtml:href="Forecast.xml"> Forecast</xhtml:a>
</dependency>
<!-- -->
<info>Unless data is archived to MARS, it cannot be accessed once it has been deleted from
thefields data base.
</info>
<!-- -->
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<ionamelist>
<type>STRING</type>
<name>FILEFORMAT</name>
<default>Archiving</default>
<label><![CDATA[Archiving]]></label>
<description>FILEFORMAT</description>
<dependency> any depend </dependency>
<guiattribute key="COMMENT">Archive parameters</guiattribute>
<guiattribute key="FILEOUT">config.h</guiattribute>
<guiattribute key="FORMAT">ShellFormat</guiattribute>
<guiattribute key="VIS">false</guiattribute>
<guiattribute key="WRITE">false</guiattribute>
</ionamelist>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<ionamelist>
<type>BOOLEANWORD</type>
<name>DO_ARCHIVE</name>
<default>true</default>
<label><![CDATA[Archive fields to MARS]]></label>
<description>DO_ARCHIVE is the highest level archiving control switch.
If DO_ARCHIVE=false,no MARS archiving will be done, even if the other archiving switches
are set to true. DO_ARCHIVE also controls all the data assimilation archiving, apart
from the long forecasts, ie <xhtml:a xhtml:href="Forecast.xml#FCLENGTH">
FCLENGTH</xhtml:a> > <xhtml:a xhtml:href="VariationalAnalysis.xml#PERIOD_4D">
PERIOD_4D</xhtml:a>
</description>
<dependency> any depend </dependency>
<guiattribute key="FILEOUT">config.h/sms_def</guiattribute>
<guiattribute key="FORMAT">ShellFormat/SMSFormat</guiattribute>
</ionamelist>
<check><![CDATA[(this.sizeOfVector("PPSTEPS")<=1)?(BRF=PPFRQ)]]></check>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
</modelnamelist>
All variables have a name and a type.
The types of variables are matched to how we like them to be displayed
in the gui.All variables of the same type are displayed the same, the author
has no control over how the variable is displayed. This ensures that the composers
will encounter a uniform user interface.
The current set of objects used to configure the variables are linked to the use of the variables: FORTRAN,shell and SMS configurations. The following types exists:
The types are rendered as in the figure below: The file used to render the above configuration looks like this Specific variations of a basic type such as INTEGER is made by extending the basic type with the required behaviour. New types can be added by providing a java class with the required interface. The importance of the type lies in how basic validity checking is performed automatically, how values are assigned internally and how they are displayed on the screen. The output of a value is decoupled from the type and explained in the next section. Other properties of a variable that are not required are:
Output from the system
Output of variables to files is directed by a special variable in the namelist,
i.e. grouping of variables named FILEFORMAT.
This variable defines the default file and format of all variables in the namelist
and also the heading displayed in the GUI for the namelist.
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
<ionamelist>
<type>STRING</type>
<name>FILEFORMAT</name>
<default>Archiving</default>
<label><![CDATA[Archiving]]></label>
<description>FILEFORMAT</description>
<dependency> any depend </dependency>
<guiattribute key="FILEOUT">config.h</guiattribute>
<guiattribute key="FORMAT">ShellFormat</guiattribute>
<guiattribute key="VIS">false</guiattribute>
<guiattribute key="WRITE">false</guiattribute>
</ionamelist>
<!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -->
As short explanation of the attributes:
If other variables needs to go to other files or to more than one file these can be specified by adding the FILEOUT and FORMAT properties to the variable in question. All formats implement the interface FormatTypeInterface. Currently the following formats are implemented:FORTRANFormat,SMSFormat and SHELLFOrmat. An example of a real file used at ecmwf is here: Dates.xml Composition constraintsThe GUI tries to limit the user input at composition time in two ways:
To make sure that the experiment later submitted does not have conflicting options or simple typographical errors we apply consistency checking routines. These routines tests the validity of the configuration of the experiment as a whole. The enforcement of constraints takes the form of an execution of a number of rules that the developer or administrator has designed to make sure that the configuration is consistent and correct at every level in the hierarchy. Definition of constraints for the composition
The rule baseThe constraints, we call them checks, are defined in files named ``*.check'',i.e. an.check. They are partitioned on the hierarchy level which is associated with the activation strategy for a rule but also to allow the separation of rules for different types, cycles etc. There is a rule file for the the cycle, experiment type and the Namelist.xml file as can be seen here.In the namelist files the declaration format is enclosed in xml markup but in the hierarchy files a simpler format is used. Their location is defined by the property DEFAULTSHOME (see the property table). The rules consist of a left hand side (LHS), the "if" part and a right hand side (RHS), the "then" part. The rules in PrepIFS are forward-chained, i.e. they are executed sequentially with the RHS firing if the LHS is true. The rules engine does not try to fire rules actively. An important aspect of rule writing is to make the rules reentrant so that no matter how many times they are activated the resulting composition will always be correct. Constraints are called checks in PrepIFS. At ECMWF the rule base today consists of around 1000 rules. The fact basePrepIFS's working memory contains the facts on which the rules operate. The facts in this case is what makes up the composition, i.e. all the variables and their values in all namelists for all experiment types. This can be, for a forecast type for example, around 700 variables. For a full experiment roughly 4000 variables. Added to this there is a dictionary for storing messages in different languages, see the property table The addressing scheme for the facts is explained later, in addressing. Below are examples of checks that will be referred to throughout this text. Line numbers are inserted for later reference. 1 (($STATUS=-1)&&($ERR=$AN_FC_INCONSISTENT_RESOLWARN)); 2 (an@Forecast@RUN_GFC=="true")?((fc@namnmi@LNMIRQ="false")&&(fc@namini@NEINI="0")); 3 (an@Forecast@RUN_GFC=="true")?((an@Archiving@LRHLP=fc@archive@LRHLP)&& 4 (an@Archiving@LRPLP=fc@archive@LRPLP)&& 5 (an@Archiving@LRSUP=fc@archive@LRSUP)&& 6 (an@Archiving@LRPVP=fc@archive@LRPVP)&& 7 (an@Archiving@LRPTP=fc@archive@LRPTP)); 8 (fc@wavgeneral@WAVE != an@wavgeneral@WAVE)?(this.addCheckWarning($AN_FC_INCONSISTENT_WAVE)); 9 ( ppgeneral@ETA = ecmwf.pifs.checkPrepIFS.CheckUtils.makeList(1.0,general@LEVELS,1.0));Checks are specified using the PrepIFS check language which is very similar to writing expressions in C or Java as can be seen above. Syntax of the languageThe syntax of the language is :
( expr1 ) ? (expr2) [ : (expr3) ] ;
{ expr1 } ? (expr2) ;
or in words :
if expr1 then expr2 else expr3
or
while expr1 do expr2
Expressions
The expression is an expression as in Java or C supporting the following operators:
Added to the operators is the ability to call functions in the experiment object or from static libraries. How to write checks, internal representation and addressingWriting checks you have access to two kinds of symbols.
Experiment values
Variables in the namelists are the ones you want to check. Internally
all variables are typed to numeric ones (floats) or texts (strings).
The typing mechanism prevents assignments between the two.
The types mapped are the following :
All types support vector arithmetic automatically and checks are performed on the existence of indices. Indices start at 0. Operations involving vectors of different size are not permitted. When setting a value for a vector each index is separated with a slash (/).
Example with integer array IA.
IA=2/4 -> 2/4
IA=IA+IA -> 4/8
IA[1]=1 -> 2/1
IA=this.MakeListOfFiveVariables() -> 1/2/3/4/5
IA=IA+3 -> 5/7
Property table
Property table values are loaded with the client interface and define values with special meanings. The above variables are defined. User level property
Setting the USERLEVEL changes the way assignments are done and should not be done
from the checks. The variable is there to help you
write your checks so that different users get the right kind of support.
The possible values are novice or expert.
The assignments are carried out according to :
Showing messages to the user and signalling errors
Showing messages in the user interface check window is done by calling
functions with predefined error messages.
The $WARNTYPE is a predefined message that will be displayed in the language set by the client. To halt the checks and signal an error you set the STATUS and ERR as shown on line 1 in the example. Defining messages to the user
Messages are defined in property files and loaded with the client GUI.
There is one file for each language. Currently only one language has been
implemented:English.
AN_FC_INCONSISTENT_RESOLWARN: \ WARNING: Inconsistent setting of the horizontal resolution in 'an' and 'fc'. The syntax being: <ERRORNAME> ``:'' <Text to be shown> Addressing variables using namespaces
Each hierarchy in the experiment defines a namespace. This means that variables with
the same name can coexist in different places without a conflict because they have
different scope, i.e. they belong to different namespaces. PrepIFS determines the
namespace and addressing as follows:
For example, when a rule is declared in the namelist file the scope is set to the namelist and all variables declared in the namelist can be addressed with their name only.Variables in other namelist or experiment types are prefixed as described by the format in the table above. Addressing property table values is done by prefixing with a dollar sign $. Function callsFunction calls can be static or dynamic.
Example Static call
ecmwf.pifs.checkPrepIFS.CheckUtils.compareDates(inidata@INITIME[0],$DATE_WAVE)
+---------------------path--------++-function-++parameters-------------------+
This also demonstrate the use of property values and experiment values.
Example Dynamic Call
this.addValueToVector("148","ppgeneral@SFCFIELDS"))
The above call adds value 148 to variable SFCFIELDS in namelist ppgeneral.
The experiment type is determined by context.
Special array exampleThe while statement was introduced in 1.6 to allow for proper array and structlist support. It was primarily a question of supporting structlists which are the "table" type in prepIFS. I will illustrate with an example from the GEMS project:
1 // GEMS
2 //($DEBUGCHECKS="1");
3 // diagnostic extra fields from YGRG_NL to D3GGFIELDSS
4 // Add to MFP3DFSSet and add extra ppfields converted
5 {Gems@NGRGTEMP < ecmwf.pifs.checkPrepIFS.CheckUtils.toFloat(Gems@NGRG) } ?
6 (Gems@NGRGTEMP2=this.getSTRUCTLISTValueAt("Gems@YGRG_NL",Gems@NGRGTEMP,1) &&
7 (this.addValueToVector(Gems@NGRGTEMP2,"namfpc@MFP3DFS")) &&
8 (Gems@NGRGTEMP=Gems@NGRGTEMP+1) );
The full namelist is available here for reference to let you see how the different variables are declared. The two temporary variables are also declared in the namelist as there is currently no way of declaring variables in the check. Here follows a line by line comment to the example. The purpose of the code is to loop over all rows in YGRG_NL retrieving the value in column 1 and adding it to a vector MFP3DFS
To support initialisation of STRUCTLISTS to other values than the ones declared as components of the STRUCT two new attributes have been introduced: <guiattribute key="INITIALISE">CO2/210061/true/false/false/CH4/210062/true/false/false</guiattribute> <guiattribute key="NAMING">TYPEDEF</guiattribute>See here for an example. Existing Dynamic libraries
Existing Static libraries
Writing your own libraries
Adding dynamic functions is done by adding methods to the class ecmwf.datacontainers.ExpVer.
This is straightforward as this implements the data model of the experiment and you will
have access to all variables in the experiment through this class.
Adding static function means adding methods to the class ecmwf.checkPrepIFS.CheckUtils.
This is a static class and so all references to variables in the experiment will have to
be done as input parameters.
Making it workIt is not practical to test that your checks are operating as intended by starting the GUI every time as this is time consuming. Before doing that you can run the ``Verifier'' that will run each check and make sure it works with the default configuration. Debug and printingRunning the verifier is done as follows:
$java ecmwf.pifs.checkPrepIFS.Verifier
U S A G E :
serverurl : http://xxx.xx.xxx OR XX (run local)
userid : name of prepifs user
lang : en or other langugage that you have implemented
defaultpath : path to location of default files
at ecmwf = /home/rd/rdx/defaults
at prism = $DEFAULTSLOCATION from system.h
userpath : where your save files are stored
at ecmwf = /home/rd/$USER/prepifs/saves
cycle : name of cycle to load
at ecmwf = ex 29r1
at prism = ex prismV1
experimentid : Name of the experimentid to load
ex em58
type : Type of experiment to load
at ecmwf = ex fc
at prism = ex toyclim
configdir : location of configfiles
at ecmwf = run ~rdx/bin/prepIFS local and use the
output 'arguments: configdir'
at prism = run $PRISM_HOME/bin/prepIFS local and use the
output 'arguments: configdir'
All arguments needs to be supplied always.
An actual run could look like:
+ java ecmwf.pifs.checkPrepIFS.Verifier XX\ nal\ en\ /home/rd/rdx/defaults\ /home/rd/nal/prepifs/saves\ 29r1\ em58\ fc\ /tmp arguments: XX arguments: nal arguments: en arguments: /home/rd/rdx/defaults arguments: /home/rd/nal/prepifs/saves arguments: 29r1 arguments: em58 arguments: fc arguments: /tmp %INFO #Apr 22 11:04:26 BST 2005#fjorgyn/136.156.112.141# PID:0 # in DawnConfigure.java#189 Server[136.156.112.141] Configuring from /tmp/ecmwf.pifs.rmi.PifsServer.ini * - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - * reading ... /home/rd/rdx/defaults/29r1 reading ... /home/rd/nal/prepifs/saves Simplechecks are:true ET::VERIFY[fc@Description]------------------------------------------------ Parse line:( DESCRIPTION == "" )?(($STATUS=-1)&&($ERR=$NODESCRIPTION)) verifyline instance=em58 ( DESCRIPTION == "" ) => FALSE NO ELSE part VERYFYLINE *** Status -> 0 ET::VERIFY[fc@inidata]------------------------------------------------ Parse line:((INIPATH=="")&&(INIORIG=="ecfs"))?(($STATUS=-1)&&($ERR=$INIORIGERR)) verifyline instance=em58 ((INIPATH=="")&&(INIORIG=="ecfs")) => FALSE NO ELSE part VERYFYLINE *** Status -> 0 Parse line:( USE_DATES_LIST=="false" )? ( INITIME = ecmwf.pifs.checkPrepIFS.CheckUtils.makeDates(INIBEGINDATE,INIENDDATE,INIBY) ) verifyline instance=em58 ( USE_DATES_LIST=="false" ) => TRUE Doing then part... THEN ->( INITIME = ecmwf.pifs.checkPrepIFS.CheckUtils.makeDates(INIBEGINDATE,INIENDDATE,INIBY) ) => TRUE VERYFYLINE *** Status -> 0 Parse line:( USE_DATES_LIST=="true" )?( INIENDDATE=INITIME[0] ) verifyline instance=em58 ( USE_DATES_LIST=="true" ) => FALSE NO ELSE part VERYFYLINE *** Status -> 0 Parse line:( ecmwf.pifs.checkPrepIFS.CheckUtils.compareDates(INIBEGINDATE,INITIME[0]) != 0 )? ( INIBEGINDATE = INITIME[0] ) verifyline instance=em58 ( ecmwf.pifs.checkPrepIFS.CheckUtils.compareDates(INIBEGINDATE,INITIME[0]) != 0 ) => FALSE NO ELSE part VERYFYLINE *** Status -> 0 Parse line:( INIBEGINSECONDS = ecmwf.pifs.checkPrepIFS.CheckUtils.Hours(INIBEGINDATE)*3600 ) verifyline instance=em58 ( INIBEGINSECONDS = ecmwf.pifs.checkPrepIFS.CheckUtils.Hours(INIBEGINDATE)*3600 ) => TRUE etc,etcIf an error is detected execution stops:
+ java -version
java version "1.4.1_02"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.1_02-b06)
Java HotSpot(TM) Client VM (build 1.4.1_02-b06, mixed mode)
+ java ecmwf.pifs.checkPrepIFS.Verifier XX nal en /home/rd/rdx/defaults \
/home/rd/nal/prepifs/saves 29r1 em58 fc /tmp
arguments: XX
arguments: nal
arguments: en
arguments: /home/rd/rdx/defaults
arguments: /home/rd/nal/prepifs/saves
arguments: 29r1
arguments: em58
arguments: fc
arguments: /tmp
%INFO #Apr 22 11:09:51 BST 2005#fjorgyn/136.156.112.141# PID:0 # in DawnConfigure.java#189
Server[136.156.112.141] Configuring from /tmp/ecmwf.pifs.rmi.PifsServer.ini
* - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - *
reading ... /home/rd/rdx/defaults/29r1
reading ... /home/rd/nal/prepifs/saves
Simplechecks are:true
ET::VERIFY[fc@Description]------------------------------------------------
Parse line:( DESCRIPTION == "" )?(($STATUS=-1)&&($ERR=$NODESCRIPTION))
verifyline instance=em58
( DESCRIPTION == "" ) => TRUE
Doing then part...
THEN ->(($STATUS=-1)&&($ERR=$NODESCRIPTION)) => TRUE
VERYFYLINE *** Status -> -1
ecmwf.pifs.datacontainers.DatacontainersExceptions$CheckFailedException:
( DESCRIPTION == "" )?(($STATUS=-1)&&($ERR=$NODESCRIPTION))
at ecmwf.pifs.datacontainers.ExpVer.verify(ExpVer.java:788)
at ecmwf.pifs.checkPrepIFS.Verifier.main(Verifier.java:760)
The error is indicated by VERYFYLINE having the value -1 and the offending expression is
reported as an exception.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||