Server-side Files

Page Contents

1. serverConfig
2. masterConfig
   1. Contents
   2. NIC overview
3. localConfig
   1. What can be Overridden
   2. Common Overrides and Additions
   3. Contents That Should Not Be Touched
   4. Cleanup
4. Maps.py
5. localMaps.py
   1. Adding or Removing a Set of Edit Areas
   2. Modifying the Characteristics of Existing Sets of Edit Areas
   3. Reliance on CAVE-Wide Shapefiles
   4. Filtering
   5. Edit Area Generation
6. Smart Inits
   1. Quality Control
   2. Troubleshooting
      1. Logs
      2. Manually Rerunning Inits
7. localVTECPartners
8. Other Files to Know About
   1. Level Mapping→gfeLevelMappingFile.xml
   2. ParamInfoMapping→gfeParamInfo.xml
   3. ParamNames→gfeParamName.xml
   4. Directory Parameter Info Files
   5. siteConfig/SiteCFG
9. Validating Your config

serverConfig

The serverConfig.py file is located in the Localization Perspective under the GFE Server Folder under the Server Config Files folder as shown in the image below.

This file is the BASE-level GFE server configuration file which should not be modified. It shouldn't be modified because changes will be overwritten in the next AWIPS build. It contains all of the BASE server configuration items some of which can be overridden with SITE-level overrides. The serverConfig.py file defines the general configuration of the database and weather elements. This file contains the set of weather elements, the weather configurations (e.g., possible coverages, weather types, and intensities), the set of map projections, office grid domains and resolutions, grid time constraint information, and database model configurations. More details will be discussed about some of the variables and parameters in the serverConfig.py file in the masterConfig and localConfig sections.

masterConfig

Contents

The masterConfig.py file contains the NwsInitsConfig (NIC) standardized settings for required models including regional variations and is managed by the AWIPS Datasets Evolution Pre-baseline Team (ADEPT) Team, previously the NIC Team. This file must be imported by your localConfig.py file. The masterConfig.py file greatly simplifies your localConfig.py file since it has the majority of settings for bias corrected grids and nationally and regionally supplied models.

• import Statements and Intro Variable Declaration Section
  • The import statements are shown below. It is very important that both serverConfig import statements are included.
    
  • Intro variables are declared as shown below.
    
• Utility Methods to Manage the GFE Configuration Section
  • This section contains methods that override some methods from serverConfig.py, methods that setup configuration settings, and configuration methods for each specific region. An example method is shown below.
    
  • Overrides and Corrections to Baseline Settings Section
    • This section creates and populates the NICgroups dictionary. An example NICgroups declaration is shown below.
      
  • Add RFCs to Regional Configs Section
    • This section adds all of the RFCs to the respective region that they are in. An example is shown below.
  • Time Ranges Section
    • This section contains various time constraints one of which is shown below.
      
• New Parameter Definitions and serverConfig Overrides Section
  • This section defines a masterParmDict dictionary and overrides the optionalParmDict from serverConfig.py by adding NIC- in front of the dictionary name in the optionalparmsDict declaration.
  • National GHWO Section
    • This section contains keys and an optionalParmDict declaration for GHWO. An example of a GHWO keys declaration is shown below.
      
• ISC Definitions Section
  • In this section, the serverConfig REQUEST_ISC, SEND_ISC_ON_SAVE, and SEND_ISC_ON_PUBLISH variables are overridden as shown below.
    • If serverConfig.REQUEST_ISC isn't set equal to one for your WFO (enabled) in masterConfig.py or serverConfig.py, then in localConfig.py it MUST be set equal to one (i.e., enabled).
      
• Master Model Definitions Section
  • This section contains serverConfig modelDict overrides for various models one of which is shown below.
    
  • NationalBlend Adjustments Section
    • This section contains serverConfig modelDict overrides for NationalBlend one of which is shown below.
      
  • ADJ Databases Section
    • This section contains serverConfig modelDict overrides for ADJ Databases one of which is shown below.
      
  • CONSMOS_PARMS Definition Section
    • This section defines CONSMOS_PARMS as a list of tuples of a list and a time constraint which is shown below.
      
  • CONSShort_PARMS Definition Section
    • This section defines CONSShort_PARMS as a list of tuples of a list and a time constraint which is shown below.
      
  • parmsTL Definition Section
    • This section defines parmsTL as a list of tuples of a list and a time constraint which is shown below.
      
• Ignore Databases Section
  • This section contains the override of the serverConfig ignoreDatabases list which is shown below.
    
• Region Specific Settings Section
  • This section contains sections for the different regions and RFC sites.
• ISC Parms section
  • This section contains various parms that are added to the REQUESTED_ISC_Parms list, the parmsISC list as a list of a tuple of a list and a time constraint, and the extraISCparms list as a list of a tuple of a list and the office type as a string which are shown below.
    
• Finalize Any Needed Settings Section
  • This section contains a few random overrides that weren't included in any of the other sections.

NIC overview

The NIC is a way to standardize SmartInits and the localConfig configuration. This is a framework to field a common infrastructure for WFOs and RFCs. The NIC is managed by the NSI (Nationalization of Smart Inits) which aims to manage underlying science issues with SmartInits and continues to strive toward a NWS-scale standard for the management of GFE SmartInits. A goal of the NIC is to standardize GFE SmartInits, which will make maintenance, experimentation, and R2O much easier to manage. A few more of the NIC goals are to make it easier to share SmartInits and SmartTools between sites, reduce forecast inconsistencies between WFO domains, improve overall performance, improve overall quality of produced parameter fields, and lower maintenance and development at the local level. NIC issues new releases as necessary to respond quickly to fix bugs, resolve issues with new AWIPS builds, implement regional requirements, and account for new data. In general, new releases should be installed as soon as possible without waiting for special regional instructions. If there are special regional instructions, they will be cited in the NIC release announcements. Installing updates normally only takes 10-15 minutes so there are minimal impacts to scheduling and operations with NIC rapid release. You can update from any previous version of the NIC. However, you may have to perform pre- and/or post-install actions for the intervening releases. For more information about the NIC and ADEPT Team (previously the NIC Team) click this link.

localConfig

The SITE-level localConfig.py file is located in the Localization Perspective under the GFE Server Folder under the Server Config Files Folder as shown in the image below.

This file contains all the SITE-level server overrides. The localConfig.py file is optional and not provided with the initial install or release. The localConfig.py file can be used to override many of the definitions in serverConfig.py but not all of them. The first lines of localConfig.py must have from serverConfig import * and import serverConfig. If your site is a WFO, after those two import statements you must include from masterConfig import *. Syntax in localConfig.py varies depending on whether you are adding new definitions or changing existing definitions. Typically when changing existing definitions you will need to prefix serverConfig to the name of the variable or method. The declaration var =(a,b,c,d,) refers to a local copy of this definition and doesn't override the definition in serverConfig.py. The declaration serverConfig.var =(a,b,c,d,) refers to the variable in serverConfig.py and overrides it. For list variables in serverConfig.py or masterConfig.py that you want to add to and not override (i.e., REQUESTED_ISC_PARMS, extraISCparms, and parms) you MUST use plus equals (+=) instead of just equals (=). Using plus equals (+=) will add to the list instead of overriding the list. If serverConfig.REQUEST_ISC isn't set to one for your WFO (enabled) in masterConfig.py or serverConfig.py, then in localConfig.py it MUST be set equal to one (i.e., enabled). Whenever changes are made to the localConfig.py file be sure to follow the procedure in the Validating Your Config section to check for syntax errors.

What can be Overridden

You can override certain features of the GFE database configuration that are defined in serverConfig.py or masterConfig.py. If you want to add a new weather element to a new database and not the forecast and official database then do not include the weather element in the parms statement. However, if it is added to a new database, then you will usually want to add it to the forecast and official database so you can edit it. If you want to add the new element to both a new database and the forecast and official database then include the weather element in the parms line and include it in the list of weather elements when you define a new database. Weather elements can be added to one of the standard model databases. Adding a new element to one of the standard model databases does not cause that field to be automatically generated by the initialization routines. A smart initialization algorithm will have to be written for that database. Modifying an existing parm group is accomplished by linking new or existing weather elements to new or existing time constraints. This is similar to the technique for adding a new weather element. You can add a new projection but should name it differently than the standard projections provided in GFE/CAVE. Adding a new projection doesn't do anything unless you use it in a modified site definition. If a new projection is added, the site definition (grid size, location, and projection) information will need to be updated. When changing a projection, it also changes the AWIPS world coordinate system. This can result in a distorted map background. You can add/create a new database. A list of weather elements will be included when you define a new database. The definition of the number of days to keep the GFESuite logs can be overridden. The base url for the ISC Routing Table web server may be changed. This is normally only done for isolated testing.

Common Overrides and Additions

For reference on the proper procedures for making overrides and additions to localConfig use the NIC Documentation.

• Changing your Site's Domain (SITES['XXX'])
  • The linked program (created by Matt Belk) will compute the GFE settings (SITES['XXX'] line where XXX is your site id) for you to use to change your Site's Domain. An example of the SITES['XXX'] line where XXX is your site id is shown below.
    
  • To run the program, edit the configuration section with your office ID, desired corner coordinates (lower left and upper right) and time zone (e.g., MST7MDT). GFE domains must start and end on an NDFD grid point, so you may not get your desired coordinates exactly.
  • The program will calculate the NDFD grid from the chosen projection, and then find the grid points which are closest to your desired coordinates. Those become the corner points for your final GFE domain. You can keep iterating until you have what you like.
  • For the best GFE performance, you will want to keep your domain as small as possible to meet your needs.
  • If you are changing your Site's Domain on an operational system, expanding your domain will keep your database intact, while shrinking your domain can be destructive.
    • If you make your domain too large, then cut it back again, there is a small risk you could lose your Fcst and Official databases. Either backup the databases before you get started, or more ideally do this on a cloud instance of AWIPS.
  • This is what the sections of the SITES['XXX'] line, where XXX is your site id, in localConfig.py mean:
    • [289, 257] - size of GFE domain in grid points (x, y).
    • (37.0, 27.0) - location of the lower left corner of the GFE domain from the origin of the AWIPS grid projection (Grid211 in this case).
    • (9.0, 8.0) - size of the GFE domain in AWIPS grid coordinates. Multiplying these numbers by the nominal AWIPS grid spacing, then dividing by the number of grid points from the first part will give you the actual grid spacing between points in your final GFE domain.
    • MST7MDT -time zone. The time zone defines the projection identifier to be used for this domain.
    • The default projection is Grid211, which is used by most of the CONUS offices.
• Weather Element Definitions (Local parm Definitions)
  • Weather Elements can be of type scalar, vector, weather, or discrete. The four elements are defined as shown in the image below where rateParm refers to whether the weather element is rate dependent and overlapCapable allows a grid point to have more than one value.
    
  • For a discrete weather element definition there is an optional parameter not shown in the image above called sizeofAuxDataField which specifies the size of the discrete auxiliary data field. The image below shows example weather element definitions of the four different weather element types.
    
  • Defining a new weather element has no effect. It needs to be added to the forecast and official databases or another database for that weather element to truly be created.
• Fixed Time and Local-time Time Constraints (TC, LT)
  • Time constraints force grids to be aligned on the constraint boundaries.
  • There are three components to a time constraint which are start, repeat, and duration.
    • Grids are aligned on a start time boundary, repeat every repeat interval, and have a duration of that specified.
    • All numbers are in seconds so a variable is declared for HOUR and DAY to make the entries more legible.
    • Start is the number of seconds since 0000Z or midnight local time for the first grid of the day.
    • Repeat is the number of seconds from the start time until the next grid starts.
    • Duration is the length of the grid.
  • The naming convention for a time constraint is TC followed by the repeat interval in hours, followed by NG if there aren't any gaps.
  • There are two different types of time constraints fixed time and local-time time constraints.
    • Local-time time constraints depend on local time while fixed time time constraints don't.
    • Local-time based constraints do not automatically account for daylight savings time.
      • These time constraints are created using the localTC method.
    • The first line below adds a new fixed time time constraint with the name TC8 that starts at 0200Z, repeats every eight hours, and has a duration of four hours. The second line below adds a new local-time time constraint with the name of LT9 that starts at 9:00 AM local time, repeats every 24 hours, has a duration of three hours, and the daylight savings time flag is set to zero which is standard time.
      
    • Simply adding a new time constraint by itself provides no benefit unless you also define new parm groupings and a database.
    • Any new time constraints to be used with new weather elements need to be defined before the definition is used.
    • Modifying an existing time constraint can be done by copying the definition from serverConfig.py or masterConfig.py into localConfig.py and then modifying it.
    • The line below modifies the maximum temperature time constraint so it starts at 5:00 AM local time every day, runs until 8:00 PM local time, and the daylight savings time flag is set to zero.
      
    • The line below modifies the TC6 time constraint to have grids every six hours starting at 0000Z, but with grid lengths of three hours.
      
    • Modifying an existing time constraint can have far-reaching effects.
      • Every subsequent use of this time constraint in serverConfig.py, masterConfig.py, and localConfig.py will use this new definition.
      • This can affect all of the model time constraints for all of the models.
      • It is better to simply create a new time constraint and use it.
• Adding a Weather Element to the Fcst and Official Databases (parms)
  • To add a new weather element (e.g., rh) to just the Fcst and Official databases, include a line similar to that below after all of the weather element definitions (and any new time constraint definitions).
  • The line is in the standard parm grouping format with the exception that the name of the parm group is "parms".
  • This format consists of a list of new weather elements and their time constraints.
  • The time constraint can be an existing one in serverConfig.py or masterConfig.py, or a new one defined in localConfig.py. Be sure that the name to the left of the entry is "parms".
    
  • If you choose to add multiple weather elements (parms) using the same time constraint then the line needs to be modified in the format below.
    
  • If you choose to add multiple weather elements (parms) using different time constraints, then the line needs to be modified in the format below.
    
  • There can be only one parms = line in the file. If you use more than one, then only the last one will be used. This is why normally plus equals (+=) is used instead of equals (=).
• Requested ISC Sites (REQUESTED_ISC_SITES)
  • The REQUESTED_ISC_SITES list contains the list of sites from which you want ISC data which is shown below.
    
  • This should always contain your own site.
  • If None is specified (the default setting), then EDEX will automatically calculate the needed ISC sites by examining the list of ISC_XXX edit areas where XXX is your ISC offices site ids.
• Requested ISC Parms (REQUESTED_ISC_PARMS)
  • The REQUESTED_ISC_PARMS list contains a list of weather elements to request for ISC which is shown below.
    
  • Parameters from a different office type have the office type name appended to them (e.g., ProposedTropWWGuidance has nc appended to it for a WFO site as shown in the image above).
  • If the list is set to NONE (the default), EDEX calculates the list of weather elements to be requested from the list of available weather elements in the forecast database.
  • This list MUST be added using plus equals (+=) and NOT equals (=) and specifying a list of desired weather elements.
  • All WPC ISC grids your office is requesting, like MaxT, Wind, etc., need to reference wpc instead of nc.
• Extra ISC Parms (extraISCparms)
  • The extraISCparms list contains a list of the baseline weather elements to be added as extra parameters to the ISC database which is shown below.
    
  • For ISC to work between different office types (e.g., wfo to rfc), the list of weather elements you want to receive from the different office types must be listed.
  • This list adds special renamed weather elements to the database.
  • The elements of the list are two-element tuples with the 1st element being a list of weather elements (the actual element definition and not the weather element name e.g., Temp not T) and the 2nd element being a string representing the office type.
  • This list is only necessary when receiving ISC grids from a site that is a different office type than your own.
  • This list MUST be added using plus equals (+=) and NOT equals (=).
  • There is no harm in trying to add weather elements for your own office type, even though they will be ignored.
  • The tuple in this list needs to have wpc instead of nc for all WPC ISC grids that your office wants to request.
• Local Model Dict Definitions (modelDict Definitions)
  • The modelDict variable is a master configuration dictionary for all GFE databases. An example modelDict override is shown below.
    
  • For every model, the following values can be set:
    • DB: Definition of the database(e.g., ("wrfems", GRID, "", NO, NO, 3, 0)).
      • This must be a tuple.
      • The name in the DB entry must be the same as the model name used as the key into the modelDict variable.
      • The format of the database definition is: (name, format, type, single, official, numVer, purgeAge)
        • name: The model name of the database.
        • format: Either GRID or 'DFM', but it is almost always GRID.
        • type: Optional type of the database.
        • single: YES or NO.
          • YES if this database always exists and is not based on model-runs.
          • NO if this database is created/destroyed and is based on model-runs.
            • When created, the names of these databases have time stamps.
        • official: YES or NO.
          • YES if this is an official database from which products can be generated.
          • NO if this is a conventional database.
        • numVer: Number of versions of this database to retain.
        • purgeAge: Number of hours in the past before grids will be automatically purged from the database.
          • If 0, then purging is disabled.
    • Parms: Definition of the weather element parameters in the database.
      • This is a list of tuples.
    • D2DMODELS: D2D metadata database name for the source model.
    • INITMODULES: Name of the SmartInit module (Python file).
      • This should be just the module name as a string, not a list.
    • D2DAccumulativeElements: List of parms that are accumulative.
    • D2DDBVERSIONS: Number of versions of a D2D model to show in the Weather Element Browser.
      • It defaults to 2 if it is not supplied.
    • INITSKIPS: Used to skip model cycles.
    • Only supply the entries that are needed for that model.
      • For a new GFE model database, you must supply at a minimum DB and Parms.
      • For databases that will be created from a SmartInit, you must also define D2DMODELS and INITMODULES.
      • If only D2DMODELS is specified, then the D2D grids will be accessible to tools and procedures, but there will not be a GFE database.
  • Make sure to always check both the serverConfig.py and masterConfig.py configuration files before adding new models or adding to existing models to ensure desired changes have not already been completed or standardized regionally.
  • Values in the modelDict definition for a particular model/database can be changed directly by indexing modelDict.
    
    • The first index is the model name, the second is the key to the variable to change.
• Specifying Additional Precision (ExtraWEPrecision)
  • The ExtraWEPrecision variable can be used to specify extra precision for certain weather elements in order to facilitate computations on the data.
  • The variable is a list of weather elements that require extra precision, or is a list of tuples that consist of a weather element and amount of extra precision.
  • The effective precision assigned to the weather elements in the list is one more than that defined in the weather element configuration section, if just the weather element name is used (an example is shown below), or is the specified number of precision boosts, if a tuple form is used.
    

Contents That Should Not Be Touched

Do not add declarations to the localConfig.py file that override system variables. Adding or removing a site to or from the server configuration cannot be done with the localConfig.py file. The localConfig.py file should not be used to override the map background definitions. Changing the weather configuration is not supported in the localConfig.py file but can be done in the localWxConfig.py file. If the weather configuration is changed you may find that your weather grids are not compatible with weather grids from other sites. You cannot modify an existing projection. Any attempts to do so will result in EDEX failing to initialize GFE successfully. Removing a time constraint, databases, parm groups, a projection, or weather elements from the server configuration cannot be done with the localConfig.py file.

Cleanup

Make sure that there are not duplicate lines of code (duplicate overrides and/or additions) or any typos in your file. Make sure that all of your overrides and additions are used in changing the GFE server configuration in some way. Make sure that the value(s) used to override a parameter are not identical to the value(s) in serverConfig.py or masterConfig.py. Remove all AWIPS1 remnants from your file and any AWIPS1 localConfig.py files. Make sure that there aren't any contents that should not be touched in your file. If the dbs, INITMODULES, D2DDBVERSIONS, D2DMODELS, INITSKIPS, D2DAccummulativeElements, and parms<ModelName> variables are in your file remove them because they are no longer supported. If your site is in the Eastern Region and your site installed ERinitsConfig, then remove the method/function addConfiguredModels if it exists in localConfig.py. Make sure that each override and addition is put in the appropriate section in your file (e.g., put REQUESTED_ISC_SITES in the Start Local ISC Definitions section). Remove any unnecessary comments and/or sections of code that have been commented out. Periodically review the file and remove obsolete or unnecessary entries. The extraISCparms and REQUESTED_ISC_PARMS lists need to be overridden/added to use plus equals (+=) and NOT equals (=). For all WFOs, REQUEST_ISC MUST be overridden if it is not set equal to one (i.e., enabled) in masterConfig.py or serverConfig.py. Any requested WPC ISC grids in the REQUESTED_ISC_PARMS list need to reference wpc instead of nc (e.g., Tnc should be Twpc), extraISCparms needs to reference wpc instead of nc for the requested WPC ISC grids, and any references to QPFwpc need to be removed. This link can be used for additional information about localConfig.py.

Maps.py

The Maps.py file is located in the Localization Perspective under the GFE Server Folder under the Server Config Files Folder as shown in the image below.

This file is the BASE-level GFE map configuration file which should not be modified. It shouldn't be modified because changes will be overwritten in the next AWIPS build. The Maps.py file is used for generating edit areas from maps database tables.

The Maps.py file defines how edit areas are generated for each map. Each map is identified by a Python variable (e.g., CWAcounties). There are three to five (if a group name is specified and filtering is used) lines that are used to declare how edit areas are generated from a map in Maps.py. The ShapeTable Object is created and first identified as the data source using the line shown below. The name of the maps database table is passed as a parameter to the ShapeTable constructor.

In the Maps.py file, the display name of the map or name of the map background is specified using the line shown below.

The map is then filtered in a variety of ways (an example is shown below), or not at all.

More information about filtering is discussed in the localMaps.py section. The name of the edit areas that correspond to the map is specified using the line below.

There are a few different formats that the .editAreaName line can take, which are discussed in the localMaps.py section. The name of the edit area group that you want the edit areas to be placed in is specified using the line below.

You can specify the edit area name without a group name which will cause all the edit areas associated with that map to appear under Misc under the Edit Areas menu in GFE. All of the Python variables (e.g., CWAcounties, States, etc.) for the maps that are defined in the Maps.py file are put in a list that is called maps. The Maps.py file also contains several filter and edit area name functions which are discussed more in the localMaps.py section.

Note that map backgrounds are created using map bundles and not with the Maps.py file.

localMaps.py

The SITE-level localMaps.py file is located in the Localization Perspective under the GFE Server Folder under the Server Config Files Folder. This file provides a mechanism for a site to override entries in the Maps.py file. By overriding the Maps.py file with the localMaps.py file, the user can select which polygon(s) will be converted into edit areas and to which edit area groups they will be assigned. The localMaps.py file is optional and not provided with the initial install or release. If your site uses this file, you MUST share your shapefiles specified in this file with your site's backup sites. To create the file in the Localization Perspective left-click the arrow next to (or double-click) the GFE Server Folder, then left-click the arrow next to (or double-click) the Server Config Files Folder, then right-click on Maps.py (or BASE) and left-click Override... .

The localMaps.py file is used to configure edit areas for your site. New edit areas may be added, edit areas may be removed from the list, and existing edit area attributes may be changed. The localMaps.py file must begin with from Maps import * .

Adding or Removing a Set of Edit Areas

To add a new set of edit areas in the localMaps.py file, you need the three to five lines from the Maps.py file that are used to specify how edit areas are generated from a maps database table (map definition). When adding a new map definition to the localMaps.py file, you need to add maps.append() and put the name of the Python variable (e.g. CWAcounties, States, etc.) in the parentheses. It is also important not to duplicate the Python variable name (e.g. CWAcounties, States, etc.) of any Python variable in the Maps.py file. For example, if a map is defined with CWAzones as the Python variable name in the Maps.py file, when adding a new set of edit areas (map definition) in the localMaps.py file do not use CWAzones as the Python variable name. This restriction is only for adding new edit areas. If your new set of edit areas (map definition) uses a new shapefile, you will also need to import the shapefile into the maps database.

To remove a set of edit areas find the Python variable name (e.g., CWAcounties, States, CWAmzones, Mzones, etc.) of the map you wish to remove by examining the Maps.py file. For example, if your office does not desire to generate the marine zones, find the marine zones definition in the Maps.py file. There are two sets of edit areas generated (map definitions). The Python variable names are CWAmzones and Mzones. In the localMaps.py file, add maps.remove(CWAmzones) and maps.remove(Mzones).

Modifying the Characteristics of Existing Sets of Edit Areas

You can modify the following characteristics of the map definition:

• name of the maps database table (ShapeTable parameter)
• the display name of the map or name of the map background (.name line)
• filtering (.filter line)
• generation of edit areas (.editAreaName line)
• group name of generated edit areas (.groupName line)

If you desire to change the name of the maps database table used for edit area generation, you will need to remove the existing definition and create a completely new map definition. If you desire to change the display name of the map or name of the map background, copy the line from the Maps.py file and change the name. This name is only used in the text formatter definition files (e.g., AFM_OAX_definition) with the mapNameForCombinations setting. If you don't change definition files for text formatters that use the old map to use the new map, then the Formatter Launcher won't display any zones selected when running those text formatters. If you change this map name to a map name that doesn't exist, the Formatter Launcher won't display any zones selected when running text formatters that use that map name. You can change the filter specification to whatever you desire by copying the line from the Maps.py file and then putting in a new filter string or by creating a new filter function and putting the name of that function in the parentheses after filter in the filter line from the Maps.py file. If you desire to turn off filtering completely for a map, you will need to remove the existing map definition and create a completely new map definition. More information on filtering is in the Filtering section below. If you desire to change the name of the set of edit areas that are generated, copy the line from the Maps.py file and put in the new name for the editAreaName definition. More information on the possible editAreaName formats is in the Edit Area Generation section below. If the generated edit areas names are not unique, undesirable results may occur. If you want to change the name of the group of generated edit areas, then copy the line from the Maps.py file and put in the new name for the groupName definition. You can also add a groupName line to a map definition set that doesn't have one. If you don't want a set of edit areas to appear in a specific group you can supply an empty string (i.e., '') for the groupName line. This will cause the set of edit areas for that particular map to appear under Misc under the Edit Areas menu in GFE.

Reliance on CAVE-Wide Shapefiles

Since map backgrounds are created from map bundles, GFE needs the CAVE-Wide Shapefiles for those maps to be able to display them. A PostGIS maps database is used in conjunction with xml configuration to display the maps. These maps are available across CAVE. Map shapefile updates must be imported into the maps database using the importShapeFile.sh script on dv1.

If your site doesn't have all of your backup sites' shapefiles (that are specified in localMaps.py) installed on your system, then make sure your system is updated by running:

• ssh dv1
• cd /data/fxa/sdc/
• sudo ./config_awips2.sh shp XXX where XXX is your site id

Then install a backup site's shapefiles by:

• copying their directories of shapefiles into their edex_static/site/XXX/shapefiles/ directory where XXX is a backup site's id
• ssh dv1
• cd /data/fxa/sdc/
• sudo ./config_awips2.sh shp XXX where XXX is the same backup site's id you used above

Repeat the above steps for your other backup site(s).

Filtering

Maps can be filtered in a variety of ways using shapefile attributes. Attribute names in the database are all stored in lowercase. Refer to the AWIPS Map Background Database provided by the National Weather Service for details on the available shapefiles and the attributes contained within each shapefile. The basic format of the filter line is shown below.

ATTRIBUTENAME is one of the attributes (e.g., cwa, state, countyname, etc.) in the shapefile and AttributeValue is the value that must match in the shapefile for this shape to be kept in the map. The lambda function is a shortcut method of writing a function. The following function behaves the same as the lambda function in the filter line above in the Maps.py section.

When you use a filter that does not use a lambda function in the filter line, the name of the filter function goes in the parentheses after filter as shown below.

There is another technique that can be used to create a filter function. An example of writing a filter function that selects certain counties from the county maps database table for a particular state and cwa is shown below.

Edit Area Generation

The edit area generation software uses the maps database tables as input. These tables are populated by importing shapefiles provided by the National Weather Service. These tables are filtered by domains (based on the site) and attributes. There are different attributes available for each table. Attribute names in the database are all stored in lowercase. Refer to the AWIPS Map Background Database provided by the National Weather Service for details on the available shapefiles and the attributes contained within each shapefile. This information is needed in order to tailor edit area generation for your site. The GFE Server automatically generates edit areas based on the information in the shapefiles. When the GFE server starts, it looks at the CONFIGURED-level Edit Areas and if the creation date is earlier than recent localMaps.py or shapefile changes, the GFE server will regenerate them. When the GFE server generates the edit areas, it uses the information in the Maps.py file or any override in the localMaps.py file to determine which edit areas to create and how to name them. More information on this process is in the Files Created/Updated By section of the What Happens when GFE Server Starts section.

The editAreaName parameter of the ShapeTable Object is specified by the user and the GFE Server uses that information to determine the name of each edit area. There are three forms of the editAreaName line. The first form is single string format (an example is shown below).

For this example the name of each generated edit area is obtained from the attribute name specified in the field editAreaName. The program looks up the value for the attribute name in the database record for each polygon and creates an edit area with that name. If an attribute that is a number is provided in the editAreaName line, then the attribute name will be appended to the beginning of the number. For example, if the zone for a polygon is 033, then the edit area name will be zone033. If an attribute is used that has the same value in two or more states that are in your CWA or Site's Domain, then the same edit area name will be generated for those polygons. For example, if the attribute you are using for editAreaName is zone and Nebraska and Colorado are in your CWA or Site's Domain and in the region of Colorado and Nebraska that are in your CWA or Site's Domain there is a zone 041, then for both the Colorado and Nebraska zone 041s the zone041 edit area name will be generated. To get around this issue the other two forms of the editAreaName line can be used.

A second more complex form of the editAreaName line involves using a Python list and is referred to as the list of strings format. For this form the name of the edit areas is determined by using more than one attribute. An example using two attributes is shown below.

When edit area names are specified in this form, between the attributes there is an underscore which is used to separate the attributes. So for example, for a state attribute of CO and a countyname attribute of Boulder the edit area name would be CO_Boulder. The software algorithm for the naming of edit areas in this form is the following:

• The value for each attribute is extracted from the shapefiles. All spaces and non-alphanumeric characters are removed from the value. For example, for a value of Washington Meadows it would be changed to WashingtonMeadows.
• If any of the values result in an empty string, then the edit area will not be generated.
• The values are put together with an underscore separator between each of the attributes.
• If the leading character of the resulting edit area name begins with a number, then the name of the first attribute is prefixed to the string.

The order of attributes matters. If the order of the state and countyname attributes were switched in the Python list above, then for the above example the edit area name would be Boulder_CO. You can put any number of attributes in the Python list, but most commonly only two are used. Note that the specification of an attribute that doesn't exist will cause the edit area generation process to simply use the name of the attribute in the string.

The final and most complex form of the editAreaName line involves using a function. Occasionally there is a need to calculate a special edit area name and the single string or list of strings formats are not sufficient. The function format requires the user to name a function and define a function. The function has one argument, which takes a dictionary of attribute names and their values. The return value of the function is a string which becomes the edit area name. For example, it is desired to have edit area names based on the county FIPS code. The desired format is stateCfips, such as NEC023, for Nebraska County #23. The county shapefile has a STATE attribute, such as NE, and it has a FIPS attribute, such as 31023 for Nebraska County #23. Using the list of strings format, ['state', 'C', 'fips'], you would end up with NE_C_31023, when it is desired to be NEC023. The following function is used to create the editAreaName string.

The if else sequence is used to reduce the chance of their being a GFE Server error when creating these fips edit areas. The editAreaName line for this example would look like the line that is shown below.

An example of the atts dictionary is shown below.

Note that changing the default edit area names as supplied in the baseline may render your text formatters and VTEC inoperable. The text formatters and VTEC assume standard UGC-named edit areas, such as COZ023 and UTC013.

Smart Inits

Smart Inits derive sensible weather elements from model data and create the IFP database from the D2D data. They only run on models and not MOS. Smart Inits field tailorable algorithms. They can add new weather elements and local models. Smart Inits allow grids to be generated as D2D data arrives. They allow algorithms to map from modules to models and certain model runs to be skipped. In Smart Inits, ctime is a tuple containing the start and end times of the source grid in epoch seconds; mtime is a tuple containing the start and end times of the destination grid in epoch seconds; and stime is a tuple containing the start and end times of a grid in seconds relative to the model base start time. Smart Inits are how the databases in GFE are created. After a data source is saved in the Weather Element Browser D2D side, a Smart Init is created to make the GFE database. Every field must have a method in the Smart Init, even if it just returns itself. All of the Smart Inits are now controlled by the NIC.

For more information on customizing Smart Inits go here. After making changes to or creating a new Smart Init perform quality control.

Quality Control

Make sure the Smart Init has a main method that calls the run method of the class (e.g., Local_GFSForecaster().run()) for the Local_GFS Smart Init). Make sure that there aren't any syntax errors in your Smart Inits. Make sure that you only make changes to the or create a Local version of a particular Smart Init (i.e. Local_GFS, Local_HRRR, Local_NAM, etc.,).

Troubleshooting

If a Smart Init doesn't seem to be working properly, look at all the methods and make sure they all have proper Python syntax. If they don't, then fix them so that they do and manually rerun the Smart Init. After any changes have been made, Validate Your config, then deactivate and reactivate your Site using the GFE Site Activation GUI or by manually running the scripts from the command line in a terminal.

Logs

All entries related to the creation of grids by Smart Inits are included in the EDEX smartInit log file (edex-ingest-smartInit-YYYYMMDD.log), which is located in the /awips2/edex/logs directory on dv3, dv4, and dv5. This log file consolidates all Smart Init entries. Despite this, finding information for individual models is simple. Entries for each model include the destination database, elements calculated, and elapsed time. The destination database is shown as the model name and time. The elements calculated are shown as the element name and grid valid time. The elapsed time is shown as the time to calculate the elements. When using the logs for troubleshooting, search for lines that begin with ERROR or PROBLEM instead of INFO.

Manually Rerunning Inits

The ifpInit script is used to manually rerun a Smart Init. The ifpInit script can take six switches: h, p, t, s, u, and a. The h switch is for the host upon which EDEX is running. Normally this switch is not needed and the software will determine the host where EDEX is running. The p switch is for the RPC port upon which EDEX is running. Normally this switch is not needed and the software will determine the port where EDEX is running. The t switch specifies the model run time in the format of yyyymmdd_hhmm. If this switch is not specified, then the latest model run is used. The s switch specifies the site id for which to run the Smart Init. This switch is the only required switch. The u switch specifies the user name of the user who is executing the Smart Init. The a switch is used to specified the creation of all the possible data grids, which will overwrite previously existing calculated grids. Normally by default, only those grids which haven't been created will be attempted to be created. After all the switches have been specified on the command line, then the algorithmFile will be specified. The algorithmFile is a mandatory argument that specifies the name of the Smart Init. The h and p switches are predefined to match your AWIPS installation, such that they will point to EDEX on installation. Therefore, the h and p switches aren't needed for normal running of ifpInit. However, if you wish to connect to a different server, the h and p switches will be needed. Manually requested Smart Inits will queue ahead of other Smart Inits. The ifpInit script is discussed in more detail in the Command Line Scripts section.

localVTECPartners

The localVTECPartners.py file is the SITE-level override of the VTEC active table coordination. It overrides the VTECPartners.py file. The VTECPartners.py file is used to configure the site filtering for the Merge VTEC and VTEC Decoder processes. It is used to specify the adjacent sites used for active table synchronization. The VTECPartners.py file is also used to configure the interval of active table synchronization with adjacent sites. The localVTECPartners.py file begins with two import statements that MUST be at the top of the file, as shown below.

Six parameters can be/usually are overridden in the localVTECPartners.py file:

• TABLE REQUEST SITES
• MERGE SITES
• DECODER SITES
• REMOTE TABLE FETCH TIME
• RESPOND TO TABLE REQUESTS
• CAPTURE TEXT CATEGORIES

The VTEC TABLE REQUEST SITES list (shown below) is a list of AWIPS WAN IDs (typically WFO IDs) which identifies the locations from which your site/WFO will request their active VTEC table for merge purposes. Requests will be sent from EDEX to these sites at the interval specified using the VTEC_REMOTE_TABLE_FETCH_TIME variable.

The VTEC MERGE SITES list (shown below) is a list of office identifiers which is used in the ingestAT/MergeVTEC software to filter out records from offices whose CWA isn't adjacent to your CWA. A site filter is applied when active tables from remote sites are merged into your site's/WFO's active table. If this was not done, your active table would soon end up containing records from every site in the country. Your adjacent sites and your backup offices should be listed in this field. Your site/WFO and SPC site are automatically added to this list in the software.

The VTEC DECODER SITES list (shown below) is a list of office identifiers which are used in the VTECDecoder software to filter out records from offices whose CWA isn't adjacent to your CWA. The VTECDecoder receives text bulletins from the AWIPS data stream and decodes them for VTEC entries. This list is used to specify which site's/wfo's will be included in the VTECDecoder stream. If a VTEC site id is detected in an incoming product and that site is not included in this list, the entry will be discarded. Your site/WFO and SPC site are automatically added to this list in the software.

The VTEC REMOTE TABLE FETCH TIME variable (shown below) defines the coordination interval for updating your active table with adjacent site's active tables. It is specified in seconds.

The VTEC RESPOND TO TABLE REQUESTS variable (shown below) determines whether your site/WFO will respond to VTEC table requests from other sites/WFOs. If it is set to one, then your site's/WFO's filtered VTEC table will be returned to the requesting site/WFO.

The VTEC CAPTURE TEXT CATEGORIES list (shown below) is a set of product categories that when decoded, the text for each segment is captured. The VTECDecoder captures the segment text for each product. Only a few of the product formatters, warning-style product formatters, require this text. Since the text is large when compared with the size of the rest of the record, text is captured only for certain product categories which are defined by this list.

Other Files to Know About

There are a few other server-side files that GFE focal points need to know about. The data flow shown here involves these files.

Level Mapping → gfeLevelMappingFile.xml

The gfeLevelMappingFile.xml file defines all of the levels that can be used for all of the maps that are used in GFE. The gfeLevelMappingFile.xml file exists at the BASE-level and so any desired changes should be made in a SITE-level override. This file is located in the Localization Perspective under the GFE Server Folder under the Level Mapping Folder as shown below.

After making any changes to the SITE-level gfeLevelMappingFile.xml file, ask your ITO or ESA to restart EDEX or call NCF and ask them to restart EDEX. The following is an example for the 1000 mb level. All of the level definitions are similar to the example definition, except sometimes the declarations don't have the unit specification and also sometimes there is a levelTwoValue specification in addition to the levelOneValue specification (e.g., for boundary layer (BL) level definitions).

Param Info Mapping → gfeParamInfo.xml

The gfeParamInfo.xml file contains aliases for various models that are used by GFE. The gfeParamInfo.xml file exists at the BASE-level and is an incremental override file, so any additions should be made to the SITE-level XXX-gfeParamInfo.xml file, where XXX is your site id, and not the SITE-level NIC-gfeParamInfo.xml file. The additions should not already be present in the BASE or NIC files, since that would be redundant. This file is located in the Localization Perspective under the GFE Server Folder under the Param Info Mapping Folder as shown below.

After making any changes to the SITE-level XXX-gfeParamInfo.xml file where XXX is your site id, ask your ITO or ESA to restart EDEX or call NCF and ask them to restart EDEX. The following is an example for the MOSGuide-HI model and a GFS model. All of the model alias definitions follow the same format as shown in the two examples.

Param Names → gfeParamName.xml

The gfeParamName.xml file contains aliases for various parameters that are used by the GFE. The gfeParamName.xml file exists at the BASE-level, so any additions and overrides should be made to the SITE-level XXX-gfeParamName.xml file, where XXX is your site id and not the SITE-level NIC-gfeParamName.xml file. This file is located in the Localization Perspective under the GFE Server Folder under the Param Names Folder as shown below.

After making any changes to the SITE-level XXX-gfeParamName.xml file where XXX is your site id, deactivate and reactivate your Site to restart the server using the GFE Site Activation GUI or by manually running the scripts from the command line with a terminal. The following is an example for the CAPE and AV parameters. All of the parameter alias definitions follow the same format as shown in the two examples.

Directory Parameter Info Files

The Parameter Info Files Folder which is located in the Localization Perspective under the GFE Server Folder (shown below) contains xml parameter information files for every model that is used by the GFE.

After making any changes to a Parameter Info file, ask your ITO or ESA to restart EDEX or call NCF and ask them to restart EDEX. These files contain all the specifications, such as units and max and min values, which are needed by GFE to display models. These files usually contain various parameter names and levels.

siteConfig/SiteCFG

The siteConfig.py file, which is located in the Localization Perspective under the GFE Server Folder under the Server Config Files Folder, (show below) is used to identify the main directories used by GFE, the server network information, and the site identifier.

This file defines the directory and site variables. The siteConfig.py file is one of three GFE server configuration files and it exists at the SITE-level. Whenever changes are made to the siteConfig.py file be sure to follow the procedure in the Validating Your Config section to check for syntax errors. The siteConfig.py file is used by the gfeConfig.py file and the configureTextProducts script. The SiteCFG.py and OtherCFG.py files are used to create the siteConfig.py file.

Validating Your Config

After making any changes to either localConfig.py or siteConfig.py, both files need to be validated. The following steps are used to complete this task. Open the GFE Perspective. From the GFE menu, select Site Activation... . In the Site ID box, select the Site (e.g., BOU) you want to validate the config files of. Then left-click Validate. If the Log Roll says "XXX siteConfig and localConfig validate ok!" where XXX is your site id, you are good to go; otherwise, you have syntax errors in one or both of the localConfig.py and siteConfig.py files.