GFE Focal Point Curriculum

Menu

Command Line Scripts

Commonly used commands

Page Contents

  1. ifpInit
  2. ifpnetCDF
  3. iscMosaic
  4. runProcedure
  5. runIFPText
  6. Launching CAVE with Switches

The ifpInit, ifpnetCDF, iscMosaic, runProcedure, and runIFPText scripts are located in the /awips2/GFESuite/bin directory.

ifpInit

The ifpInit script is used to manually run/rerun a Smart Init. The ifpInit script can take six switches: h, p, t, s, u, and a.

Switch Required ? Description
h hostname no This switch specifies the host upon which EDEX is running.
p port no This switch specifies the RPC port upon which EDEX is running.
t modelTime no This switch specifies the model run time in the format of yyyymmdd_hhmm. If this switch is not specified, then the latest model run is used.
s site yes This switch specifies the site id for which to run the Smart Init.
u userID no This switch specifies the user name of the user who is executing the Smart Init.
a no This switch specifies the creation of all the possible data grids, which will overwrite previously existing calculated grids. Normally by default, only those grids which haven't been created will be attempted to be created.

The h and p switches are predefined to match your AWIPS installation, such that they will point to EDEX on installation. Therefore, the h and p switches aren't needed for normal running of ifpInit. However, if you wish to connect to a different server, the h and p switches will be needed. Somewhere on the command line the algorithmFile will need to be specified. The algorithmFile is a required argument that specifies the name of the Smart Init you wish to run. Manually requested Smart Inits will queue ahead of other Smart Inits. Here are a few examples of running the ifpInit script:

  • /awips2/GFESuite/bin/ifpInit -s BOU Local_GFS -a
    • This command runs the Local_GFS Smart Init as BOU for the latest model run of the GFS and creates all possible grids.
  • /awips2/GFESuite/bin/ifpInit -s LIX -t 20220121_1400 Local_HRRR
    • This command runs the Local_HRRR Smart Init as LIX for the January 21, 2022 14Z run of the HRRR.
  • /awips2/GFESuite/bin/ifpInit -s OAX -t 20220215_1200 Local_NAM12 -a
    • This command runs the Local_NAM12 Smart Init as OAX for the February 15, 2022 12Z run of the NAM12 and creates all possible grids.

ifpnetCDF

The ifpnetCDF script is the GFE grid extraction program. This script extracts grids from ifpServer databases and puts them into netCDF format. The ifpnetCDF script is used to create compressed netCDF files. This script is also used to archive the forecast database for WES or a cloud case. The ifpnetCDF script can use 17 different switches: o, h, r, u, d, p, s, e, m, g, c, f, t, k, C, v, and S.

Switch Required ? Description
o outputFile no This switch specifies the output file path and name. The directory specified by the path must be one that user awips can write to and your user can access. A good directory to use is /data/fxa/ .
h hostname no This switch specifies the host upon which EDEX is running.
r port no This switch specifies the RPC port upon which EDEX is running.
u userID no This switch specifies the user name for connection to the server. If this switch is not provided, then it is assumed you are running the script as SITE. This switch is important only when using edit area masks and the mask is USER-level and not BASE- or SITE-Level.
d databaseID yes This switch provides the database id for the gridded information. The format of the database id is: site_GRID_optType_modelName_modelRunTime where modelRunTime is in the form of yyyymmdd_hhmm.
p parmName no This switch specifies the weather elements to process. If this switch is not provided, then all weather elements in the specified databases will be processed. If this switch is provided, then only those weather elements will be stored. There can be multiple switches specified in the form of -p wx1 -p wx2 -p wx3. The parmName refers to SFC weather elements, to denote a weather element for a different level, use the parmName_level format such as T_3K.
s startTime no This switch specifies the start time in the format of yyyymmdd_hhmm. By default, the start time is set to the beginning of time.
e endTIme no This switch specifies the end time in the format of yyyymmdd_hhmm. By default, the end time is set way into the future.
m mask no This switch specifies an edit area to be used as the clip mask. If no mask is specified, then the entire domain will be used. All values outside the mask will be assigned the missing value for the weather element type.
g no This switch is used to have a grid of topography, latitude, and longitude written to the netCDF file.
c no This switch is used to have the output compressed.
f factor no This switch specifies the gzip factor to be applied to compression. The default is 6 but the value can range from 1 to 9 with 9 being the maximum, but slowest, compression.
t no This switch specifies that data values will be rounded to the precision of the data as defined by EDEX.
k no This switch is used to reformat the data values into smaller units, as in converting a 32-bit float into a 8-bit byte in order to save space. When this switch is provided and the data has been reformatted, two new attributes are provided (dataMultipler and dataOffset) which are used to convert the data back into real numbers.
C configIntervalFileName no This switch specifies a configuration interval file which controls the interval/spacing of the grids. The name identifies a file within the textUtilites directory and must be a Python file in the correct format.
v logFileName no This switch specifies the log file name and path.
S siteIDOverride no This switch will override the site ID used in the netCDF file to the provided site ID.

The o switch while technically optional needs to be provided so that you can access the netCDF file that the script creates. The h and r switches are predefined to match your AWIPS installation, such that they will point to EDEX on installation. Therefore, the h and r switches aren't needed for normal running of ifpnetCDF. However, if you wish to connect to a different server, the h and r switches will be needed. When the f switch is provided, the c switch must also be provided. If the k switch is provided, then the t switch is required.

Task: ifpnetCDF
This task demonstrates how to run the ifpnetCDF script.
View Jobsheet

iscMosaic

The iscMosaic script puts netCDF files back together into a database. This script was originally created and is used for intersite coordination. The iscMosaic script can be used to ingest netCDF files into the server. It is useful for archive case creation/restoration on a cloud instance or WES. It is also used when backing up a site and extracting the failed site's grids and configuration that were downloaded from the Central Server. The iscMosaic script can use 23 different switches: h, r, u, d, p, f, b, s, e, x, z, n, a, w, k, i, D, o, S, T, v, A, and y.

Switch Required ? Description
h hostname no This switch specifies the host upon which EDEX is running.
r port no This switch specifies the RPC port upon which EDEX is running.
u userID no This switch specifies the user name which is used for obtaining edit areas from the server.
d databaseID no This switch specifies the database to store the grids. This switch is predefined based on your installed configuration of GFESuite. The database is preset to the ISC database for the site installed. The format of the database id is: site_GRID_optType_modelName_modelRunTime where modelRunTime is in the form of yyyymmdd_hhmm.
p parmName no This switch specifies the weather elements to store in the database. If this switch is not provided, then all weather elements found in the netCDF file that exist in the destination database will be stored. If the p switch is provided then only those weather elements will be stored. There can be multiple switches in the form of -p wx1 -p wx2 -p wx3. The parmName refers to SFC weather elements, to denote a weather element for a different level, use the parmName_level format such as T_3K.
f fileName yes This switch specifies the input netCDF file for processing. This file must be in the same format as the output that the ifpnetCDF script produces. There can be multiple switches, in the form of -f file1 -f file2 -f file3.
b no This switch specifies blanking mode. If enabled, then for all times, other than the times in the incoming grids, any existing grids will have the area defined by the area mask blanked, i.e., set to the minimum data value. If disabled, then no data within the area defined by the area mask will be modified in time periods that are not contained within the incoming grids.
s startTime no This switch specifies the start time in the format of yyyymmdd_hhmm. By default, the start time is set to 00Z January 1, 1970. Only incoming grids that contain the start time or start after the start time (but before the end time) will be processed.
e endTime no This switch specifies the end time in the format of yyyymmdd_hhmm. By default, the end time is set way into the future. Only incoming grids that contain the end time or end before the ending time will be processed.
x no This switch specifies replace mode. By default, iscMosaic performs merges of the incoming data grids into any existing grids over the region defined by the area mask. If this switch is provided, then iscMosaic will not perform a merge, but instead will replace any existing grid with the incoming grid.
z no This switch tells the system to remove all grids from the destination database. By default, no grids are removed from the destination database. If this switch is provided, then before any incoming grids are stored, any existing grids in the database are removed.
n no This switch tells the system to ignore the area mask. By default, the site identifier field in the input netCDF file is used to determine the area mask. For example, if the netCDF file site id field is CYS, then the program will attempt to get the edit area called ISC_CYS from EDEX and then only merge those grid points. If this switch is provided, then the area mask will be ignored and all grid points in the input netCDF file will be merged into the database. This mode is not quite identical to the replace mode (x switch) unless the input netCDF file and the destination database contain the same domain (geographical information).
a alternativeMask no This switch specifies an alternative mask. By default, the site identifier field in the input netCDF file is used to determine the area mask. For example, if the netCDF file site id field is OAX, then the program will attempt to get the edit area called ISC_OAX from EDEX and then only merge those grid points. This switch identifies the name of the edit area used for masking.
w message no This switch generates a message that warns/alerts all clients connected to EDEX that data has been processed. This produces a message on AlertViz. The site identifier, number of grids, valid time range, and weather elements received are displayed in the message
k no This switch is used to delete the input file(s) after processing has been completed.
i parmName no This switch specifies the weather elements to skip. If this switch is not provided, then all weather elements found in the netCDF file that exist in the destination database will be stored or those specified by the p switch will be stored. If this switch is provided, then the identified weather element(s), if found in the input file, will be skipped, even if a p switch is provided. There can be multiple switches in the form of -i wx1 -i wx2 -i wx3. The parmName refers to SFC weather elements, to denote a weather element for a different level, use the parmName_level format such as T_3K.
D gridDelay no This switch specifies the delay between the processing of grids. The grid delay defaults to 0.25 seconds. The delay is used to lesson the effect of ISC traffic on the loading of EDEX and therefore the effect on GFE's performance. The value is in seconds and can be fractional. To eliminate the delay use zero.
o no This switch is provided to use the office type from EDEX. If this switch is provided, then incoming weather elements will be renamed if the data is from a different office than your own.
S no This switch enables ISC Sending for this process. If Send ISC On Save is set, then Fcst grids will be sent upon saving. If Send ISC on Save is not set, then the user must manually send the grids via a procedure through the Send Intersite Grids capability.
T no This switch enables the translate mode for WEATHER and DISCRETE data. If this switch is provided and if incoming data is not considered valid per EDEX definitions, then the data keys will be translated to simpler keys that are understood by EDEX.
v logFileName no This switch specifies the log file name and path.
A additionalRoutingSiteID no This switch applies AdditionalISCRouting for the source site Id.
y no This switch is used so that you do not have to wait for processing to complete.

The h and r switches are predefined to match your AWIPS installation, such that they will point to EDEX on installation. Therefore, the h and r switches aren't needed for normal running of iscMosaic. However, if you wish to connect to a different server, the h and r switches will be needed. The n and a switches cannot be provided together since they are mutually exclusive. If more than one f switch is provided, then the z switch cannot be provided and the x switch should not be provided. For the purposes of ISC, the s, e, x, z, n, and a switches must not be provided and the b switch should be provided. The iscMosaic script ignores the database time constraints, so if the data to be stored does not fall within a valid time constraint, the grid will not be stored. This isn't an issue for ISC, since its grids are stored into the ISC database, which has time constraints of 1 hour.

Task: iscMosaic
This task demonstrates how to run the iscMosaic script.
View Jobsheet

runProcedure

The runProcedure script is used to run/execute a procedure from the command line. This script doesn't allow for user interaction with the procedure and so procedure GUIs will not open when running a procedure using this script. The runProcedure script can use 10 different switches: n, u, c, a, s, e, t, m, z, and V.

Switch Required ? Description
n procedureName yes This switch specifies the name of the procedure.
u userID no This switch specifies the user name. If this switch is not provided, then it is assumed you are running the procedure as SITE.
c configurationFileName no This switch specifies the GFE configuration filename. If this switch is not provided, then the script uses the gfeConfig configuration.
a editAreaName no This switch specifies the edit area name. If no edit area is specified Smart Tools invoked by the procedure will automatically operate on the entire grid.
s startTime no This switch specifies the start time in the format of yyyymmdd_hhmm.
e endTime no This switch specifies the end time in the format of yyyymmdd_hhmm.
t timeRange no This switch specifies a named time range (e.g., Today, Tonight, etc.,).
m mutableModel no This switch overrides the mutable model. The format for this switch is type_model, type_model_time, _model, or _model_time where time is in the format of yyyymmdd_hhmm (e.g., _Fcst or _NAM_20220229_1200, or D2D_GFS_20220201_1800).
z displacedRealTimeMode no This switch is used to run the procedure in Displaced Real Time Mode in the format of yyyymmdd_hhmm.
V variableList no This switch specifies a run-time variable list. The var dict (V switch argument) must be in the form of a Python dictionary string (e.g., '{"Input Variable":"variable value", "Another variable":25 }').

If the s switch is specified, then the e switch is required. The t switch or the s and e switches usually must be specified, unless a time is provided in the variable list argument (V switch), since procedures usually run over a time range. The V switch is only optional (not required) for procedures that don't take user input. The u switch is only optional (not required) when using a SITE-level configuration file or gfeConfig.py as the configuration file. Here are a few examples of running the runProcedure script:

  • /awips2/GFESuite/bin/runProcedure -n PlotSPCWatches -c BOU_gfeConfig -a BOU -t Today
    • This command runs the PlotSPCWatches.py procedure on the BOU edit area for the Today time range using the BOU_gfeConfig.py configuration file.
  • /awips2/GFESuite/bin/runProcedure -n Aviation_CloudBaseFromRH -c OAX_gfeConfig -s 20220120_1200 -e 20220121_0300 -V '{"Use AGL cloud heights:":"Yes","Populate:":"CloudBasePrimary","NAM12":"0","Previous NAM12":"0","GFS":"1","Previous GFS":"0","RAP13":"0","Previous RAP13":"0","Alter RH thresholds for clouds by (%):":20}'
    • This command runs the Aviation_CloudBaseFromRH.py procedure starting from 02Z January 20, 2022 to 03Z January 21, 2022 using AGL cloud heights, populating the CloudBasePrimary weather element, weighting the current and previous runs of the NAM12 and RAP13 at 0, weighting the current run of the GFS at 1, weighting the previous run of the GFS at zero, and increasing the RH threshold for clouds by 20% using the OAX_gfeConfig.py configuration file.
  • /awips2/GFESuite/bin/runProcedure -n Align_Grids -c LIX_gfeConfig -a LIX -V '{"Select Time Range:": "All", "Source Grid:": "PoP", "Aligned Grids:": ["Wx", "PoP", "Sky", "QPF","SnowAmt","IceAccum","Wind","WindGust"]}'
    • This command runs the Align_Grids.py procedure on the LIX edit area using the All Select Time Range option with the PoP Source Grid on the Wx, PoP, Sky, QPF, SnowAmt, IceAccum, Wind, and WindGust grids using the LIX_gfeConfig.py configuration file.
  • /awips2/GFESuite/bin/runProcedure -n CheckTandTd -c siteConfig -s 20220208_0000 -e 20220210_0200 -V '{ "Check or Force:":"Check Only"}' -m _NAM12_20220208_0000
    • This command runs the CheckTandTd.py procedure starting from 00Z February 8, 2022 to 02Z February 10, 2022 on grids from the 00Z February 8, 2022 run of the NAM12 using the Check Only Check or Force option and the siteConfig.py configuration file.

runIFPText

The runIFPText script is used to run/execute a text formatter from the command line. The script is also used by default when a text formatter is run using the formatter launcher. The runIFPText script can use 20 different switches: d, t, c, o, S, u, l, A, V, O, z, T, E, v, a, r, w, s, e, and i.

Switch Required ? Description
d databaseID yes This switch specifies the database source for the gridded information. The format of the database id is: site_GRID_optType_modelName_modelRunTime where modelRunTime is in the form of yyyymmdd_hhmm.
t forecastType yes This switch specifies the text formatter (e.g., ZFP, AFD_XXX where XXX is your site id) to run .
c configurationFileName no This switch specifies the GFE configuration filename. If this switch is not provided, then the script uses the gfeConfig configuration.
o outputFile no This switch specifies the output file path and name for the formatter.
S specialOutputFile no This switch specifies the GFE Special Output File path and name. This is a special file that is used for background processing text formatters to store the output forecast text. The GFE uses this switch for formatters that are run locally by the GFE so the forecast text may be gathered at the end of the formatter execution. Formatters run by EDEX use this switch for the same purpose. The connection allows for proper behavior in the GFE's Formatter Launcher and Process Monitor.
u userID no This switch specifies the user name. If this switch is not provided, then it is assumed you are running the formatter as SITE.
l language no This switch specifies a language for formatters that support foreign languages. The options are Spanish and French. The default is English.
A appendFileName no This switch specifies the path and name of a file to append the output text from the formatter to.
V variableList no This switch specifies a run-time variable list. The var dict (V switch argument) must be in the form of a Python dictionary string (e.g., '{("Product Issuance", "productIssuance"): "Morning" }').
O serverOutputFileName no This switch specifies the server output file for the formatter.
z displacedRealTimeMode no This switch is used to run the text formatter in Displaced Real Time Mode and is provided in the format of yyyymmdd_hhmm.
T no This switch places the formatter into TEST mode. In TEST mode the output of the script is a TEST product. If vtecMode is specified, it is set to T. TEST mode adds ...TEST... to headlines and MND headers.
E no This switch places the formatter into EXPERIMENTAL mode. In EXPERIMENTAL mode, the output of the script is an EXPERIMENTAL product. If vtecMode is specified, it is set to E. EXPERIMENTAL mode adds ...EXPERIMENTAL... to headlines and MND headers.
v vtecMode no This switch specifies the vtec mode for the product. The mode can be X, O, T, or E. If not specified, then VTEC is disabled for the product.
a vtecActiveTable no This switch defines the active table to use for VTEC comparisons. It is either PRACTICE or active (OPERATIONAL). If not specified, the active (OPERATIONAL) table is used.
r editAreaName no This switch specifies the edit area name.
w timeRange no This switch specifies a named time range (e.g., Today, Tonight, etc.,).
s startTime no This switch specifies the start time in the format of yyyymmdd_hhmm.
e endTime no This switch specifies the end time in the format of yyyymmdd_hhmm.
i timePeriod no This switch specifies the time period for tables with a variable period (rows or columns)

The E and T switches cannot not be provided together since they are mutually exclusive. If the s switch is specified, then the e switch is required. The u switch is only optional (not required) when using a SITE- or BASE-Level configuration file or edit area. Here are a few examples of running the runIFPText script:

  • /awips2/GFESuite/bin/runIFPText -t ZFP_BOU -d BOU_GRID__Official_00000000_0000 -A /awips2/GFESuite/products/TEXT/ZFP.txt -V '{("Product Issuance", "productIssuance"): "Morning", ("Issued By", "issuedBy"): ""}' -v O -a active
    • This command runs the Morning version of the ZFP_BOU text formatter using BOU's Official database in the O vtecMode using the active active table and appending the output of the formatter to the ZFP.txt file located in the /awips2/GFESuite/products/TEXT/ directory.
  • /awips2/GFESuite/bin/runIFPText -t AFD_LIX -d LIX_GRID__Official_00000000_0000 -V '{("Product Issuance", "productIssuance"): "Morning", ("""Short Term\nForecaster""", "shortTermFcstrNumber"): "01", ("Issued By", "issuedBy"): "", ("""Optional\nTopics""", "optionalTopics"): [".UPDATE..."], ("""Aviation\nForecaster""", "aviationFcstrNumber"): "01", ("""Include\nPrevious AFD?""", "includePreviousAFD"): "NO", ("""Long Term\nForecaster""", "longTermFcstrNumber"): "02"}' -v E
    • This command runs the Morning version of the AFD_LIX text formatter using LIX's Official database in the E vtecMode as the Short Term Forecaster number 01, as an Update to a previous version of the formatter, as the Aviation Forecaster number 01, without including the previous AFD, and as the Long Term Forecaster number 02.
  • /awips2/GFESuite/bin/runIFPText -t AFM_OAX -d OAX_GRID__Official_00000000_0000 -V '{("Product Issuance", "productIssuance"): "Morning", ("Issued By", "issuedBy"): ""}' -o /awips2/GFESuite/products/TEXT/AFM.txt -c OAX_gfeConfig -v X
    • This command runs the Morning version of the AFM_OAX text formatter using OAX's Official database in the X vtecMode writing the output of the formatter to the AFM.txt file located in the /awips2/GFESuite/products/TEXT/ directory using the OAX_gfeConfig.py configuration file.
  • /awips2/GFESuite/bin/runIFPText -t Hazard_FFA -d FSD_GRID__Official_00000000_0000 -V '{("Issued By", "issuedBy"): "", ("Flood Reason (HVTEC)", "floodReason"): "SM (Snow Melt)"}' -v T
    • This command runs the Hazard_FFA text formatter using FSD's Official database in the T vtecMode where the Flood Reason (HVTEC) is SM (Snow Melt).
  • /awips2/GFESuite/bin/runIFPText -t AQA_ER -d RNK_GRID__Official_00000000_0000 -V '{("Alert Level", "alertCode"): "Orange", ("Issued By", "issuedBy"): "", ("Alert Type", "alertType"): "Ozone Action Day"}' -E
    • This command runs the AQA_ER text formatter using RNK's Official database in EXPERIMENTAL mode using the Orange Alert level and Ozone Action Day Alert Type.
  • /awips2/GFESuite/bin/runIFPText -t CivilEmerg_CDW_LA_Local -d LIX_GRID__Official_00000000_0000 -V '{("Issued By", "issuedBy"): "", ("Source", "source"): "Louisiana EMERGENCY MANAGEMENT AGENCY New Orleans Louisiana", ("EAS Level", "eas"): "URGENT - IMMEDIATE BROADCAST REQUESTED"}' -a PRACTICE
    • This command runs the CivilEmerg_CDW_LA_Local text formatter using LIX's Official database using the PRACTICE active table using Louisiana EMERGENCY MANAGEMENT AGENCY New Orleans Louisiana as the Source and URGENT - IMMEDIATE BROADCAST REQUESTED as the EAS Level.
  • /awips2/GFESuite/bin/runIFPText -t ESF_Local -d DMX_GRID__Fcst_00000000_0000 -z 20211231_1200 -T
    • This command runs the ESF_Local text formatter using DMX's Forecast database in displaced real time mode at 12Z on December 31, 2021 in TEST mode.

Launching CAVE with Switches

CAVE can be launched using various switches which will be described in the jobsheet below.

Task: Launching CAVE with Switches
This task demonstrates how to launch cave with common switches.
View Jobsheet