APRSWXNET LDAD Instructions                           02/24/12
---------------------------

To access APRSWXNET csv data from MADIS, filter out stations outside your
area of interest, and incorporate it into LDAD:


- Ftp to rftp.madis-data.noaa.gov:

   ftp rftp.madis-data.noaa.gov     [e.g., account= CRP_madis_wfo, password= ?????]


- Get the LDAD metadata files (*Station.txt and *.desc files).
  and put them into /data/fxa/LDAD/data:

  cd wfo/LDAD/tables     

  get APRSWXNETStation.txt           (Station table)
  get APRSWXNET.desc                 (Descrip file)


- Get the following scripts:

  cd ../../scripts
  bin

  get MADISfilter.pl                (Lat/lon filtering script)
  get build_station_idx.pl          (Creates station list used by MADISfilter.pl)


- As user ldad, login to ls1 and make a directory to use for the 
  lat/lon filtering.  The example here (and inside the scripts)
  uses "/ldad/madisfilter", but you can use whatever you like.


- Put the MADISfilter.pl & build_station_idx.pl scripts into 
  /ldad/madisfilter, and make them executable.


- Edit MADISfilter.pl:

  - For the location of perl (if not in /usr/local/bin/perl).

  - For the filter directory (if you're not using /ldad/madisfilter).


- Edit build_station_idx.pl:

  - For the location of perl (if not in /usr/local/bin/perl).

  - For the filter directory (if you're not using /ldad/madisfilter).

  - For the latitude/longitude corners of the rectangle to use for
    the filtering.


- Copy over all the *Station.txt files from px2 /data/fxa/LDAD/data for
  the mesonets you want to filter (APRSWXNETStation.txt as well as any
  other mesonets you want to be filtered).  Then run the 
  build_station_idx.pl script to create an "include list" that specifies 
  only the stations that are in the rectangle:

  - cd /ldad/madisfilter
  - ./build_station_idx.pl

  The include list that's created is called "included_stations.txt".
  The script will also save the previous version of that file as
  "included_stations.txt.old", so you can go back if something goes
  wrong.

  * Whenever you update your LDAD station tables on px2, you will also
    need to update them here on ls1, and rerun build_station_idx.pl.


- Add this entry to your /data/fxa/LDAD/data/LDADinfo.txt:

     APRSWXNET |APRSWXNET |87    |0   |CVS_TYPE |mesonet  | |


- FTP real-time data access:

  Access the data files on rftp in wfo/LDAD/mesonet/csv on the regular schedule 
  shown below:

    Minute                 File
    ------                 ----

    7, 22, 37, 52          APRSWXNET.dat

  You should put the files into any directory *other* than /data/Incoming,
  then run them through the filter script:

   /ldad/madisfilter/MADISfilter.pl APRSWXNET.dat


- LDM real-time data access:

  LDM users should add the following ldmd.conf entry (if you request
  data from a regional server, specify that server):

  request FSL5 "LDAD\.raw\.APRSWXNET" ldm.madis-data.noaa.gov

  Make sure you have this entry in your pqact.conf file (with tabs between
  the fields):

  FSL5	^LDAD\.raw\.(APRSWXNET\..*)	PIPE	-close /ldad/madisfilter/MADISfilter.pl APRSWXNET.dat

  To activate the changes in pqact.conf:  ldmadmin pqactHUP.


- Determine if you want this dataset, and/or any other mesonet datasets,
  in your hydrological database.  If so, get README.shef_encode from
  rftp, docs directory, and follow the instructions in that file.

  If you don't want any mesonet data in the hydrological database:

  Edit the /data/fxa/LDAD/data/LdadPatterns.txt file so that the line
  for the routerShefEncoder only specifies mesonets that don't exist,
  e.g.:

     routerShefEncoder      .*(\.not_a_mesonet)\.decoded


- Restart ldad on px2.


Notes
-----

The station table is updated on rftp once a week on Wednesdays,
sometime during Colorado business hours.  You should set up a cron job
to automatically download the new files.  In order to have maximum
control over the progressive disclosure on AWIPS, you'll need to run
the "station" localization task on your workstations every time you
update the file.  However, you can display the data (even with
progressive disclosure) without doing the localization.  For the
station table changes to take effect in the LDAD ingest you will have
to run the station localization task on px2, then restart ldad.

The descrip file doesn't change on a regular basis, but may change
occasionally if new variables are added, or the data provider changes
the format, etc.  We will notify you of any changes, and you can then
get a new descrip file from our ftp server, and restart LDAD to get
the changes to take effect.

Please address any questions to madis-support@noaa.gov.