Forums

Back

Weird Behavior in GFE with wexp RTMA grids starting around 8/1.

VS
Victor Stegemiller, modified 7 Months ago.

Weird Behavior in GFE with wexp RTMA grids starting around 8/1.

Youngling Posts: 3 Join Date: 6/25/13 Recent Posts

Hi RTMA Forum, 

Since the beginning of August We’ve seen some odd behavior with the RTMA grids we import from the NOMADS FTP server. These are the RTMA Western Expansion (wexp) grids, i.e.:

https://nomads.ncep.noaa.gov/pub/data/nccf/com/rtma/prod/rtma2p5.YYYYMMDD/rtma2p5.tHHz.2dvaranl_ndfd.grb2_wexp

We smart init the hourly temperature data into GFE, but it mostly stopped populating around 8/1/24.

All of the grids are showing up in D2D, and we can see them in the CAVE Product Browser.

The edex-ingest-smartinit logs show this error: 

WARN  2024-08-10 00:36:22,757 0940 [smartInitThreadPool-2] D2DGridDatabase: EDEX - Error getting grid slice for ParmID: t_FHAG2:PTR_GRID_D2D_NWRFCRTMA_20240810_0000 TimeRange: (Aug 10 24 00:00:00 GMT, Aug 10 24 01:00:00 GMT)
com.raytheon.uf.common.dataplugin.gfe.exception.GfeException: Error retrieving hdf5 record. PTR_GRID_D2D_NWRFCRTMA_20240810_0000: GFE domain does not overlap dataset domain.

(NWRFCRTMA is our label for the wexp RTMA).

There have been no changes to our GFE configuration since well before this issue started.

Curiously we do get an occasional hour populating in GFE though, again with no changes to our config, so we’re wondering if there could be some change in the grids themselves.

This problem is not affecting the other URMA/RTMA grids that come in via the SBN, or the URMA WEXP grids we get from NOMADS. Just the NOMADS RTMA WEXP.

We have a NCF ticket, #A391213 associated with this problem. 

Do you all know if there has been a change to the wexp grids posted on NOMADS that might have happened around 8/1 that would be causing this error? 
 

MM
Matthew Morris, modified 7 Months ago.

RE: Weird Behavior in GFE with wexp RTMA grids starting around 8/1.

Youngling Posts: 169 Join Date: 12/6/17 Recent Posts
Hi Victor,

At this point, we suspect that this is an issue with GFE.  Have you received any updates on your NCF ticket?

I retrieved the "rtma2p5.tHHz.2dvaranl_ndfd.grb2_wexp" files from NOMADS for August 31st.  These files are bitwise identical to those in the NCEP tape archive, and I was also able to generate graphics in Python.  Thus, there doesn't appear to be an issue with the files on NOMADS.

Thanks,
Matt

On Wed, Aug 28, 2024 at 3:11 PM VLab Notifications <VLab.Notifications@noaa.gov> wrote:

Hi RTMA Forum, 

Since the beginning of August We’ve seen some odd behavior with the RTMA grids we import from the NOMADS FTP server. These are the RTMA Western Expansion (wexp) grids, i.e.:

https://nomads.ncep.noaa.gov/pub/data/nccf/com/rtma/prod/rtma2p5.YYYYMMDD/rtma2p5.tHHz.2dvaranl_ndfd.grb2_wexp

We smart init the hourly temperature data into GFE, but it mostly stopped populating around 8/1/24.

All of the grids are showing up in D2D, and we can see them in the CAVE Product Browser.

The edex-ingest-smartinit logs show this error: 

WARN  2024-08-10 00:36:22,757 0940 [smartInitThreadPool-2] D2DGridDatabase: EDEX - Error getting grid slice for ParmID: t_FHAG2:PTR_GRID_D2D_NWRFCRTMA_20240810_0000 TimeRange: (Aug 10 24 00:00:00 GMT, Aug 10 24 01:00:00 GMT)
com.raytheon.uf.common.dataplugin.gfe.exception.GfeException: Error retrieving hdf5 record. PTR_GRID_D2D_NWRFCRTMA_20240810_0000: GFE domain does not overlap dataset domain.

(NWRFCRTMA is our label for the wexp RTMA).

There have been no changes to our GFE configuration since well before this issue started.

Curiously we do get an occasional hour populating in GFE though, again with no changes to our config, so we’re wondering if there could be some change in the grids themselves.

This problem is not affecting the other URMA/RTMA grids that come in via the SBN, or the URMA WEXP grids we get from NOMADS. Just the NOMADS RTMA WEXP.

We have a NCF ticket, #A391213 associated with this problem. 

Do you all know if there has been a change to the wexp grids posted on NOMADS that might have happened around 8/1 that would be causing this error? 
 


--
Victor Stegemiller RTMA/URMA Discussion Group Virtual Lab Forum https://vlab.noaa.gov/web/715073/discussions-forums-/-/message_boards/view_message/39684167VLab.Notifications@noaa.gov


--
Matthew Morris
SAIC at NOAA/NWS/NCEP/EMC
5830 University Research Ct., Rm. 2038
College Park, MD 20740
301-683-3758
VS
Victor Stegemiller, modified 7 Months ago.

RE: Weird Behavior in GFE with wexp RTMA grids starting around 8/1.

Youngling Posts: 3 Join Date: 6/25/13 Recent Posts
Matthew Morris:
Hi Victor,
 
At this point, we suspect that this is an issue with GFE.  Have you received any updates on your NCF ticket?
 
I retrieved the "rtma2p5.tHHz.2dvaranl_ndfd.grb2_wexp" files from NOMADS for August 31st.  These files are bitwise identical to those in the NCEP tape archive, and I was also able to generate graphics in Python.  Thus, there doesn't appear to be an issue with the files on NOMADS.
 
Thanks,
Matt
 
On Wed, Aug 28, 2024 at 3:11 PM VLab Notifications <VLab.Notifications@noaa.gov> wrote:

Hi RTMA Forum, 

Since the beginning of August We’ve seen some odd behavior with the RTMA grids we import from the NOMADS FTP server. These are the RTMA Western Expansion (wexp) grids, i.e.:

https://nomads.ncep.noaa.gov/pub/data/nccf/com/rtma/prod/rtma2p5.YYYYMMDD/rtma2p5.tHHz.2dvaranl_ndfd.grb2_wexp

We smart init the hourly temperature data into GFE, but it mostly stopped populating around 8/1/24.

All of the grids are showing up in D2D, and we can see them in the CAVE Product Browser.

The edex-ingest-smartinit logs show this error: 

WARN  2024-08-10 00:36:22,757 0940 [smartInitThreadPool-2] D2DGridDatabase: EDEX - Error getting grid slice for ParmID: t_FHAG2:PTR_GRID_D2D_NWRFCRTMA_20240810_0000 TimeRange: (Aug 10 24 00:00:00 GMT, Aug 10 24 01:00:00 GMT)
com.raytheon.uf.common.dataplugin.gfe.exception.GfeException: Error retrieving hdf5 record. PTR_GRID_D2D_NWRFCRTMA_20240810_0000: GFE domain does not overlap dataset domain.

(NWRFCRTMA is our label for the wexp RTMA).

There have been no changes to our GFE configuration since well before this issue started.

Curiously we do get an occasional hour populating in GFE though, again with no changes to our config, so we’re wondering if there could be some change in the grids themselves.

This problem is not affecting the other URMA/RTMA grids that come in via the SBN, or the URMA WEXP grids we get from NOMADS. Just the NOMADS RTMA WEXP.

We have a NCF ticket, #A391213 associated with this problem. 

Do you all know if there has been a change to the wexp grids posted on NOMADS that might have happened around 8/1 that would be causing this error? 
 


--
Victor Stegemiller RTMA/URMA Discussion Group Virtual Lab Forum https://vlab.noaa.gov/web/715073/discussions-forums-/-/message_boards/view_message/39684167VLab.Notifications@noaa.gov
 
 
--
Matthew Morris
SAIC at NOAA/NWS/NCEP/EMC
5830 University Research Ct., Rm. 2038
College Park, MD 20740
301-683-3758

 

Hi Matt, 

Thank you for looking into this for us. I think you are right, that this is not a NOMADS grid problem.

We were able to fix this problem by re-editing CAVE parameter files, using the Data Add-ons Manager (DAM): https://vlab.noaa.gov/redmine/projects/nwsscp/repository/show/Edex/DamConfigNws

 

Thanks again, -Victor

Victor Stegemiller
Hydrometeorologist
Northwest River Forecast Center
Portland OR