Improving Forecasts of Severe Convection through Real-Time Sensitivity-Based Ensemble Adjustment within an FV3 Framework |
Texas Tech University |
|
Brian Ancell and Christopher Weiss |
May 2020 |
May 2024 |
Ensemble sensitivity reveals the flow features at early forecast times that are relevant to the predictability of high-impact forecast aspects later in the forecast window. Previous research at Texas Tech University funded through the CSTAR program has exploited this characteristic of ensemble sensitivity to extract information within ensembles specific to the prediction of severe convection and its individual hazards to improve probabilistic forecasts. More specifically, ensemble subsets chosen by retaining members with the smallest errors in sensitive regions have been shown to beneficially adjust forecast probabilities of severe convection for 12-48hr forecasts, showing substantial promise for an operational sensitivity-based subsetting tool at the NWS. Further, the technique has been evaluated at the Hazardous Weather Testbed Spring Forecast Experiment, where the majority of participant feedback supported both the success of the technique and its usefulness in an operational environment. A main theme of this work is to refine the subsetting technique to apply generally and provide consistent benefits to forecasts of severe convection within an FV3 modeling framework.
While ensemble sensitivity-based subsetting has been shown to significantly improve forecasts, achieving that result required rigorous experimentation with limited convective parameters such as the magnitude and coverage of updraft helicity and simulated reflectivity. Thus, the first primary goal of this work will be to refine the subsetting technique in a way that it can be generally applied successfully in an operational environment. This will involve expanded testing of different time windows, spatial areas, and severe convective variables (e.g. hail size, lighting, surface windspeed, quantity of precipitation, mode, timing) to gain a complete understanding of the utility of the method. In addition to the linear regression involved with ensemble sensitivity, more advanced techniques such as nonlinear regression and machine learning will be explored to understand if further benefits to convection forecasts can be realized through more complex real time data mining and subsequent ensemble adjustment. Extending ensemble sensitivity analysis to warn-on-forecast time and space scales is also a major goal; this is expected to yield a tool that can be integrated into the NOAA Warn-on-Forecast ensemble framework to realize the full potential probabilistic forecast-based warnings can provide.Formulating ensemble sensitivity analysis and the subsetting technique within an FV3 framework will be of utmost importance in this project given the research-to-operations theme of this work and the recent switch of NOAA operational products to a unified FV3 modeling system. Substantial collaboration with several NWS offices, the SPC, and NSSL will be a key aspect of this project and will stem from previous collaborations during prior CSTAR work at Texas Tech University.
|
|
|
Extending the Forecast Lead-Time of Pulse Severe Storms using ProbSevere, GLM, and Radar Data |
Florida State University |
|
Henry Fuelberg |
May 2020 |
May 2024 |
Pulse severe thunderstorms are a major warm season forecast problem, especially in the Southeast. Pulse severe storms are considered to be ordinary single cell thunderstorms that produce severe winds or hail for a brief period of time. Forecasters presently do not have sufficient guidance to know which cell will become severe or when that sudden transition will occur. This is a major limitation to the timely issuance of warnings. Often, by the time a forecaster realizes that a cell has reached severe limits and issued a warning, the severe phase often is about to end or already has ended. Thus, the lead time for issuing warnings for pulse severe storms is short compared to other types of convection.
The time is right to make a renewed effort to better understand pulse severe storms and develop guidance techniques for improving their detection and warning lead times. This project will use output from the NOAA/CIMSS ProbSevere model, data from the Geostationary Lightning Mapper (GLM), high spatial and temporal resolution model output, and radar data to seek indicators that an ordinary cell thunderstorm in the Southeast United States will soon become severe. This project has five objectives:
1. Do any of the four basic GLM parameters (Max FED, Max TOE, Avg AFA, Sum FCD) provide information about which pulse storms will become severe and when it will occur?
2. Would a statistically derived combination of these four GLM-derived parameters provide useful guidance for NWS forecasters?
3. Does ProbSevere (specifically ProbWind and ProbHail) perform well or poorly for pulse severe storm events?
4. After training a new algorithm on single cell storms and pulse severe storms, are the results superior to those from the present algorithm?
5. Does the inclusion of dual-polarized radar parameters help predict the damaging winds and hail often associated with pulse severe storms?
|
|
|
Collaborative Research to Advance Probabilistic Forecasting and Hazard Assessment in Mountainous Regions |
University of Utah |
|
James Steenburgh, John Horel, and Courtenay Strong |
May 2020 |
May 2024 |
This project will advance National Weather Service (NWS) forecast and hazard assessment capabilities through collaborative research addressing the CSTAR science and technology theme, “improving the lead-time and accuracy of forecasts and warnings for high-impact weather, water, and climate events.” Focusing on the western contiguous United States, we plan: 1) to advance the capabilities and expand the use of ensemble-based probabilistic quantitative precipitation forecasts by field offices and in the National Blend of Models, and 2) to develop advanced data fusion and data analytical techniques to assess hazardous atmospheric conditions from observations and numerical forecasts. The former includes the testing and development of a new orographic precipitation gradient model (OPGM) for downscaling precipitation forecasts using regime-dependent precipitation-altitude relationships, and the latter involves the development of new techniques for utilizing massive volumes of probabilistic information from observations and models. We will also continue efforts to improve the quality control of mesonet observations. These research activities address several CSTAR program priorities including: (1) improving the application of numerical weather prediction (NWP) information in the forecast and warning process at various time scales, (2) improving the use of ensemble prediction systems in order to enable more effective forecaster assessment of uncertainty and historical context of potential high-impact events and develop probabilistic hazard information, and (3) developing improved surface analyses using data fusion to aid in the identification and characterization of high-impact events in complex terrain, including rain, snowfall, and terrain-driven wind events that often contribute to the growth of wild- fires. This research project addresses forecast and hazard assessment challenges in the NWS Western Region and other areas of complex terrain and builds on our ongoing O2R2O collaborations with the NWS Western Region Science and Technology Infusion Division, NWS Weather Forecast Offices, and the National Centers for Environmental Prediction.
|
|
|
Mobilizing a Collaborative Community for Probabilistic Hazard Information |
University of Colorado-Denver |
|
Hamilton Bean |
May 2020 |
October 2022 |
Introduction of the Problem: The intersection of (a) Impact-Based Decision Support Services (IDSS), (b) Probabilistic Hazard Information (PHI), and (c) mobile public alert and warning technology creates new challenges and opportunities for improving public understanding and response to weather warnings. On one hand, NOAA/NWS intends to communicate PHI- enhanced warnings that may include indicators of lower confidence and higher uncertainty. On the other hand, we know from decades of research concerning public alert and warning that protective action compliance improves when warning messages are specific, consistent, confident, clear, and accurate (Mileti & Sorensen, 1990). The proposed project therefore provides social science evidence for how to best manage this tension. Making weather warning more “effective” has not only to do with improved PHI and related forecast capabilities among technical experts, but also with “translating” PHI in ways that improve public sensemaking and response (Miran, Ling, Gerard, & Rothfusz, 2019). The proposed project foregrounds how PHI can be integrated with developments in mobile public alert and warning (Bean, 2019).
Rationale: This project aims to generate empirical evidence concerning how to: (a) strengthen decision-making guidelines for the issuance of PHI-enhanced weather warnings as a component of IDSS by associating probability thresholds, time-of-arrival thresholds, and/or forecaster decision points to values within the Common Alert Protocol (CAP); (b) improve PHI-enhanced weather warning effectiveness between NWS Core Partners and publics, as well as among publics; (c) pinpoint inclusion criteria and formatting options for PHI-enhanced text and images within existing mobile public alert and warning platforms (i.e., Wireless Emergency Alerts, local opt-in systems, and apps); and (d) ascertain the value of rendering textual impact-based historical context alongside PHI content within mobile weather warning messages. These four objectives are operationally applicable and directly respond to the research roadmap laid out in the National Academies of Sciences, Engineering, and Medicine’s 2018 report, Emergency Alert and Warning Systems: Current Knowledge and Future Research Directions.
Brief Summary: This project centers upon the Collaborative Community Weather Information (CCWI) paradigm and involves a multifaceted stakeholder workshop in Denver, CO that incorporates pre-, during, and post-workshop interviews, surveys, and think-out-loud experiments. Pre-workshop efforts address rationale (a) above. The workshop itself addresses (b), and post-workshop efforts target (c) and (d). The workshop will explore how the inclusion of PHI might enhance the NWS WEA360 templates already developed (in English and Spanish). The workshop will involve three groups of stakeholders: (1) personnel from the Denver/Boulder WFO (support offered) and NOAA’s Emerging Dissemination Tech. Office (invited); (2) officials from the City and County of Denver (support offered); and (3) 48 community members from Denver (pending). The CCWI workshop elicits and ranks local weather hazard information held by residents, generates PHI-enhanced warning message content in response, and facilitates interaction to improve IDSS and communication of PHI through the collaborative construction of weather warning messages for mobile devices.
|
Yes
|
|
Improved Operational Prediction of Blowing and Falling Snow, and Extreme Wind Events in the Rocky Mountain Region and Northern High Plains |
University of Wyoming |
|
Bart Geerts, Zachary Lebo, and Larry Oolman |
June 2019 |
Nov 2023 |
Winter storms in Wyoming and surrounding areas often produce hazardous conditions for travelers, and impacts of high winds, blowing snow, and snow squalls continue to result in numerous road closures, chain-reaction pile-up crashes, and truck blow-overs each year. Here, we propose to work closely with at least four NWS Regional Forecast Offices and the HRRR (High- Resolution Rapid Refresh) development team to provide more specific, useful, and precise forecasts of high-impact winter weather in the region, building on the 3-km resolution HRRR model output.
In particular, we aim to (a) validate HRRR forecasts for extreme surface wind speeds and wind gusts near complex terrain; (b) use spatially resolved HRRR output for real-time prediction of extreme winds, including extreme crosswinds along highways and airport runways; (c) use HRRR output to develop a physically based blowing snow product as well as a snow squall product and to examine the accuracy of the HRRR forecasts in terms of the occurrence and intensity of blowing snow and snow squalls; (d) use HRRR output for fine-scale real-time prediction of blowing snow and snow squalls; and (e) conduct higher-resolution (1.0 km) convection- permitting HRRR-like WRF simulations, in case studies and in real time, to examine whether they can substantially improve the forecast of strong wind events, snow squalls, and blowing snow.
We plan to work with the NWS Regional Forecast Offices at all stages of this effort, including in the model validation and in the disseminating of this information to stakeholders using existing channels, e.g., the Pikalert® and the Wyoming Dept of Transportation Commercial Vehicle Operator Portal. We also plan to work with the HRRR development team at NOAA ESRL, mainly in the validation of the HRRR wind gust field, the improvement of the surface visibility algorithm, and the production of a blowing snow product. These products, as well as the web portal with user-friendly hazardous weather forecasts on zoomable Google maps with color- coded highways, will become part of the NWS’s toolkit upon completion of this project.
The main intellectual merit of this effort regards a better understanding of the characteristics and predictability of high-impact winter weather near complex terrain, a quantification of the marginal value of a 3x resolution refinement, and the development of a blowing snow algorithm that can be adapted to other models.
The broader impact of the project lies in more refined and more accurate predictions of adverse weather conditions to travelers out to 36 hours, saving lives, and allowing traffic management personnel in charge of road maintenance and closures as well as airlines, Interstate freight carriers, and individuals to plan ahead, thereby saving millions of dollars.
|
|
|
Characteristics and Evolution of Observed and Simulated Supercell Thunderstorms in the Central and Southern Appalachians |
UNC-Charlotte |
|
Casey Davenport and Matthew Eastin |
June 2019 |
May 2024 |
The Appalachian Mountains in the Eastern United States have a considerable impact on the day-to-day weather across the region. Significant weather events such as severe convection present a substantial challenge to forecast accurately when in the presence of complex terrain. This has been underscored by several high-impact supercell events in the Appalachian region, including those with significant tornadoes occurring at high elevations that caused damage and injuries. While there have been several studies that have explored terrain-induced effects on supercellular tornadoes and tornadogenesis, outside of individual case studies, few have more broadly examined the storm-scale modifications to a supercell thunderstorm as it interacts with topography. Additionally, the typical environments that produce severe wind, hail, or tornadoes across the elevated region has yet to be documented. This research project will assess numerous storm characteristics of supercells in the Appalachian region to provide better situational awareness for the numerous National Weather Service Forecasting Offices (WFOs) in the area, as well as the Storm Prediction Center (SPC), with the goal of enhancing short-term forecasts of supercells.
Utilizing numerous tornadic and non-tornadic supercell case studies within the central and southern Appalachians, the project will categorize each case according to a variety of climatological, environmental, and radar-based characteristics. One component will document numerous observed storm characteristics related to supercell longevity, organization, and severe weather production. Next, changes in radar-based measurements of storm structure and intensity will be correlated to variations in terrain height and slope; this information will then be related to the production of severe wind, hail, or tornadoes. Another component of the project will evaluate how the inflow environment varies as each supercell traverses through the mountains, and whether those changes can predict supercell behavior (e.g., production of severe weather, storm lifetime, etc.). Lastly, to engender the development of a generalized conceptual model to be used in operations and clarify the governing physics of how terrain can modify supercell structure and potential to produce severe weather, idealized simulations initialized with composite soundings from subsets of observed cases will be performed. A series of simulations will identify the sensitivity of storm intensity and morphology during transit over elevated terrain to variations in the near-storm environment using a base-state substitution technique; these changes will also be tested using varying terrain configurations. The results of these efforts will be regularly presented through a variety of means to WFOs in the Appalachian region, as well as to the SPC to enable forecasters to provide increasingly accurate forecasts of supercells.
|
|
|
Improving Analyses, Numerical Models, and Situational Awareness of High-Impact Severe Convective and Mixed-Phase Precipitation Events in Complex Terrain |
SUNY Albany |
|
Kristen Corbosiero |
June 2019 |
May 2024 |
This project is expected to lead to increased scientific understanding and improved forecast lead time, and accuracy, of high-impact weather events, including heavy snowfall and mixed-phase precipitation accumulations, damaging convective-scale winds, severe hail, and tornadoes, which have the potential to cause substantial societal and economic disruption. The weather events chosen will be addressed to focus on the following challenging forecast problems: 1) severe convection in complex terrain and across different severe-weather environments, and 2) winter-precipitation type in regions of complex terrain. Emphasis will be placed on exploring the representation of these high-impact weather events in current, and future, convection-allowing numerical weather prediction models (CAMs) compared to observations, understanding model biases related to poorly-resolved physical processes, and developing real-time data fusion products combining CAM output with observations to highlight areas of enhanced forecast uncertainty. Although our projects are focused on cold-season cyclones and convective weather events in the eastern U.S., our scientific methods and operational diagnostics will be formulated as to be transferrable to other regions of hazardous weather in complex terrain across the country.
Our research efforts on high societal impact weather systems are designed to facilitate the transfer of research findings into operations (R2O) by taking advantage of the well-established programmatic assets and research infrastructure in the Department of Atmospheric and Environmental Sciences (DAES) at the University at Albany (UAlbany), and by enabling the participation of a significant number of National Weather Service (NWS) personnel on CSTAR- related research. In order to achieve this objective, we will build upon the framework for NWS participation and R2O that has been adopted in our current and previously funded CSTAR grants, which allows for the cost-effective generation of new scientific understanding and improved Warn-on-Forecast (WoF) capabilties to protect life and property. These efforts will be conducted in collaboration with multiple Eastern Region NWS offices and the New York State Mesonet (NYSM), as well as NOAA’s Storm Prediction Center (SPC), Weather Prediction Center (WPC), and Earth System Research Laboratory (ESRL). These new and continuing R2O partnerships will: 1) ensure the development of conceptual models, forecast checklists, and decision trees at the completion of each project, 2) support promising visualization, diagnostic, and machine learning techniques to be formally evaluated in operational testbeds, and 3) ensure model biases and potential improvements are effectively communicated to developers.
|
|
|
Improving Snowband Risk Assessments through High-Resolution Ensemble Verification and Visualization |
North Carolina State University |
|
Gary Lackmann |
June 2019 |
May 2024 |
Mesoscale snow bands have been the subject of extensive research over the past few decades, owing in large part to their ability to impart significant societal impacts. In prior work, we developed an automated snowband detection algorithm, and applied it to produce an objective snowband climatology and to evaluate band predictability in the operational HRRR model. We also tested object-oriented band verification strategies. An additional component of the project explored two novel model diagnostics that show promise in the prediction of heavy snowfall: vertical snow flux and depositional snow growth. This project will extend each of these previous efforts into the realm of high-resolution ensemble prediction, with emphasis on maximizing the extraction of actionable hazard information from operational ensemble modeling systems.
Having developed and tested automated detection methods for mesoscale snowbands, we are able to run the detection algorithm on model forecasts, and for the case of convection-allowing ensembles, we can use the algorithm to generate probabilistic snowband predictions. Novel visualization strategies will be used for band forecasts, including paintball plots, glyphs, probabilistic heat maps, and ensemble surface slicing; we will draw on visualization and science communication expertise at NC State in other departments to optimize these displays. User ability to interpret such graphics is equally important to the development of the visuals. In this regard, we will take advantage of the recent addition of science-communication expertise at NC State to design and test graphics that maximize comprehension of forecast uncertainty among varied audiences.
Modern numerical weather prediction models often include sophisticated cloud and precipitation microphysical parameterizations (MP). Crucially important microphysical quantities, such as depositional snow growth, are computed with high accuracy in the Thompson scheme, but is not written as output. We modified a recent release of the Weather Research and Forecasting model, configured similarly to the HRRR operational model, to output the vertical snow flux and the depositional and total snow growth. The vertical snow flux reveals distinct signatures within and upwind of heavy snowfall, with lofting of snow being ubiquitous near mesoscale snowbands. Furthermore, forecasters often utilize visualization software such as BUFKIT to subjectively evaluate the presence or absence of depositional snow growth. Our preliminary results indicate that a more complete examination of this quantity, output directly from the HRRR model, would be useful to forecasters if visualized as a 2- or 3-dimensional field. For an ensemble, this quantity could be combined with the aforementioned snowband visualization techniques to provide forecasters with a more complete picture of the microphysical structure of winter storms.
During previous collaborative research, we have established partnerships at the Weather Prediction Center (WPC), the Environmental Modeling Center (EMC), and several NWS Forecast Offices. We have participated in the WPC Winter Experiment in the past, and have been invited to participate again this winter (February 2019). Such participation provides an ideal opportunity to implement and test our methods in an operational setting.
|
|
|
Understanding fundamental processes and evaluating high-resolution model forecasts in high-shear low-CAPE severe storm environments |
North Carolina State University |
|
Matthew D. Parker and Gary M. Lackmann |
June 2017 |
June 2022 |
Severe storms in the Southeastern U.S. are associated with higher fatality rates and lower warning skill scores than those in the Plains. Many such events occur in environments characterized by large environmental vertical wind shear (“high shear”) but weak instability (“low CAPE”). These high-shear low-CAPE (“HSLC”) conditions are associated with both low predictability and high tornado warning false alarm rates. Our regional NWS collaborators have identified these issues as among the most pressing concerns for their WFOs. This project addresses several remaining gaps in our understanding of HSLC severe weather predictability and facilitates an effective transition of the results into operations. The NCSU research team is particularly well prepared to undertake work on this problem, having conducted a number of CSTAR studies specifically targeting HSLC severe weather, and having a long-established relationship with many regional NWS WFOs.
For short-term HSLC forecasting, our regional NWS partners now use a variety of convection-allowing models (“CAMs”, e.g. the 4-km NAM, HRRR, NCAR 3-km ensemble, SSEO). However, the interpretation of CAM products (e.g., simulated updraft helicity) is hindered by a lack of systematic verification against observations, particularly for null cases. Previous work by the PIs demonstrates the potential utility of CAMs in HSLC prediction, although results have not been evaluated for a sufficiently large event sample. Our NWS partners have also communicated to us that they have great uncertainty about the processes they are observing on radar during HSLC tornadogenesis, which ultimately leads to many false alarm warnings. Both of these gaps in the knowledge base undermine forecaster confidence and skill.
To address these challenges and connect the new knowledge to operational practice, the following coordinated research projects will be conducted: 1) observational verification of CAM NWP performance (archived operational CAM output and case study simulations) during HSLC events vs. nulls; 2) hypothesis-driven process studies of HSLC tornadogenesis using an idealized numerical model, with implementation of a radar emulator that links model output to WSR-88D radar data in a physically consistent way. Each of these projects represents an advance within an area in which we have completed promising preliminary work, and where additional effort is likely to yield operational benefit in challenging and common forecast scenarios.
These emphases will facilitate the transition of previous and ongoing CSTAR research into thoughtful operational use. The expected outcomes of this approach are: 1) specific guidance to forecasters on the strengths, weaknesses, and biases of CAM NWP products that are already in widespread (but speculative) use in HSLC environments, and potentially, HSLC customization of severe-proxy CAM output diagnostics; and, 2) an improved conceptual understanding of HSLC tornadogenesis that is tightly linked to the radar observations that are available to forecasters in real time.
|
Yes
|
|
Better Use of Ensembles in the Forecast Process: Scenario-Based Tools for Predictability Studies and Hazardous Weather Communication |
SUNY Stony Brook |
|
Brian A. Colle and Christine O'Connell |
July 2017 |
July 2022 |
The 2011 Strategic Plan for the National Weather Service (NWS) emphasizes the goal of a “Weather-Ready Nation,” which will help the public and stakeholders better synthesize NWS forecasts in order to protect life and property. The plan emphasizes the need for more probabilistic products and their integration into various public products and services. Ensemble weather prediction is an important component of this decision-making process, but there is a general lack of tool/graphics to display ensemble data and ensemble training for those tools. Forecasters need more opportunities to interact with ensemble data other than the conventional mean, spread, and probabilistic products. The science motivation is how to effectively combine and validate information from multiple models/ensembles together to better understand the predictability of high impact weather. The practical motivation is how this information can be effectively communicated to stakeholders to render them more useful.
The utility of these ensemble forecasts are only as good as the communication of the predictions in order for users to take appropriate action. There are numerous challenges: (a) Communicating to different types of users; (b) media wanting short bursts of information (sound bites); and (c) how to communicate uncertainty information. The public needs multiple information sources and layers to take action and current decision support tools are often not probabilistic. In order to help the NWS communicate information there needs to be more training and workshops helping the forecaster distill the message and communicate in ways that users will understand.
This project will address CSTAR objectives #1 “Improving the lead-time and accuracy of forecasts and warnings for high impact weather -- Improving the use of ensemble predictions systems in order to enable more effective forecaster assessment of uncertainty”, #2 “Improving Impact-Based Decision Support Services”, and #3 “Improving water resource information (precipitation) for decision support and situational awareness” Our focus area is the Eastern U.S. for high impact weather during the cool season; however, our approach can be expanded to other parts of the country and phenomena. The primary goals are: (1) To extend our newly developed fuzzy clustering approach to high impact weather events including precipitation, freezing level, and 10-m wind for days 1-7 using the short-range and global ensembles; (2) Expand our new spread-anomaly ensemble tool; (3) Use these tools to verify these phenomena in the ensembles and understand the large-scale flows attached to the less predictable events; and (4) Integrate the Alan Alda Center for Communicating Science (www.aldacenter.org) into our CSTAR work to help forecasters better communicate probabilistic information through a series of three workshops, some of which involving stakeholders. Partners include several NWS offices and operational centers at National Centers for Environmental Prediction (the Weather Prediction Center, Environmental Prediction Center, and Ocean Prediction Center), as well as the Social Science group within the NWS.
|
Yes
|
|
Improving Situational Awareness of Impactful Post-Fire Debris Flows |
Nevada - Desert Research Institute |
|
Benjamin Hatchett |
June 2019 |
May 2022 |
Post-fire debris flows have long threatened life and property situated within the complex and fire-prone terrain of the western United States. Several destructive, and in some cases deadly, events have been observed in the West in recent history. With recent prolonged droughts, large and high-intensity wildfires and population expansion into the wildland-urban interface, post-fire debris flow hazards have emerged as a major concern for Weather Forecast Offices (WFOs) covering areas of complex terrain. There is an expressed need for tools and information to help hydrologists and forecasters provide impact-based decision support services (IDSS) with respect to post-fire debris flows.
We have selected two areas of focus to improve forecasting capabilities, forecaster confidence, and IDSS. First, because of its high spatial and temporal resolution, the High Resolution Rapid Refresh (HRRR) model is used operationally by WFOs to assess post-fire debris flow hazards and is used in Western Region’s new HRRR-debris flow tool. We propose to evaluate HRRR performance. Second, we assess methods for determining the likelihood of an incoming storm to produce minor debris flows or ones that have devastating impacts on life, property, and infrastructure. We address these topics through three tasks:
Task 1: Quantify performance of the HRRR model in terms of orientation, timing, and propagation of features producing rainfall exceeding USGS debris flow thresholds at different lead times during various storm events across the Southwestern U.S. Evaluate whether any systematic biases exist in HRRR precipitation simulation in this region, and assess whether HRRR performance is improved under certain atmospheric conditions.
Task 2: Use rain gauges in the vicinity of select burn areas to identify all storm events that exceeded the USGS debris flow threshold (may precede fire occurrence). Use the National Severe Storms Laboratory’s Multi-Radar Multi-Sensor (MRMS) precipitation estimates for all relevant events as an input to existing post-fire debris flow likelihood and volume models. Compute the total number of drainage basins expected to produce debris flows and the total volume of sediment mobilized by debris flows as a result of each storm, and relate these outcomes to storm characteristics (i.e., is it frontal precipitation, isolated convection, etc.).
Task 3: Perform high-resolution runoff and debris flow simulations at the drainage basin scale to quantify the sensitivity of post-fire debris flow magnitude to changes in rainfall intensity and duration within the different geologic and climatologic settings represented by the four chosen burn areas. Use simulations of 15-minute precipitation time series based on historic storms observed in the area as well as vegetation and soil hydrologic parameters associated with various western US landscapes.
Results will provide a better understanding of the storm types and characteristics associated with impactful and non-impactful post-fire runoff and debris flow events. Further, we will develop framework for a model capable of assessing whether a forecast storm will produce impactful or minor debris flow activity. Lastly, results will provide insight into our current ability to forecast debris-flow-producing rainfall rates at different lead times. Through all tasks, we will engage with various WFOs and Western Region Headquarters to share information, establish lasting partnerships, and develop materials for communicating debris flow hazards.
|
Yes
|
|
Determining Criteria for Messaging NWS Red Flag Warnings |
Nevada - Desert Research Institute |
|
Timothy Brown and Tamara Wall |
June 2019 |
May 2022 |
The National Weather Service (NWS) is currently exploring alternatives to more effectively communicate hazard messages under the Hazard Simplification project. For wildland fire management and the public, the primary fire hazard messaging is the Fire Weather Watch and the Red Flag Warning. The current Analyze, Forecast and Support (AFS) Service Program high level requirements includes several elements that support the need to define threshold criteria for messaging Red Flag Warnings. These include among others: 1) Create, adapt, and improve holistic end-to-end dissemination and messaging to improve effective Red Flag Warning messaging for both fire management and the public; 2) Improve consistency for fire weather hazard messaging to support effective firefighting response and positioning of resources; 3) Consistent methodology for offering gridded forecasts via National Blend of Models; and 4) Transform from product-centric to interpretive services to improve fire weather products that efficiently communicate high risk fire days.
The need for a revamped Red Flag Warning has been expressed over recent years by both fire management and fire weather meteorologists. It is widely recognized that Red Flag Warnings are often issued too frequently, thus increasing the risk of the intended audience ignoring the hazard communication, and that Red Flag Warnings are not actionable. In response to these concerns and as part of the AFS requirements highlighted above, an NWS workshop was held in Boise, ID in September 2018 that brought together NWS fire weather personnel, USFS researchers, and academic researchers to examine needs and potential research for improving both consistency and messaging of the Red Flag Warning product. Changing the Red Flag Warning product has challenges not seen in most other NWS hazard messaging products because it needs to include aspects of both weather and vegetation conditions, and may or may not include the presence of a wildfire. The Boise workshop successfully helped identify some possible product definitions such that follow-up physical and social science research work can determine and assess the needed inputs and criteria to inform the development of a new Red Flag Warning definition.
An outcome of the Boise workshop was identifying that a watch/warning decision matrix with quantitative thresholds could be developed providing both watch and warning criteria serving both fire management and the public. But critical to these thresholds is the utilization of fuels related information (e.g., fuel moisture, fire danger) that, combined with weather, indicates threshold levels of fire behavior. Of particular interest are conditions that yield extreme fire behavior that threatens firefighter and public safety. This translates to providing fire management with operational decision breakpoints related to staffing levels and resources needed for firefighting efforts with respect to fire behavior levels, and utilizing these breakpoints for public warning. Four project tasks will be undertaken: 1) Quantitative analyses of fire weather-danger-behavior indices to determine the best consistent set of inputs needed for established breakpoint criteria for fire weather watch/warning decisions; 2) Determine breakpoint criteria linked to fire management and public notifications and actions necessary for safety; 3) Identify potential messaging for communicating forecast confidence and uncertainty of probabilistic hazard information to fire management; and 4) With these inputs, develop a prototype decision matrix with NWS, fire management, and emergency services.
|
Yes
|
|
Investigation and Forecast Improvements of Tornadoes in Landfalling Tropical Cyclones |
Texas A&M University |
|
Christopher Nowotarski and Matthias Katzfuss |
June 2019 |
May 2023 |
Tornadoes in recent landfalling tropical cyclones (TCs) in the United States underscore the threat these phenomena pose to society and the unique forecast challenge they present to operational forecasters. Despite a fairly robust body of research in this area, significant gaps in our knowledge remain regarding the tropical cyclone tornado (TCTOR) climatology, radar-based storm attributes, and near-cell environments of tornadic and nontornadic convective cells in TCs. Moreover, recent improvements in observational networks (e.g., nationwide dual-polarization radar) and high-resolution operational models afford opportunities to study these phenomena in greater detail.
Leveraging and expanding existing collaborations between the NWS and Texas A&M, this study seeks to advance our understanding of TCTOR cell attributes and environments, focusing on differences between verified tornado warnings and false alarms. The second major goal of this project is to improve the operational forecasting and warning decision process through integration of observed cell attributes and modeled near-cell environments. Specific objectives of this project include:
1: Build a database of all tornadoes and tornado warnings in TCs in the United States since the NEXRAD dual-polarization upgrade that includes radar-based storm attributes and near-storm environment information from model analyses.
2: Assess the skill of high-resolution model analyses and forecasts in depicting 1) the low-level, near-cell environment for convective cells in TCs and 2) forecast proxies for low-level rotation (e.g., updraft helicity).
3: Compare near-cell environment and storm attribute information between verified warnings and false alarms in the climatology to determine differences that may be leveraged to reduce false alarms.
4: In partnership with NWS collaborators, assess the performance of current radar, high-resolution NWP, and storm-environment based TCTOR forecasting practices and heuristics.
5: In partnership with NWS collaborators, improve and streamline TCTOR warning practices using information gained from the climatology, including development and evaluation of probabilistic hazard information (PHI) produced by a statistical model trained on data produced in our climatological database.
|
|
|
Addressing Geographical and Social Diversity in Heat-Health Messaging |
University of South Carolina |
|
Kirstin Dow |
May 2022 |
April 2025 |
The National Weather Service Forecast Office in Columbia, SC (CAE) is responsible for providing forecast and warning information for hazardous weather for central South Carolina and east central Georgia, a region known as the Midlands. This area includes cities, such as Columbia, SC and Augusta, GA, as well as large rural and agricultural areas. Within the CAE forecast area, ASOS observation stations are relatively sparce in rural areas. While hurricanes and tornadoes grab the headlines as significant weather-related hazards, excessive heat is a more common cause of illness and fatalities (NWS 2021b). While overall forecast capabilities have improved, approaches to communicating heat risks reflecting the needs of the user, including existing health concerns, and the broader decision context informed by multiple sources of health information are often missing. The relative lack of observational data and the gap in service are particularly significant for historically underserved and socially vulnerable communities (HUSVCs). In these areas there is an especially high prevalence of health concerns that increase susceptibility to heat risks, less access to medical care, and local geography and land cover can lead to higher heat values than currently captured by observational systems and forecast thresholds for warnings.
This project addresses three CSTAR program priorities: 1) improving weather, water, and climate services to historically underserved and socially vulnerable communities (HUSVCs); 2) enhancing NWS services for HUSVCs at greater risk to negative impacts of heat; and 3) developing new messages and innovating communication processes to deliver forecasts and protective messages. To address the understanding of geographic variability in not only the need for improved observational systems, but also in a multilayered approach to communicating heat risk information for HUSVCs where high prevalence of health issues exacerbates heat risk, we have established five project objectives.
1. Characterize variability of WBGT across different land cover areas.
2. Evaluate WBGT estimation methods with different inputs and compare WBGT against the standard NWS heat index-based advisories and experimental products.
3. Identify the pre-existing attitudes and behaviors of underserved populations including their awareness of heat-health risks, sources of information about heat-weather, and frequency in accessing such information.
4. Develop a NWS message on heat forecasts and preparedness designed to meet the needs of members in the Midlands Public Health Preparedness Coalition.
5. Identify populations with greater susceptibility to heat based on geographic variability and pre-existing conditions.
A University of South Carolina-based team will work with the NWS CAE forecast office and the SC Department of Health and Environmental Control’s Public Health Preparedness Office, in advancing heat warnings to underserved populations through a multi-disciplinary approach most closely aligned with program priority of enhancing NWS forecasts and services for HUSVCs that are at greater risk of experiencing negative impacts due to heat exposure. This full team has a range of research experiences in climate and weather, hazards and warning messaging, and identifying vulnerable communities, in addition to translating science to practice by working
directly with emergency managers and public health providers at state and local levels.
|
|
|
Communication with Highly Vulnerable Societal Groups through Partnerships, Audience Analysis, Crowd-Sourced Information, and Workshops |
SUNY Stony Brook |
|
Brian Colle |
May 2022 |
April 2025 |
Many metropolitan areas, such as New York City and Long Island, NY (NYC-LI) are highly vulnerable to weather extremes, such as storm surge (e.g. Sandy 2012), urban flash floods (e.g. Ida on 2 September 2021), heavy snow, and damaging winds. These events cause major societal impacts, especially for highly vulnerable society groups (HVSGs). Impact-based Decision Support Services (IDSS) within the National Weather Service (NWS) would benefit by knowing how HVSGs perceive risk, where they obtain their information about the impending weather hazard, and improving the communication between various stakeholders (e.g. emergency managers), community leaders, and residents. Research will first focus on work with NYC Emergency Mangers (EMs), NWS, city officials, and others to connect with HVSG communities, initiate community listening through focus groups and interviews, and learn from HVSG community members about the challenges they encounter in taking protective action in the face of weather hazards. Building on step 1, researchers will conduct a survey to better identify HVSGs within the NYC Metro area and determine profiles based on their geolocation, demographic information, media consumption habits, knowledge of extreme weather hazards, risk perceptions, as well as emotional and behavioral reactions to extreme weather events to improve our understanding of the human psychology and behavior within these HVSGs during these extreme weather events. To better obtain a broader understanding of perception of risk and preparedness, this project will also modify and utilize a unique Smartphone ("WeatherCitizen") app for crowd-sourced information gathering and distribution. A Protective Action Decision Model (PADM) will be utilized and evaluated, which is based on findings derived from research on human responses to environmental hazards and disasters. The PADM provides a theoretical framework for exploring the factors that influence an individual's protective actions. We will improve communication between community
officials and residents within these HVSG communities, as well as with the NWS forecast offices, for these extreme weather events through workshops and relationship-building. We will utilize the Alan Alda Center for Communicating Science at Stony Brook to host two online workshops. In year 2 workshop #1 will bring forecasters and HVSG community representatives together to learn about warning dissemination and preparedness in these communities (using also results from our survey results). In year 3, workshop #2 will focus on the most appropriate messaging strategies and channels to use in communicating with HVSGs as well as ways to improve these communication components. Our approach, in a collaboration with the NWS Social Science group, will be transferable to other metropolitan locations.
|
|
|
Increasing the Reach and Effectiveness of Heat Risk Education and Warning Messaging in HUSVCs |
Nevada - Desert Research Institute |
|
Kristin VanderMolen |
May 2022 |
April 2024 |
Extreme heat is a major public health issue and the leading cause of weather-related mortality in the United States. Exposure to extreme heat can cause numerous direct and indirect health impacts, ranging from dehydration and heat stroke to complications of cardiovascular, renal, respiratory and other bodily systems. In this context, different strategies have been proposed to mitigate the health impacts associated with extreme heat. The most common strategy is to develop heat warning systems. In the US, heat warning systems (also known as heat alerts) are disseminated by the National Weather Service (NWS) to state and local governments and other partner organizations to inform operational and/or service response, and to the public to inform individual protective action. However, research has reported that in the absence of targeted heat risk education in vulnerable (or historically underserved and socially vulnerable communities, HUSVCs), the reach and effectiveness of heat warning systems is lacking. This suggests that the full public health potential of NWS heat warning systems has yet to be realized. It also points to a critical need to develop methods for connecting HUSVCs to heat-related information, resources, and services in conjunction with warnings to achieve that end. The goal of this project is to increase the reach and effectiveness of heat risk education and warning messaging in HUSVCs through the development, implementation, and evaluation of a targeted heat-health
train-the-trainer (TTT) curriculum piloted in collaboration with the County of San Diego’s Resident Leadership Academy (RLA) Network and similar partner networks. TTT is a widely acknowledged educational model, particularly for its use in teaching public health preparedness,
wherein lay instructors, or “peer- trainers,” receive instruction on specific content and how to teach it to others. The RLA Network is a county-sponsored TTT program that provides local leaders in HUSVCs with the knowledge and tools for improving health and quality of life at the
community-level. Specific objectives to achieve the project goal include: (1) develop a heat-health TTT curriculum for use within the RLA Network (and partner networks) focused on increasing education and awareness of NWS, county, and other heat-related information, resources, and services, that will consider both differential needs across communities and micro-climate zones; (2) train a sample group of 10-12 peer-trainers on the heat-health curriculum and evaluate capacity building; (3) pilot application of the heat- health curriculum with select community audiences (i.e., the sample group of peer-trainers will deliver the information to the public); and (4) evaluate the overall effectiveness of the heat-health TTT program in increasing heat risk education, awareness and use of available resources, and behavior change (and revise the program as needed). In addition to the project serving the San Diego County public, the overall framework and approach used for developing, implementing, and evaluating the heat-health TTT curriculum and program promises to be transferable to heat-vulnerable counties in other regions.
|
|
|
Integrating Environmental Justice into NWS Services to Reduce the Vulnerability of HUSVCs to Extreme Weather Events |
Mississippi State University |
|
Farshid Vahedifard |
May 2022 |
April 2025 |
The goal of this project is to enhance the resilience of Historically Underserved Socially Vulnerable Communities (HUSVCs) to extreme weather events by incorporating environmental justice into National Weather Service (NWS) products and services for monitoring and forecasting meteorological and climatic variables and extreme weather events such as hurricanes, tropical storms, tornadoes, flash floods, and coastal flooding. This project is motivated by the vision that fulfilling justice and equity criteria requires further and special improvements in the quality and accessibility of NWS services and products in HUSVCs. This vision is based on the well-known fact that HUSVCs are disproportionally impacted by extreme weather events, and are at greater risk for experiencing negative health and socio-economic impacts related to natural hazards and extreme events.
This project will develop a multi-attribute analysis framework that uses a suite of natural science, engineering, and socio-economic metrics to evaluate the vulnerability of different HUSVCs across the country to different types of extreme weather events. This general framework will help the NWS identify the priority zones and hotspots that are in urgent need of enhanced communication and service improvements and detect the types of improvements needed to the quality of each product and its delivery in different HUSVCs. This framework will empower the NWS to develop a least-cost path to maximize the resilience of HUSVCs to extreme weather events through improving the quality of its products based on their heterogeneous needs across the nation. Additionally, the project will benefit from a range of surveys and community engagement exercises to identify potential means of service delivery and communication improvements.
The output of the project will NOAA and NWS, federal organizations, and state agencies reduce the vulnerability of HUSVCs to extreme weather events by addressing the barriers to optimize the effectiveness of forecasts and warning at both the top (policy makers and service
providers) and bottom (communities) levels. At the top level, the project seeks to expand the knowledge and understanding of the heterogeneous needs and conditions of different HUSVCs, including their socio-economic, and cultural differences and their different levels of
access to soft (e.g., information and insurance tools) and hard (e.g., effective drainage and cooling/heating systems) infrastructure. At the bottom level, the project will identify the diverse causes of limited demand for and use of NWS products by people, such as distrust,
underestimation of risk and vulnerability, confusion, and lack of awareness about the available products and how to use them due to ineffective or unavailable communications.
|
|
|
Smoke Exposure and Underserved Wildland Fire Communities |
University of Colorado |
|
Kathryn Goldfarb |
May 2022 |
April 2025 |
A substantial proportion of the wildfire responders in the United States are migrant and immigrant workers with origins in Mexico and Central America. Migrant workers in the U.S. are historically underserved and vulnerable communities (HUSVCs) that face intersecting structural barriers to receiving and using climate services due to factors such as language access, contingent documentation statuses and institutional racism. These contract wildland firefighters, who travel seasonally around the United States to large project fires, may be some of the most exposed people in the U.S., if not the world, to pollutants such as PM2.5 and tropospheric ozone. Residents living close to environmental hazards such as polluting industries, floods and wildfire- prone areas in the wildland-urban interface have been understood as environmental justice communities and included in participatory air quality monitoring efforts for some time. But such efforts have not gained traction with mobile workers. Despite the relevance to fire responders’ work success and safety of NOAA NWS products such as National Air Quality Forecast Capability (NAQFC), the current models lack in situ observations of their performance in extreme wildfire conditions. Scientists do not know how the burden of exposure faced by wildland fire responders compares with model outputs. The atmospheric science community also has limited knowledge of how migrant fire response crews use NOAA NWS forecasting models and understand their own exposure to pollutants in tandem with the other workplace hazards they manage.
This project fill this major gap in our understanding of a) the air quality monitoring and forecasting needs of migrant and immigrant wildland fire response teams, and b) the performance of NOAA forecasting models in extreme environments. Specifically, this proposal will evaluate the performance of National Air Quality Forecast Capability (NAQFC) for PM2.5, ozone and other gases and make recommendations to correct biases in the models through a partnership with the National Weather Service NCEP Environmental Monitoring Center (EMC). Through an applied anthropology approach to systems evaluation and needs assessment we will recommend steps to create culturally-informed smoke forecast communication tools that can guide migrant wildland fire responders and related incident management teams in their work. The science community deliverables will be two peer-reviewed manuscripts published in an interdisciplinary journal that bridges social and atmospheric science, such as Elementa: Science of the Anthropocene. One of the articles will describe the new implications of this research on community-based participatory air quality monitoring methods and their capacities to evaluate large-scale modeling, and the other will describe the qualitative dimensions of how migrant wildland firefighters and adjacent communities understand and experience air quality during wildfire incidents. This project is archived in a publicly available Esri StoryMap that combines qualitative data such as voice recordings, photographs and videos of fire air quality with quantitative in situ observations of PM2.5, ozone and other gases.
|
|
|
Understanding and Improving the Full Hydrometeorological Forecasting Chain Using Multimodel Ensembles |
Penn State University |
|
Alfonso Mejia and Christopher Duffy |
May 2014 |
April 2018 |
|
Yes
|
|
Improving Warning Decision Support for Convective Storm Events in the Eastern United States |
Penn State University |
|
Paul Markowski, Yvette Richardson, & Matt Kumjian |
May 2014 |
April 2018 |
|
Yes
|
|
Collaborative Research to Advance Analysis, Forecast, and Decision Support Services for High-Impact Weather Events |
University of Utah |
|
James Steenburgh, Court Strong, and John Horel |
July 2017 |
June 2021 |
|
Yes
|
|
Advancing Analysis, Forecast, and Warning Decision Support Capabilities for High-Impact Weather Events |
University of Utah |
|
James Steenburgh and John Horel |
September 2013 |
August 2017 |
|
Yes
|
|
Towards Objective Multi-Modeling for Multi-Institutional Seasonal Water Supply Forecasting |
Portland State University |
|
Hamid Moradkhani |
May 2011 |
April 2016 |
|
Yes
|
|
Development of an Integrated Wave-Current-Wind Forecasting System for Cook Inlet: Supplementing NCEP’s Forecasting Efforts |
Texas A&M University |
|
Vijay Panchang |
May 2010 |
April 2017 |
|
Yes
|
|
Collaborative Research with the National Weather Service on Cool and Warm-Season Precipitation Forecasting over the Northeastern United States. |
SUNY Albany |
|
Lance F. Bosart and Daniel Keyser |
May 2010 |
April 2015 |
|
Yes
|
|
A New Statistical Model of Streamflow Forecast Error |
University of Connecticut and UCLA |
|
Mekonnen Gebremichael and Richard Anyah |
May 2010 |
April 2015 |
|
Yes
|
|
Cooperative Research with the National Weather Service on the Occurrence and Predictability of High-Impact Precipitation Events in the Northeastern United States |
SUNY Albany |
|
Kristen Corbosiero |
September 2013 |
August 2017 |
|
Yes
|
|
A Partnership to Develop, Conduct, and Evaluate Real-time Advanced Data Assimilation and High-Resolution Ensemble and Deterministic Forecasts for Convective-scale Hazardous Weather |
University of Oklahoma |
|
Ming Xue, Fanyou Kong, Keith Brewster, Youngsun Jung |
July 2013 |
June 2016 |
|
Yes
|
|
Improved Forecasting of Extreme Rainfall Events Associated with Tropical Cyclones |
Florida State University |
|
Henry Fuelberg, Robert Hart, and Tristan Hall |
September 2013 |
August 2017 |
|
Yes
|
|
Application of Dense Surface Observations for High-Resolution Ensemble-Based Analysis and Prediction |
University of Washington |
|
Clifford Mass and Gregory Hakim |
September 2013 |
August 2017 |
|
Yes
|
|
An Evaluation and Application of Multi-Model Ensembles in Operations for High Impact Weather over the Eastern U.S. |
SUNY Stony Brook |
|
Brian Colle and Edmund Chang |
September 2013 |
August 2017 |
|
Yes
|
|
The Use of Radar Data Assimilation in High-resolution WRF Runs for Improved Short-term QPF for Flood Forecasting, Convective Morphology Prediction, and Probability of Precipitation Guidance |
Iowa State University |
|
William Gallus and Kristie Franz |
May 2014 |
April 2017 |
|
Yes
|
|
Ensemble Subsetting within Optimized Ensembles to Improve Probabilistic Prediction of Severe Convection |
Texas Tech University |
|
Brian Ancell and Christopher Weiss |
July 2017 |
June 2021 |
|
Yes
|
|
Development of Probabilistic and Sensitivity-Based Forecast Tools to Improve High-Impact Forecasting Guidance at the NWS |
Texas Tech University |
|
Brian Ancell and Christopher Weiss |
May 2014 |
April 2018 |
|
Yes
|
|
An Ensemble-Based Approach to Forecasting Surf, Set-Up, and Surge in the Coastal Zone |
Florida Institute of Technology |
|
Steven Lazarus and Robert Weaver |
July 2014 |
June 2019 |
|
Yes
|
|
Improving Understanding and Prediction of High Impact Weather Associated with Low-Topped Severe Convection in the Southeastern U.S. |
North Carolina State University |
|
Matthew Parker, Gary Lackmann, and Lian Xie |
May 2014 |
April 2018 |
|
Yes
|
|
Major Risks, Uncertain Outcomes: Making Ensemble Forecasts Work for Multiple Audiences |
East Carolina University |
|
Dr. Burrell Montz with Rachel Hogan Carr (Nurture Nature Center) |
May 2016 |
April 2018 |
|
Yes
|
|
Adaptive, High Resolution Modeling for the Arctic Test Bed at NWS Alaska |
University of Alaska Fairbanks |
|
John Pace |
May 2016 |
April 2019 |
|
Yes
|
|
Using Re-Forecasts and Historical Observations to Assess the Potential for Severe Weather in the Extended Forecast Period |
Saint Louis University |
|
Charles Graves |
May 2016 |
April 2019 |
|
Yes
|
|
Collaborative Research on Improved Understanding and Prediction of Warm-Season (Derecho) and Cold-Season (Intense Mesoscale Banding) High Impact Events |
Florida State University |
|
Robert Hart and Henry Fuelberg |
May 2016 |
April 2020 |
|
Yes
|
|
Development of Improved Diagnostics, Numerical Models, and Situational Awareness of High-Impact Cyclones and Convective Weather Events |
SUNY Albany |
|
Kristen Corbosiero |
May 2016 |
April 2020 |
|
Yes
|
|
Prediction of Heavy Banded Snowfall: Resolution Requirements, Microphysical Sensitivity, and Hydrometeor Lofting |
North Carolina State University |
|
Gary Lackmann |
May 2016 |
April 2020 |
|
Yes
|
|
A Partnership to Develop and Evaluate Optimized Realtime Convective-Scale Ensemble Data Assimilation and Prediction Systems for Hazardous Weather |
University of Oklahoma |
|
Ming Xue, Fanyou Kong, Keith Brewster, Youngsun Jung, Nathan Snook |
May 2016 |
April 2020 |
|
Yes
|
|
Applications of HRRR Ensembles for Ensemble Hydrologic Prediction using the WRF-Hydro and SACSMA Models as Testbeds |
Iowa State University |
|
Kristie Franz and William Gallus |
July 2017 |
June 2021 |
|
Yes
|
|