Note

Click here to download the full example code

5.2.2.6. Surrogate Severe Calculation: PCPCombine, EnsembleStat, and RegridDataPlane

model_applications/convection_allowing_model/EnsembleStat_fcstHRRR_fcstOnly_SurrogateSevere.conf

Scientific Objective

Run PCPCombine, EnsembleStat, and RegridDataPlane tools to create surrogate severe probability forecasts (SSPFs) for a given date. SSPFs are a severe weather forecasting tool and is a techniqu used by the Storm Prediction Center (SPC) as well as others. SSPFs are based on updraft helicity (UH; UH = ∫z0 to zt (ω * ζ) dz) since certain thresholds of UH have been shown as good proxies for# severe weather. SSPFs can be thought of as the perfect model forecast. They are derived as follows:

  1. Regrid the maximum UH value over the 2-5km layer at each grid point to the NCEP 211 grid (dx = ~80km).

  2. Create a binary mask of points that meet a given threshold of UH)

  3. Convert the binary mask into a probability field by applying a Gaussian filter.

For more information, please reference Sobash et al. 2011 (https://journals.ametsoc.org/doi/full/10.1175/WAF-D-10-05046.1).

Datasets

There are two dates that can be used as input data for this use case 20190518 or 20200205.

  • Input Data: HRRR data - There should 24 grib2 files. - Variable of interest: MXUPHL; the maximum updraft helicity - Level: Z2000-5000; from 2 - 5km - Format: grib2 - Projection: Lambert Conformal

  • Location: kiowa:/d1/projects/METplus/METplus_Data/model_applications/convection_allowing_models/surrogate_severe_calc/

  • Data Source: Originally received from Burkely Gallo at the Storm Prediction Center.

METplus Components

This use case runs the PCPCombine, EnsembleStat, and RegridDataPlane MET tools.

METplus Workflow

This workflow loops over the data by process, meaning that each MET tool will run over all times before moving onto the tool. PCPCombine is called first, followed by EnsembleStat, and then, finally, RegridDataPlane.

METplus Configuration

METplus first loads all of the configuration files found in parm/metplus_config. Then, it loads any configuration files passed to METplus by the command line with the -c option.

# HRRR Surrogate Severe Calculation

[config]
# time looping - options are INIT, VALID, RETRO, and REALTIME
LOOP_BY = INIT

# Format of INIT_BEG and INIT_END
INIT_TIME_FMT = %Y%m%d%H

# Start time for METplus run
INIT_BEG=2020020500

# End time for METplus run
INIT_END=2020020500

# Increment between METplus runs in seconds. Must be >= 60
INIT_INCREMENT=86400

# list of forecast leads to process
LEAD_SEQ = 36

# Options are times, processes
# times = run all items in the PROCESS_LIST for a single initialization
# time, then repeat until all times have been evaluated.
# processes = run each item in the PROCESS_LIST for all times
#   specified, then repeat for the next item in the PROCESS_LIST.
LOOP_ORDER = processes

# List of applications to run
PROCESS_LIST = PCPCombine, EnsembleStat, RegridDataPlane

# PCP_COMBINE (Step 1)
# run pcp_combine on forecast data
FCST_PCP_COMBINE_RUN = True

# method to run pcp_combine on forecast data
# Options are ADD, SUM, SUBTRACT, and DERIVE
FCST_PCP_COMBINE_METHOD = DERIVE
FCST_PCP_COMBINE_STAT_LIST = MAX

# field name of 1 hr accumulation in forecast files
FCST_PCP_COMBINE_INPUT_ACCUMS = 1
FCST_PCP_COMBINE_INPUT_NAMES = MXUPHL
FCST_PCP_COMBINE_INPUT_LEVELS = Z2000-5000
FCST_PCP_COMBINE_OUTPUT_NAME = MXUPHL_24
FCST_PCP_COMBINE_OUTPUT_ACCUM = 24
FCST_PCP_COMBINE_DERIVE_LOOKBACK = 24
FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB

#ENSEMBLE_STAT (Steps 2 and 3)
# number of expected members for ensemble. Should correspond with the
#  number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE
ENSEMBLE_STAT_N_MEMBERS = 1

# ens.ens_thresh value in the MET config file
# threshold for ratio of valid files to expected files to allow app to run
ENSEMBLE_STAT_ENS_THRESH = 1.0

# Used in the MET config file for: regrid to_grid field
ENSEMBLE_STAT_REGRID_TO_GRID = G211

ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_Surrogate_severe

ENS_VAR1_NAME = {FCST_PCP_COMBINE_OUTPUT_NAME}
ENS_VAR1_LEVELS = "(*,*)"
ENS_VAR1_THRESH = >=14.2, >=19.0, >=26.0, >=38.0, >=61.0


# REGRID_DATA_PLANE (Step 4)
# Run regrid_data_plane on forecast data
FCST_REGRID_DATA_PLANE_RUN = True

# If true, process each field individually and write a file for each
# If false, run once per run time passing in all fields specified
REGRID_DATA_PLANE_ONCE_PER_FIELD = False

# Name of input field to process
FCST_REGRID_DATA_PLANE_VAR1_INPUT_FIELD_NAME = MXUPHL_24_A1_ENS_FREQ_ge14.2
FCST_REGRID_DATA_PLANE_VAR2_INPUT_FIELD_NAME = MXUPHL_24_A1_ENS_FREQ_ge19.0
FCST_REGRID_DATA_PLANE_VAR3_INPUT_FIELD_NAME = MXUPHL_24_A1_ENS_FREQ_ge26.0
FCST_REGRID_DATA_PLANE_VAR4_INPUT_FIELD_NAME = MXUPHL_24_A1_ENS_FREQ_ge38.0
FCST_REGRID_DATA_PLANE_VAR5_INPUT_FIELD_NAME = MXUPHL_24_A1_ENS_FREQ_ge61.0

# Level of input field to process
FCST_REGRID_DATA_PLANE_VAR1_INPUT_LEVEL = "(*,*)"
FCST_REGRID_DATA_PLANE_VAR2_INPUT_LEVEL = "(*,*)"
FCST_REGRID_DATA_PLANE_VAR3_INPUT_LEVEL = "(*,*)"
FCST_REGRID_DATA_PLANE_VAR4_INPUT_LEVEL = "(*,*)"
FCST_REGRID_DATA_PLANE_VAR5_INPUT_LEVEL = "(*,*)"

# Name of output field to create
FCST_REGRID_DATA_PLANE_VAR1_OUTPUT_FIELD_NAME = MXUPHL_prob_75
FCST_REGRID_DATA_PLANE_VAR2_OUTPUT_FIELD_NAME = MXUPHL_prob_80
FCST_REGRID_DATA_PLANE_VAR3_OUTPUT_FIELD_NAME = MXUPHL_prob_85
FCST_REGRID_DATA_PLANE_VAR4_OUTPUT_FIELD_NAME = MXUPHL_prob_90
FCST_REGRID_DATA_PLANE_VAR5_OUTPUT_FIELD_NAME = MXUPHL_prob_95

# Mask to use for regridding
REGRID_DATA_PLANE_VERIF_GRID = G211

# Method to run regrid_data_plane, not setting this will default to NEAREST
REGRID_DATA_PLANE_METHOD = MAXGAUSS

# Regridding width used in regrid_data_plane, not setting this will default to 1
REGRID_DATA_PLANE_WIDTH = 1

# Set Gaussian dx value to add as command line argument - not added if unset or blank
REGRID_DATA_PLANE_GAUSSIAN_DX = 81.271

# Set Gaussian filter radius value to add as command line argument - not added if unset or blank
REGRID_DATA_PLANE_GAUSSIAN_RADIUS = 120

[dir]

# location of configuration files used by MET applications
CONFIG_DIR={PARM_BASE}/use_cases/model_applications/convection_allowing_models

# input and output data directories for each application in PROCESS_LIST
FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE}/model_applications/convection_allowing_models/surrogate_severe_calc
FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/convection_allowing_models/surrogate_severe_calc

FCST_ENSEMBLE_STAT_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR}
ENSEMBLE_STAT_OUTPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR}

FCST_REGRID_DATA_PLANE_INPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR}
FCST_REGRID_DATA_PLANE_OUTPUT_DIR = {FCST_PCP_COMBINE_OUTPUT_DIR}

[filename_templates]
# format of filenames
# Input HRRR
FCST_PCP_COMBINE_INPUT_TEMPLATE = {init?fmt=%Y%m%d}/hrrr_ncep_{init?fmt=%Y%m%d%H}f{lead?fmt=%HHH}.grib2
FCST_PCP_COMBINE_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d}/hrrr_ncep_{init?fmt=%Y%m%d%H}f{lead?fmt=%HHH}.nc

# To enemble_stat
FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = {FCST_PCP_COMBINE_OUTPUT_TEMPLATE}

# To regrid_data_plane
FCST_REGRID_DATA_PLANE_INPUT_TEMPLATE = ensemble_stat_{valid?fmt=%Y%m%d}_120000V_ens.nc
FCST_REGRID_DATA_PLANE_OUTPUT_TEMPLATE = surrogate_severe_{init?fmt=%Y%m%d}_{lead?fmt=%HHH}V_regrid.nc

MET Configuration

METplus sets environment variables based on the values in the METplus configuration file. These variables are referenced in the MET configuration file. YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS! If there is a setting in the MET configuration file that is not controlled by an environment variable, you can add additional environment variables to be set only within the METplus environment using the [user_env_vars] section of the METplus configuration files. See the ‘User Defined Config’ section on the ‘System Configuration’ page of the METplus User’s Guide for more information.

EnsembleStatConfig_Surrogate_severe

////////////////////////////////////////////////////////////////////////////////
//
// Ensemble-Stat configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
model = "${MODEL}_ens";

//
// Output description to be written
// May be set separately in each "obs.field" entry
//
desc = "NA";

//
// Output observation type to be written
//
obtype = "ANALYS";

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
//
regrid = {
   to_grid    = ${REGRID_TO_GRID};
   method     = MAX;
   width      = 27;
   vld_thresh = 0.0;
   shape      = SQUARE;
   censor_thresh = [==-9999];
   censor_val    = [0.0];
}

////////////////////////////////////////////////////////////////////////////////

//
// May be set separately in each "field" entry
//
censor_thresh = [];
censor_val    = [];
cat_thresh    = [];
nc_var_str    = "";

////////////////////////////////////////////////////////////////////////////////

//
// Ensemble product fields to be processed
//
ens = {
   ens_thresh = ${ENS_THRESH};
   vld_thresh = 1.0;

   field = [ ${ENS_FIELD}  ];
}

////////////////////////////////////////////////////////////////////////////////

//
// Neighborhood ensemble probabilities
//
nbrhd_prob = {
   width      = [ 5 ];
   shape      = CIRCLE;
   vld_thresh = 0.0;
}

//
// NMEP smoothing methods
//
nmep_smooth = {
   vld_thresh      = 0.0;
   shape           = CIRCLE;
   gaussian_dx     = 81.27;
   gaussian_radius = 120;
   type = [
      {
         method = GAUSSIAN;
         width  = 1;
      }
   ];
}


////////////////////////////////////////////////////////////////////////////////

cnt_thresh  = [ NA ];
cnt_logic   = UNION;
wind_thresh = [ NA ];
wind_logic  = UNION;

//
// Forecast and observation fields to be verified
//
fcst = {
   field = [ ${FCST_FIELD}  ];
};

//
// Ensemble bin sizes
// May be set separately in each "obs.field" entry
//
obs = {
   field = [ ${OBS_FIELD}   ];
};
obs = { wind_thresh = >2.572; };

////////////////////////////////////////////////////////////////////////////////

//
// Point observation filtering options
// May be set separately in each "obs.field" entry
//
message_type       = [ "ADPSFC" ];
sid_exc            = [];
obs_thresh         = [ NA ];
obs_quality        = [];
duplicate_flag     = UNIQUE;
obs_summary        = NONE;
obs_perc_value     = 50;
skip_const         = TRUE;

//
// Observation error options
// Set dist_type to NONE to use the observation error table instead
// May be set separately in each "obs.field" entry
//
obs_error = {
   flag             = TRUE;    // TRUE or FALSE
   dist_type        = NONE;    // Distribution type
   dist_parm        = [];      // Distribution parameters
   inst_bias_scale  = 1.0;     // Instrument bias scale adjustment
   inst_bias_offset = 0.0;     // Instrument bias offset adjustment
   min              = NA;      // Valid range of data
   max              = NA;
}

//
// Mapping of message type group name to comma-separated list of values.
//
message_type_group_map = [
   { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";               },
   { key = "ANYAIR";  val = "AIRCAR,AIRCFT";                      },
   { key = "ANYSFC";  val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; },
   { key = "ONLYSF";  val = "ADPSFC,SFCSHP";                      }
];

//
// Ensemble bin sizes
// May be set separately in each "obs.field" entry
//
ens_ssvar_bin_size = 1.0;
ens_phist_bin_size = 0.05;

////////////////////////////////////////////////////////////////////////////////

//
// Climatology data
//
climo_mean = {

   file_name = [];
   field     = [];

   regrid = {
      method     = NEAREST;
      width      = 1;
      vld_thresh = 0.5;
      shape      = SQUARE;
   }

   time_interp_method = DW_MEAN;
   day_interval       = 31;
   hour_interval      = 6;
}

climo_stdev = climo_mean;
climo_stdev = {
   file_name = [];
}

//
// May be set separately in each "obs.field" entry
//
climo_cdf = {
   cdf_bins    = 1;
   center_bins = FALSE;
   write_bins  = TRUE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Point observation time window
//
obs_window = {
   beg = ${OBS_WINDOW_BEGIN};
   end = ${OBS_WINDOW_END};
}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
//
mask = {
   grid = [];
   poly = [];
   sid  = [];
   llpnt = [];
}

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
//
ci_alpha  = [ 0.01 ]; 

////////////////////////////////////////////////////////////////////////////////

//
// Interpolation methods
//
interp = {
   field      = NONE;
   type       = [ ];
   vld_thresh = 1.0;
   shape  = SQUARE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Statistical output types
//
output_flag = {
   ecnt  = NONE;
   rhist = NONE;
   phist = NONE;
   orank = NONE;
   ssvar = NONE;
   relp  = NONE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Ensemble product output types
//
ensemble_flag = {
   latlon    = TRUE;
   mean      = FALSE;
   stdev     = FALSE;
   minus     = FALSE;
   plus      = FALSE;
   min       = FALSE;
   max       = FALSE;
   range     = FALSE;
   vld_count = FALSE;
   frequency = TRUE;
   nep       = FALSE;
   nmep      = FALSE;
   rank      = FALSE;
   weight    = FALSE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Random number generator
//
rng = {
   type = "mt19937";
   seed = "1";
}


////////////////////////////////////////////////////////////////////////////////

grid_weight_flag = NONE;
output_prefix    = "${OUTPUT_PREFIX}";
//version          = "V9.0";

////////////////////////////////////////////////////////////////////////////////
See the following file for more information about the environment variables set in this configuration file:

parm/use_cases/met_tool_wrapper/EnsembleStat/EnsembleStat.py

Running METplus

The command to run this use case is:

master_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/convection_allowing_models/EnsembleStat_fcstHRRR_fcstOnly_SurrogateSevere.conf

Expected Output

# A successful run of this use case will output the following to the screen and logfile::
#
#    INFO: METplus has successfully finished runing.
#
# A successful run will have the following output files in the location defined by {OUTPUT_BASE}, which
# is located in the metplus_system.conf configuration file located in /path/to/METplus/parm/metplus_config.
# This list of files should be found for every time run through METplus. Using the output for 20190518 as an example.
#
# **PCPCombine output**:
#
# * 20190518/hrrr_ncep_2019051800f036.nc
#
# **EnsembleStat output**:
#
# * ensemble_stat_20190519_120000V_ens.nc
#
# **RegridDataPlane output**:
#
# * surrogate_severe_20190518_036V_regrid.nc
#

Keywords

sphinx_gallery_thumbnail_path = ‘_static/convection_allowing_models-EnsembleStat_fcstHRRR_fcstOnly_SurrogateSevere.png’

Note

PCPCombineUseCase, EnsembleStatUseCase, RegridDataPlaneUseCase

Total running time of the script: ( 0 minutes 0.000 seconds)

Gallery generated by Sphinx-Gallery