Note
Click here to download the full example code
5.1.3.1. EnsembleStat: Basic Use Case¶
This use case will run the MET Ensemble-Stat tool to compare gridded ensemble forecast data to gridded AND point observation data.
Scientific Objective¶
To provide useful statistical information on the relationship between observation data (in both grid and point formats) to an ensemble forecast. These values can be used to help correct ensemble member deviations from observed values.
Datasets¶
METplus Components¶
This use case utilizes the METplus EnsembleStat wrapper to search for files that are valid at a given run time and generate a command to run the MET tool EnsembleStat if all required files are found.
METplus Workflow¶
EnsembleStat is the only tool called in this example. It processes the following run times:
METplus Configuration¶
METplus first loads all of the configuration files found in parm/metplus_config, then it loads any configuration files passed to METplus via the command line with the -c option, i.e. -c parm/use_cases/met_tool_wrapper/EnsembleStat/EnsembleStat.conf
# Ensemble Stat
# This METplus conf file runs the MET met_test unit test ensemble_stat command.
#ensemble_stat \
# 6 \
# /path/totrunk/met/data/sample_fcst/2009123112/*gep*/d01_2009123112_02400.grib \
# /path/totrunk/met/scripts/config/EnsembleStatConfig \
# -grid_obs /path/to/trunk/met/data/sample_obs/ST4/ST4.2010010112.24h \
# -point_obs /path/to/MET_test_output/met_test_scripts/ascii2nc/precip24_2010010112.nc \
# -outdir /path/to/MET_test_output/met_test_scripts/ensemble_stat \
# -v 2
[config]
## Configuration-related settings such as the process list, begin and end times, etc.
PROCESS_LIST = EnsembleStat
# Looping by times: steps through each 'task' in the PROCESS_LIST for each
# defined time, and repeats until all times have been evaluated.
LOOP_ORDER = times
# LOOP_BY: Set to INIT to loop over initialization times
LOOP_BY = INIT
# Format of INIT_BEG and INT_END
INIT_TIME_FMT = %Y%m%d%H
# Start time for METplus run
INIT_BEG=2009123112
# End time for METplus run
INIT_END=2009123112
# Increment between METplus runs in seconds. Must be >= 60
INIT_INCREMENT=3600
# List of forecast leads to process
LEAD_SEQ = 24
# Used in the MET config file for: model, output_prefix
MODEL = WRF
# Name to identify observation data in output
OBTYPE = MC_PCP
# The MET ensemble_stat logging level
# 0 quiet to 5 loud, Verbosity setting for MET ensemble_stat output, 2 is default.
# This takes precendence over the general LOG_MET_VERBOSITY set in metplus_logging.conf
#LOG_ENSEMBLE_STAT_VERBOSITY = 2
OBS_ENSEMBLE_STAT_WINDOW_BEGIN = -5400
OBS_ENSEMBLE_STAT_WINDOW_END = 5400
OBS_FILE_WINDOW_BEGIN = 0
OBS_FILE_WINDOW_END = 0
# number of expected members for ensemble. Should correspond with the
# number of items in the list for FCST_ENSEMBLE_STAT_INPUT_TEMPLATE
ENSEMBLE_STAT_N_MEMBERS = 6
# ens.ens_thresh value in the MET config file
# threshold for ratio of valid files to expected files to allow app to run
ENSEMBLE_STAT_ENS_THRESH = 1.0
# Used in the MET config file for: regrid to_grid field
ENSEMBLE_STAT_REGRID_TO_GRID = NONE
ENSEMBLE_STAT_OUTPUT_PREFIX =
ENSEMBLE_STAT_CONFIG_FILE = {CONFIG_DIR}/EnsembleStatConfig_wrapped
# ENSEMBLE_STAT_MET_OBS_ERR_TABLE is not required.
# If the variable is not defined, or the value is not set
# than the MET default is used.
#ENSEMBLE_STAT_MET_OBS_ERR_TABLE =
# Ensemble Variables and levels as specified in the ens field dictionary
# of the MET configuration file. Specify as ENS_VARn_NAME, ENS_VARn_LEVELS,
# (optional) ENS_VARn_OPTION
ENS_VAR1_NAME = APCP
ENS_VAR1_LEVELS = A24
ENS_VAR1_THRESH = >0.0, >=10.0
ENS_VAR2_NAME = REFC
ENS_VAR2_LEVELS = L0
ENS_VAR2_THRESH = >=35.0
ENS_VAR2_OPTIONS = GRIB1_ptv = 129;
ENS_VAR3_NAME = UGRD
ENS_VAR3_LEVELS = Z10
ENS_VAR3_THRESH = >=5.0
ENS_VAR4_NAME = VGRD
ENS_VAR4_LEVELS = Z10
ENS_VAR4_THRESH = >=5.0
ENS_VAR5_NAME = WIND
ENS_VAR5_LEVELS = Z10
ENS_VAR5_THRESH = >=5.0
# Forecast Variables and levels as specified in the fcst field dictionary
# of the MET configuration file. Specify as FCST_VARn_NAME, FCST_VARn_LEVELS,
# (optional) FCST_VARn_OPTION
FCST_VAR1_NAME = APCP
FCST_VAR1_LEVELS = A24
FCST_VAR1_OPTIONS = ens_ssvar_bin_size = 0.1; ens_phist_bin_size = 0.05;
# Observation Variables and levels as specified in the obs field dictionary
# of the MET configuration file. Specify as OBS_VARn_NAME, OBS_VARn_LEVELS,
# (optional) OBS_VARn_OPTION
OBS_VAR1_NAME = {FCST_VAR1_NAME}
OBS_VAR1_LEVELS = {FCST_VAR1_LEVELS}
OBS_VAR1_OPTIONS = {FCST_VAR1_OPTIONS}
[dir]
# MET config directory.
# CONFIG_DIR and the value it expands to is set as an environment variable
# and is used in the MET configuration file.
CONFIG_DIR={PARM_BASE}/met_config
# Forecast model input directory for ensemble_stat
FCST_ENSEMBLE_STAT_INPUT_DIR = {INPUT_BASE}/met_test/data/sample_fcst
# Point observation input dir for ensemble_stat
OBS_ENSEMBLE_STAT_POINT_INPUT_DIR = {INPUT_BASE}/met_test/out/ascii2nc
# Grid observation input dir for ensemble_stat
OBS_ENSEMBLE_STAT_GRID_INPUT_DIR = {INPUT_BASE}/met_test/data/sample_obs/ST4
# directory containing climatology mean input to EnsembleStat
# Not used in this example
ENSEMBLE_STAT_CLIMO_MEAN_INPUT_DIR =
# directory containing climatology mean input to EnsembleStat
# Not used in this example
ENSEMBLE_STAT_CLIMO_STDEV_INPUT_DIR =
# output directory for ensemble_stat
ENSEMBLE_STAT_OUTPUT_DIR = {OUTPUT_BASE}/ensemble
[filename_templates]
# FCST_ENSEMBLE_STAT_INPUT_TEMPLATE - comma separated list of ensemble members
# or a single line, - filename wildcard characters may be used, ? or *.
FCST_ENSEMBLE_STAT_INPUT_TEMPLATE = {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib
# ie. comma seperated list
#FCST_ENSEMBLE_STAT_INPUT_TEMPLATE =
# {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib,
# {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib,
# {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib,
# {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib,
# {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib,
# {init?fmt=%Y%m%d%H}/arw-???-gep?/d01_{init?fmt=%Y%m%d%H}_0{lead?fmt=%HH}00.grib
# Template to look for point observations.
# Example precip24_2010010112.nc
OBS_ENSEMBLE_STAT_POINT_INPUT_TEMPLATE = precip24_{valid?fmt=%Y%m%d%H}.nc
# Template to look for gridded observations.
# Example ST4.2010010112.24h
OBS_ENSEMBLE_STAT_GRID_INPUT_TEMPLATE = ST4.{valid?fmt=%Y%m%d%H}.24h
# Template to look for climatology input to EnsembleStat relative to ENSEMBLE_STAT_CLIMO_MEAN_INPUT_DIR
# Not used in this example
ENSEMBLE_STAT_CLIMO_MEAN_INPUT_TEMPLATE =
# Template to look for climatology input to EnsembleStat relative to ENSEMBLE_STAT_CLIMO_STDEV_INPUT_DIR
# Not used in this example
ENSEMBLE_STAT_CLIMO_STDEV_INPUT_TEMPLATE =
ENSEMBLE_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H%M}/ensemble_stat
MET Configuration¶
METplus sets environment variables based on the values in the METplus configuration file. These variables are referenced in the MET configuration file. YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS! If there is a setting in the MET configuration file that is not controlled by an environment variable, you can add additional environment variables to be set only within the METplus environment using the [user_env_vars] section of the METplus configuration files. See the ‘User Defined Config’ section on the ‘System Configuration’ page of the METplus User’s Guide for more information.
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble-Stat configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////
//
// Output model name to be written
//
model = "${MODEL}";
//
// Output description to be written
// May be set separately in each "obs.field" entry
//
desc = "NA";
//
// Output observation type to be written
//
obtype = "${OBTYPE}";
////////////////////////////////////////////////////////////////////////////////
//
// Verification grid
//
regrid = {
to_grid = ${REGRID_TO_GRID};
method = NEAREST;
width = 1;
vld_thresh = 0.5;
shape = SQUARE;
}
////////////////////////////////////////////////////////////////////////////////
//
// May be set separately in each "field" entry
//
censor_thresh = [];
censor_val = [];
cat_thresh = [];
nc_var_str = "";
//
// Ensemble product fields to be processed
//
ens = {
${ENS_FILE_TYPE}
ens_thresh = ${ENS_THRESH};
vld_thresh = 1.0;
field = [ ${ENS_FIELD} ];
}
////////////////////////////////////////////////////////////////////////////////
//
// Neighborhood ensemble probabilities
//
nbrhd_prob = {
width = [ 5 ];
shape = CIRCLE;
vld_thresh = 0.0;
}
//
// NMEP smoothing methods
//
nmep_smooth = {
vld_thresh = 0.0;
shape = CIRCLE;
gaussian_dx = 81.27;
gaussian_radius = 120;
type = [
{
method = GAUSSIAN;
width = 1;
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Forecast and observation fields to be verified
//
fcst = {
${FCST_FILE_TYPE}
field = [ ${FCST_FIELD} ];
}
obs = {
${OBS_FILE_TYPE}
field = [ ${OBS_FIELD} ];
}
////////////////////////////////////////////////////////////////////////////////
//
// Point observation filtering options
// May be set separately in each "obs.field" entry
//
message_type = [ "ADPSFC" ];
sid_exc = [];
obs_thresh = [ NA ];
obs_quality = [];
duplicate_flag = NONE;
obs_summary = NONE;
obs_perc_value = 50;
skip_const = FALSE;
//
// Observation error options
// Set dist_type to NONE to use the observation error table instead
// May be set separately in each "obs.field" entry
//
obs_error = {
flag = FALSE;
dist_type = NONE;
dist_parm = [];
inst_bias_scale = 1.0;
inst_bias_offset = 0.0;
min = NA; // Valid range of data
max = NA;
}
//
// Mapping of message type group name to comma-separated list of values.
//
message_type_group_map = [
{ key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; },
{ key = "ANYAIR"; val = "AIRCAR,AIRCFT"; },
{ key = "ANYSFC"; val = "ADPSFC,SFCSHP,ADPUPA,PROFLR,MSONET"; },
{ key = "ONLYSF"; val = "ADPSFC,SFCSHP"; }
];
//
// Ensemble bin sizes
// May be set separately in each "obs.field" entry
//
ens_ssvar_bin_size = 1.0;
ens_phist_bin_size = 0.05;
////////////////////////////////////////////////////////////////////////////////
//
// Climatology data
//
climo_mean = {
file_name = [ ${CLIMO_MEAN_FILE} ];
field = [];
regrid = {
method = NEAREST;
width = 1;
vld_thresh = 0.5;
shape = SQUARE;
}
time_interp_method = DW_MEAN;
day_interval = 31;
hour_interval = 6;
}
climo_stdev = climo_mean;
climo_stdev = {
file_name = [ ${CLIMO_STDEV_FILE} ];
}
//
// May be set separately in each "obs.field" entry
//
climo_cdf = {
cdf_bins = 1;
center_bins = FALSE;
write_bins = TRUE;
}
////////////////////////////////////////////////////////////////////////////////
//
// Point observation time window
//
obs_window = {
beg = ${OBS_WINDOW_BEGIN};
end = ${OBS_WINDOW_END};
}
////////////////////////////////////////////////////////////////////////////////
//
// Verification masking regions
//
mask = {
grid = [ "FULL" ];
poly = [ "MET_BASE/poly/HMT_masks/huc4_1605_poly.nc",
"MET_BASE/poly/HMT_masks/huc4_1803_poly.nc",
"MET_BASE/poly/HMT_masks/huc4_1804_poly.nc",
"MET_BASE/poly/HMT_masks/huc4_1805_poly.nc",
"MET_BASE/poly/HMT_masks/huc4_1806_poly.nc" ];
sid = [];
llpnt = [];
}
////////////////////////////////////////////////////////////////////////////////
//
// Confidence interval settings
//
ci_alpha = [ 0.05 ];
////////////////////////////////////////////////////////////////////////////////
//
// Interpolation methods
//
interp = {
field = BOTH;
vld_thresh = 1.0;
shape = SQUARE;
type = [
{
method = NEAREST;
width = 1;
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Statistical output types
//
output_flag = {
ecnt = BOTH;
rhist = BOTH;
phist = BOTH;
orank = BOTH;
ssvar = BOTH;
relp = BOTH;
}
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product output types
//
ensemble_flag = {
latlon = TRUE;
mean = TRUE;
stdev = TRUE;
minus = TRUE;
plus = TRUE;
min = TRUE;
max = TRUE;
range = TRUE;
vld_count = TRUE;
frequency = TRUE;
nep = FALSE;
nmep = FALSE;
rank = TRUE;
weight = FALSE;
}
////////////////////////////////////////////////////////////////////////////////
//
// Random number generator
//
rng = {
type = "mt19937";
seed = "1";
}
////////////////////////////////////////////////////////////////////////////////
grid_weight_flag = NONE;
output_prefix = "${OUTPUT_PREFIX}";
//version = "V9.0";
////////////////////////////////////////////////////////////////////////////////
Note the following variables are referenced in the MET configuration file.
${MODEL} - Name of forecast input. Corresponds to MODEL in the METplus configuration file.
${OBTYPE} - Name of observation input. Corresponds to OBTYPE in the METplus configuration file.
${REGRID_TO_GRID} - Grid to remap data. Corresponds to ENSEMBLE_STAT_REGRID_TO_GRID in the METplus configuration file.
${ENS_THRESH} - Threshold for ratio of valid files to expected files to allow application to run. Corresponds to ENSEMBLE_STAT_ENS_THRESH in the METplus configuration file.
${ENS_FIELD} - Formatted ensemble product fields information. Generated from ENS_VAR<n>_[NAME/LEVEL/THRESH/OPTIONS] in the METplus configuration file.
${FCST_FIELD} - Formatted forecast field information. Generated from [FCST/BOTH]_VAR<n>_[NAME/LEVEL/THRESH/OPTIONS] in the METplus configuration file.
${OBS_FIELD} - Formatted observation field information. Generated from [OBS/BOTH]_VAR<n>_[NAME/LEVEL/THRESH/OPTIONS] in the METplus configuration file.
${OBS_WINDOW_BEGIN} - Corresponds to OBS_WINDOW_BEGIN or OBS_ENSEMBLE_STAT_WINDOW_BEGIN in the METplus configuration file.
${OBS_WINDOW_END} - Corresponds to OBS_WINDOW_END or OBS_ENSEMBLE_STAT_WINDOW_END in the METplus configuration file.
${CLIMO_MEAN_FILE} - Optional path to climatology mean file. Corresponds to ENSEMBLE_STAT_CLIMO_MEAN_INPUT_[DIR/TEMPLATE] in the METplus configuration file.
${CLIMO_STDEV_FILE} - Optional path to climatology standard deviation file. Corresponds to ENSEMBLE_STAT_CLIMO_STDEV_INPUT_[DIR/TEMPLATE] in the METplus configuration file.
${OUTPUT_PREFIX} - String to prepend to the output filenames. Corresponds to ENSEMBLE_STAT_OUTPUT_PREFIX in the METplus configuration file.
Running METplus¶
It is recommended to run this use case by:
Passing in EnsembleStat.conf then a user-specific system configuration file:
master_metplus.py -c /path/to/METplus/parm/use_cases/met_tool_wrapper/EnsembleStat/EnsembleStat.conf -c /path/to/user_system.conf
The following METplus configuration variables must be set correctly to run this example.:
INPUT_BASE - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs).
OUTPUT_BASE - Path where METplus output will be written. This must be in a location where you have write permissions
MET_INSTALL_DIR - Path to location where MET is installed locally
Example User Configuration File:
[dir]
INPUT_BASE = /path/to/sample/input/data
OUTPUT_BASE = /path/to/output/dir
MET_INSTALL_DIR = /path/to/met-X.Y
NOTE: All of these items must be found under the [dir] section.
Expected Output¶
A successful run will output the following both to the screen and to the logfile:
INFO: METplus has successfully finished running.
Refer to the value set for OUTPUT_BASE to find where the output data was generated. Output for this use case will be found in ensemble/200912311200/ensemble_stat (relative to OUTPUT_BASE) and will contain the following files:
ensemble_stat_20100101_120000V.stat
ensemble_stat_20100101_120000V_ecnt.txt
ensemble_stat_20100101_120000V_rhist.txt
ensemble_stat_20100101_120000V_phist.txt
ensemble_stat_20100101_120000V_orank.txt
ensemble_stat_20100101_120000V_ssvar.txt
ensemble_stat_20100101_120000V_relp.txt
ensemble_stat_20100101_120000V_ens.nc
ensemble_stat_20100101_120000V_orank.nc
Keywords¶
Note
EnsembleStatToolUseCase, PythonEmbeddingFileUseCase, EnsembleAppUseCase, ProbabilityGenerationAppUseCase GRIBFileUseCase,
sphinx_gallery_thumbnail_path = ‘_static/met_tool_wrapper-EnsembleStat.png’
Total running time of the script: ( 0 minutes 0.000 seconds)