5.2.3.6. Multi_Tool: Feature Relative by Lead (with lead groupings)

Calls multiple tools to produce diagnostic plots of systematic errors relative to a feature (e.g. cyclone, MCS, etc…). This example computes scores over varying forecast leads. (ADeck,GFS:BDeck,GFS:ATCF,Grib2)

Scientific Objective

By maintaining focus of each evaluation time (or evaluation time series, in this case) on a user-defined area around a cyclone, the model statistical errors associated with cyclonic physical features (moisture flux, stability, strength of upper-level PV anomaly and jet, etc.) can be related directly to the model forecasts and provide improvement guidance by accurately depicting interactions with significant weather eatures around and within the cyclone. This is in contrast to the traditional method of regional averaging cyclone observations in a fixed grid, which “smooths out” system features and limits the meaningful metrics that can be gathered. Specifically, this use case creates bins of forecast lead times as specified by the given ranges which provides additional insight directly into forecast lead time accuracy.

Datasets

Relevant information about the datasets that would be beneficial include:

• TC-Pairs/TC-Stat Forecast dataset: ADeck modified-ATCF tropical cyclone data

• Series-Analysis Forecast dataset: GFS

• TC-Pairs/TC-Stat Observation dataset: BDeck modified-ATCF tropical cyclone data

• Series-Analysis Observation dataset: GFS Analysis

• Sources of data (links, contacts, etc…)

METplus Components

This use case first runs TcPairs and ExtractTiles wrappers to generate matched tropical cyclone data and regrid them into appropriately-sized tiles along a storm track. The MET tc-stat tool is used to filter the track data, and the MET regrid-dataplane tool is used to regrid the data (GRIB1 or GRIB2 into netCDF). Next, a series analysis by lead time is performed on the results and plots (.ps and .png) are generated for all variable-level-stat combinations from the specified variables, levels, and requested statistics. The final results are aggregated into forecast hour groupings as specified by the start, end and increment in the METplus configuration file, as well as labels to identify each forecast hour grouping.

METplus Workflow

The following tools are used for each run time:

TcPairs > RegridDataPlane, TCStat > SeriesAnalysis

This example loops by forecast/lead time (with begin, end, and increment as specified in the METplus TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.conf file). The following will be run based on the availability of data corresponding to the initialization time (in this example, we only have 20141214 as our initialization time) and the requested forecast leads, resulting in the run times below.

Run times:

Init: 20141214_0Z

Forecast lead: 6

Init: 20141214_0Z

Forecast lead: 12

Init: 20141214_0Z

Forecast lead: 18

METplus Configuration

METplus first loads all of the configuration files found in parm/metplus_config, then it loads any configuration files passed to METplus via the command line with the -c option, i.e. -c parm/use_cases/model_applications/medium_range/TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.conf

#
#  CONFIGURATION
#
[config]

# Loop over each process in the process list (set in PROCESS_LIST) for all times in the time window of
# interest.
LOOP_ORDER = processes
# Configuration files
TC_PAIRS_CONFIG_FILE = {PARM_BASE}/use_cases/model_applications/medium_range/TCPairsETCConfig
TC_STAT_CONFIG_FILE = {PARM_BASE}/use_cases/model_applications/medium_range/TCStatConfig

SERIES_ANALYSIS_CONFIG_FILE = {PARM_BASE}/use_cases/model_applications/medium_range/SeriesAnalysisConfig
PROCESS_LIST = TcPairs, ExtractTiles, SeriesByLead

SERIES_ANALYSIS_REGRID_TO_GRID = FCST

## NOTE: "TOTAL" is a REQUIRED cnt statistic used by the series analysis scripts
SERIES_ANALYSIS_STAT_LIST = TOTAL, FBAR, OBAR, ME

# The init time begin and end times, increment
LOOP_BY = INIT
INIT_TIME_FMT = %Y%m%d
INIT_BEG = 20141214
INIT_END = 20141214

# This is the step-size. Increment in seconds from the begin time to the end
# time
INIT_INCREMENT = 21600 ;; set to every 6 hours=21600 seconds

# A list of times to include, in format YYYYMMDD_hh
INIT_INCLUDE =

# A list of times to exclude, in format YYYYMMDD_hh
INIT_EXCLUDE =

#
# Specify model valid time window in format YYYYMM[DD[_hh]].  Only tracks
# that fall within the valid time window will
# be used.
#
VALID_BEG =
VALID_END =

# Used by extract tiles and series analysis to define the records of
# interest to be retrieved from the grib2 file
#
BOTH_VAR1_NAME = TMP
BOTH_VAR1_LEVELS = Z2

# if true, run series_by_lead for groups of forecasts
#  must set LEAD_SEQ_[N] and LEAD_SEQ_[N]_LABELS for N > 0 groups
# if false, run series_by_lead for all forecast hours
#  must set LEAD_SEQ
SERIES_ANALYSIS_GROUP_FCSTS = True

# forecast lead sequence 1 list (0, 6, 12, 18)
LEAD_SEQ_1 = begin_end_incr(0,18,6)
# forecast lead sequence 1 label
LEAD_SEQ_1_LABEL = Day1

# forecast lead sequence 2 list (24, 30, 36, 42)
LEAD_SEQ_2 = begin_end_incr(24,42,6)
# forecast lead sequence 2 label
LEAD_SEQ_2_LABEL = Day2


# Constants used in creating the tile grid, used by extract tiles
EXTRACT_TILES_NLAT = 60
EXTRACT_TILES_NLON = 60

# Resolution of data in degrees, used by extract tiles
EXTRACT_TILES_DLAT = 0.5
EXTRACT_TILES_DLON = 0.5

# Degrees to subtract from the center lat and lon to
# calculate the lower left lat (lat_ll) and lower
# left lon (lon_ll) for a grid that is 2n X 2m,
# where n = EXTRACT_TILES_LAT_ADJ degrees and m = EXTRACT_TILES_LON_ADJ degrees.
# For this case, where n=15 and m=15, this results
# in a 30 deg X 30 deg grid.  Used by extract tiles
EXTRACT_TILES_LON_ADJ = 15
EXTRACT_TILES_LAT_ADJ = 15


##
#
# MET TC-Pairs
#
##

#
# Run MET tc_pairs by indicating the top-level directories for the A-deck
# and B-deck files. Set to 'yes' to run using top-level directories, 'no'
# if you want to run tc_pairs on files paired by the wrapper.
TC_PAIRS_READ_ALL_FILES = no

# List of models to be used (white space or comma separated) eg: DSHP, LGEM, HWRF
# If no models are listed, then process all models in the input file(s).
MODEL =

# List of storm ids of interest (space or comma separated) e.g.: AL112012, AL122012
# If no storm ids are listed, then process all storm ids in the input file(s).
TC_PAIRS_STORM_ID =

# Basins (of origin/region).  Indicate with space or comma-separated list of regions, eg. AL: for North Atlantic,
# WP: Western North Pacific, CP: Central North Pacific, SH: Southern Hemisphere, IO: North Indian Ocean, LS: Southern
# Hemisphere
TC_PAIRS_BASIN =

# Cyclone, a space or comma-separated list of cyclone numbers. If left empty, all cyclones will be used.
TC_PAIRS_CYCLONE =

# Storm name, a space or comma-separated list of storm names to evaluate.  If left empty, all storms will be used.
TC_PAIRS_STORM_NAME =

# DLAND file, the full path of the file that contains the gridded representation of the
# minimum distance from land.
TC_PAIRS_DLAND_FILE = {MET_INSTALL_DIR}/share/met/tc_data/dland_global_tenth_degree.nc

TC_PAIRS_REFORMAT_DECK = yes
TC_PAIRS_REFORMAT_TYPE = SBU

TC_PAIRS_MISSING_VAL_TO_REPLACE = -99
TC_PAIRS_MISSING_VAL = -9999

##
#
# TC-STAT filtering options, used for extract tiles and series analysis
#
##
EXTRACT_TILES_FILTER_OPTS = -basin ML
SERIES_ANALYSIS_FILTER_OPTS = -init_beg {INIT_BEG} -init_end {INIT_END}

# Run tc_stat using a config file or as command line
# if running via MET tc_stat config file, set to CONFIG.  Leave blank or
# anything other than CONFIG if running via command line.  In this case,
# tc_stat is called by one of the METplus wrappers, so don't assign anything to
# this keyword.
TC_STAT_RUN_VIA =

# For arithmetic expressions such as:
# -column 'ABS(AMSLP-BMSLP)', enclose the expression in ''.
# Notice that there are no
# whitespaces within the arithmetic expression.  White spaces are to be used to
# separate options from values
# (e.g. -job summary -by AMODEL,LEAD,AMSLP -init_hour 00 -column 'AMSLP-BMSLP').
# eg. -lookin {OUTPUT_BASE}/tc_pairs -job filter -dump_row
#             {OUTPUT_BASE}/tc_stat_filter.out -basin ML -init_hr 00
# or -lookin {OUTPUT_BASE}/tc_pairs -job summary -by AMODEL,LEAD -column AMSLP
#    -column AMAX_WIND -column 'ABS(AMAX_WIND-BMAX_WIND)'
#    -out {OUTPUT_BASE}/tc_stat/tc_stat_summary.tcst

# NOTE: Since we aren't explicitly indicating TC_STAT in our process list
# (i.e. tc_stat is being called within another wrapper), leave this
# blank/undefined.
TC_STAT_CMD_LINE_JOB =

# Specify whether only those track points common to both the ADECK and BDECK
# tracks should be written out.  This is only used when explicitly calling
# TC_STAT in the PROCESS_LIST.  This is not used in this use case, so setting
# it to either false or true has no impact.
TC_STAT_MATCH_POINTS = false

##
# !!!DO NOT MODIFY THESE!!!, they are required for all config files with
# processes that call tc_stat, but only need valid values when tc_stat
# is explicitly listed in the PROCESS_LIST.  This use case does not
# require tc_stat to be explicitly listed in the PROCESS_LIST, tc_stat
# is invoked from within one or more of the other items in the PROCESS_LIST
# (i.e. when filtering within extract tiles or within series analysis).
#
#  FILL in the following values if running multiple jobs which
#  requires a MET tc_stat config file.
#
##

# These all map to the options in the default TC-Stat config file, except these
# are pre-pended with TC_STAT to avoid clashing with any other similarly
# named options from other MET tools (eg TC_STAT_AMODEL corresponds to the
# amodel option in the default MET tc-stat config file, whereas AMODEL
# corresponds to the amodel option in the MET tc-pairs config file).

# Stratify by these columns:
TC_STAT_AMODEL =
TC_STAT_BMODEL =
TC_STAT_DESC =
TC_STAT_STORM_ID =
TC_STAT_BASIN =
TC_STAT_CYCLONE =
TC_STAT_STORM_NAME =

# Stratify by init times via a comma-separate list of init times to
# include or exclude.  Time format defined as YYYYMMDD_HH or YYYYMMDD_HHmmss
TC_STAT_INIT_BEG = 20141213
TC_STAT_INIT_END = 20141220
TC_STAT_INIT_INCLUDE =
TC_STAT_INIT_EXCLUDE =
TC_STAT_INIT_HOUR = 00
# Stratify by valid times via a comma-separate list of valid times to
# include or exclude.  Time format defined as YYYYMMDD_HH or YYYYMMDD_HHmmss
TC_STAT_VALID_BEG =
TC_STAT_VALID_END =
TC_STAT_VALID_INCLUDE =
TC_STAT_VALID_EXCLUDE =
TC_STAT_VALID_HOUR =
TC_STAT_LEAD_REQ =
TC_STAT_INIT_MASK =
TC_STAT_VALID_MASK =
# Stratify by the valid time and lead time via comma-separated list of
# times in format HH[MMSS]
TC_STAT_VALID_HOUR =
TC_STAT_LEAD =

# Stratify over the watch_warn column in the tcst file.  Setting this to
# 'ALL' will match HUWARN, HUWATCH, TSWARN, TSWATCH
TC_STAT_TRACK_WATCH_WARN =

# Stratify by applying thresholds to numeric data columns.  Specify with
# comma-separated list of column names and thresholds to be applied.
# The length of TC_STAT_COLUMN_THRESH_NAME should be the same as
# TC_STAT_COLUMN_THRESH_VAL.
TC_STAT_COLUMN_THRESH_NAME =
TC_STAT_COLUMN_THRESH_VAL =

# Stratify by a list of comma-separated columns names and values corresponding
# to non-numeric data columns of the values of interest.
TC_STAT_COLUMN_STR_NAME =
TC_STAT_COLUMN_STR_VAL =

# Stratify by applying thresholds to numeric data columns only when lead=0.
# If lead=0 and the value does not meet the threshold, discard the entire
# track.  The length of TC_STAT_INIT_THRESH_NAME must equal the length of
# TC_STAT_INIT_THRESH_VAL.
TC_STAT_INIT_THRESH_NAME =
TC_STAT_INIT_THRESH_VAL =

# Stratify by applying thresholds to numeric data columns only when lead = 0.
# If lead = 0 but the value doesn't meet the threshold, discard the entire
# track.
TC_STAT_INIT_STR_NAME =
TC_STAT_INIT_STR_VAL =

# Excludes any points where distance to land is <=0. When set to TRUE, once land
# is encountered, the remainder of the forecast track is NOT used for the
# verification, even if the track moves back over water.
TC_STAT_WATER_ONLY =

# TRUE or FALSE.  To specify whether only those track points occurring near
# landfall should be retained. Landfall is the last bmodel track point before
# the distance to land switches from water to land.
TC_STAT_LANDFALL =

# Define the landfall retention window, which is defined as the hours offset
# from the time of landfall. Format is in HH[MMSS]. Default TC_STAT_LANDFALL_BEG
# is set to -24, and TC_STAT_LANDFALL_END is set to 00
TC_STAT_LANDFALL_BEG = -24
TC_STAT_LANDFALL_END = 00

# Specify whether only those track points common to both the ADECK and BDECK
# tracks should be written out
TC_STAT_MATCH_POINTS = false

# IMPORTANT  Refer to the README_TC for details on setting up analysis
# jobs (located in {MET_INSTALL_DIR}/share/met/config

# Separate each option and value with whitespace, and each job with a whitespace.
# No whitespace within arithmetic expressions or lists of items
# (e.g. -by AMSLP,AMODEL,LEAD -column '(AMAX_WIND-BMAX_WIND)')
# Enclose your arithmetic expressions with '' and separate each job
# by whitespace:
#  -job filter -dump_row /path/to,  -job summary -line_type TCMPR  -column 'ABS(AMAX_WIND-BMAX_WIND)' -out {OUTPUT_BASE}/tc_stat/file.tcst

TC_STAT_JOBS_LIST = -job summary -line_type TCMPR -column 'ABS(AMAX_WIND-BMAX_WIND)' -dump_row {OUTPUT_BASE}/tc_stat/tc_stat_summary.tcst

#
# !!!END of DO NOT MODIFY!!!
#

# OVERWRITE OPTIONS
# Don't overwrite filter files if they already exist.
# Set to no if you do NOT want to override existing files
# Set to yes if you do want to override existing files
EXTRACT_TILES_OVERWRITE_TRACK = no

# overwrite modified track data (non-ATCF to ATCF format)
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS = yes

# overwrite tc_pairs output
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS = yes

# PLOTTING Relevant to series analysis plots.
# By default, background map is turned off. Set
# to no to turn of plotting of background map.
SERIES_ANALYSIS_BACKGROUND_MAP = no


#
#  FILENAME TEMPLATES
#
[filename_templates]
# Define the format of the filenames
FCST_EXTRACT_TILES_INPUT_TEMPLATE = gfs_4_{init?fmt=%Y%m%d}_{init?fmt=%H}00_{lead?fmt=%HHH}.grb2
OBS_EXTRACT_TILES_INPUT_TEMPLATE = gfs_4_{valid?fmt=%Y%m%d}_{valid?fmt=%H}00_000.grb2

TC_PAIRS_ADECK_TEMPLATE = {date?fmt=%Y%m}/a{basin?fmt=%s}q{date?fmt=%Y%m}*.gfso.{cyclone?fmt=%s}
TC_PAIRS_BDECK_TEMPLATE = {date?fmt=%Y%m}/b{basin?fmt=%s}q{date?fmt=%Y%m}*.gfso.{cyclone?fmt=%s}
TC_PAIRS_OUTPUT_TEMPLATE = {date?fmt=%Y%m}/{basin?fmt=%s}q{date?fmt=%Y%m%d%H}.gfso.{cyclone?fmt=%s}

#
#  DIRECTORIES
#
[dir]
# Location of your model data of interest
# Using further reduced data set than what was previously used
EXTRACT_TILES_GRID_INPUT_DIR = {INPUT_BASE}/met_test/new/reduced_model_data

# track data, set to your data source
TC_PAIRS_ADECK_INPUT_DIR = {INPUT_BASE}/met_test/new/track_data
TC_PAIRS_BDECK_INPUT_DIR = {TC_PAIRS_ADECK_INPUT_DIR}
TC_PAIRS_REFORMAT_DIR = {OUTPUT_BASE}/track_data_atcf
TC_PAIRS_OUTPUT_DIR = {OUTPUT_BASE}/tc_pairs

EXTRACT_TILES_PAIRS_INPUT_DIR = {TC_PAIRS_OUTPUT_DIR}

# The TC_STAT_INPUT_DIR, TC_STAT_OUTPUT_DIR, and TC_STAT_CONFIG_FILE
# must be present but are not needed in this use case since tc_stat is
# being invoked from within another wrapper, rather than explicitly
# listed in the PROCESS_LIST to run on its own.
# TC-Stat input data (uses output from tc-pairs)
TC_STAT_INPUT_DIR = {OUTPUT_BASE}/tc_pairs

# TC-Stat output data (creates .tcst ASCII files which can be read or used as
# input to TCMPR_Plotter_wrapper (the Python wrapper to plot_tcmpr.R) to create plots.
TC_STAT_OUTPUT_DIR = {OUTPUT_BASE}/tc_stat

# Use this setting to separate the filtered track files from
# the series analysis directory.
EXTRACT_TILES_OUTPUT_DIR = {OUTPUT_BASE}/extract_tiles
SERIES_ANALYSIS_INPUT_DIR = {EXTRACT_TILES_OUTPUT_DIR}
SERIES_ANALYSIS_FILTERED_OUTPUT_DIR = {OUTPUT_BASE}/series_lead_filtered

# Define the output directories for Series analysis by lead and init
SERIES_ANALYSIS_OUTPUT_DIR = {OUTPUT_BASE}/series_analysis_lead

#
#  REGEX PATTERNS
#
[regex_pattern]
# Regular expressions that are used in series analysis
# Forecast and Analysis tile files, and ASCII files
# created by the series analysis by init and lead time

# Indicate the prefix to the output netCDF files generated by 
# MET series_analysis
# For example, MET will produce files with the following format:
#  <prefix>__F018_gfs_4_20141215_0000_018.nc
# and the <prefix> enables the user to distinguish the forecast
# files from the analysis files
FCST_EXTRACT_TILES_PREFIX = FCST_TILE_F
OBS_EXTRACT_TILES_PREFIX = ANLY_TILE_F


#
# Indicate the regular expression of the files that were created by the
# extract tiles wrapper.  This information is used to check that input tiles
# exist before proceeding with the series analysis process.
#
# These are the forecast and analysis input files that are expected if regridding
# is to be performed by MET regrid_data_plane
FCST_SERIES_ANALYSIS_TILE_REGEX = .*FCST_TILE_F.*.grb2
OBS_SERIES_ANALYSIS_TILE_REGEX = .*ANLY_TILE_F.*.grb2

# These are the forecast and analysis input files that are expected if regridding
# is to be done via the NOAA wgrib2 tool.
FCST_SERIES_ANALYSIS_NC_TILE_REGEX = .*FCST_TILE_F.*.nc
OBS_SERIES_ANALYSIS_NC_TILE_REGEX = .*ANLY_TILE_F.*.nc

# The regular expression describing the format of the forecast and analysis files 
# that contain a list of the gridded files included in the series analysis.  This
# information is used to clean up pre-existing forecast and analysis files.
FCST_SERIES_ANALYSIS_ASCII_REGEX_LEAD = FCST_FILE_F.*
OBS_SERIES_ANALYSIS_ASCII_REGEX_LEAD = ANLY_FILE_F.*

[exe]

MET Configuration

METplus sets environment variables based on the values in the METplus configuration file. These variables are referenced in the MET configuration file. YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS! If there is a setting in the MET configuration file that is not controlled by an environment variable, you can add additional environment variables to be set only within the METplus environment using the [user_env_vars] section of the METplus configuration files. See the ‘User Defined Config’ section on the ‘System Configuration’ page of the METplus User’s Guide for more information.

////////////////////////////////////////////////////////////////////////////////
//
// Default TCPairs configuration file
//
////////////////////////////////////////////////////////////////////////////////

//
// ATCF file format reference:
//   http://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
//

//
// Models
//
model = ${MODEL};

//
// Description
//
desc = "NA";

//
// Storm identifiers
//
storm_id = ${STORM_ID};

//
// Basins
//
basin = ${BASIN};

//
// Cyclone numbers
//
cyclone = ${CYCLONE};

//
// Storm names
//
storm_name = ${STORM_NAME};

//
// Model initialization time windows to include or exclude
//
init_beg = "${INIT_BEG}";
init_end = "${INIT_END}";
init_inc = ${INIT_INCLUDE};
init_exc = ${INIT_EXCLUDE};
//
// Valid model time window
//
valid_beg = "${VALID_BEG}";
valid_end = "${VALID_END}";

//
// Model initialization hours
//
init_hour = [];

//
// Required lead time in hours
//
lead_req = [];

//
// lat/lon polylines defining masking regions
//
init_mask  = "";
valid_mask = "";

//
// Specify if the code should check for duplicate ATCF lines
//
check_dup = FALSE;

//
// Specify special processing to be performed for interpolated models.
// Set to NONE, FILL, or REPLACE.
//
interp12 = REPLACE;

//
// Specify how consensus forecasts should be defined
//
consensus = [];

//
// Forecast lag times
//
lag_time = [];

//
// CLIPER/SHIFOR baseline forecasts to be derived from the BEST
// and operational (CARQ) tracks.
//
best_technique = [ "BEST" ];
best_baseline  = [];
oper_technique = [ "CARQ" ];
oper_baseline  = [];

anly_track = BDECK;

//
// Specify if only those track points common to both the ADECK and BDECK
// tracks be written out.
//
match_points = TRUE;

//
// Specify the NetCDF output of the gen_dland tool containing a gridded
// representation of the minimum distance to land.
//
dland_file = "${DLAND_FILE}";

//
// Specify watch/warning information:
//   - Input watch/warning filename
//   - Watch/warning time offset in seconds
//
watch_warn = {
   file_name   = "MET_BASE/tc_data/wwpts_us.txt";
   time_offset = -14400;
}

//
// Indicate a version number for the contents of this configuration file.
// The value should generally not be modified.
//
//version = "V9.0";

////////////////////////////////////////////////////////////////////////////////
//
// Default TCStat configuration file
//
////////////////////////////////////////////////////////////////////////////////

//
// The parameters listed below are used to filter the TC-STAT data down to the
// desired subset of lines over which statistics are to be computed.  Only
// those lines which meet ALL of the criteria specified will be retained.
//
// The settings that are common to all jobs may be specified once at the top
// level.  If no selection is listed for a parameter, that parameter will not
// be used for filtering.  If multiple selections are listed for a parameter,
// the analyses will be performed on their union.
//

//
// Stratify by the AMODEL or BMODEL columns.
//
amodel = ${AMODEL};
bmodel = ${BMODEL};

//
// Stratify by the DESC column.
//
desc = ${DESC};

//
// Stratify by the STORM_ID column.
//
storm_id = ${STORM_ID};

//
// Stratify by the BASIN column.
// May add using the "-basin" job command option.
//
basin = ${BASIN};

//
// Stratify by the CYCLONE column.
// May add using the "-cyclone" job command option.
//
cyclone = ${CYCLONE};

//
// Stratify by the STORM_NAME column.
// May add using the "-storm_name" job command option.
//
storm_name = ${STORM_NAME};

//
// Stratify by the INIT times.
// Model initialization time windows to include or exclude
// May modify using the "-init_beg", "-init_end", "-init_inc",
// and "-init_exc" job command options.
//
init_beg = "${INIT_BEG}";
init_end = "${INIT_END}";
init_inc = ${INIT_INCLUDE};
init_exc = ${INIT_EXCLUDE};

//
// Stratify by the VALID times.
//
valid_beg = "${VALID_BEG}";
valid_end = "${VALID_END}";
valid_inc = ${VALID_INCLUDE};
valid_exc = ${VALID_EXCLUDE};

//
// Stratify by the initialization and valid hours and lead time.
//
init_hour = ${INIT_HOUR};
valid_hour = ${VALID_HOUR};
lead = ${LEAD};

//
// Select tracks which contain all required lead times.
//
lead_req = ${LEAD_REQ};

//
// Stratify by the INIT_MASK and VALID_MASK columns.
//
init_mask = ${INIT_MASK};
valid_mask = ${VALID_MASK};

//
// Stratify by the LINE_TYPE column.
//
line_type = [];

//
// Stratify by checking the watch/warning status for each track point
// common to both the ADECK and BDECK tracks.  If the watch/warning status
// of any of the track points appears in the list, retain the entire track.
//
track_watch_warn = ${TRACK_WATCH_WARN};

//
// Stratify by applying thresholds to numeric data columns.
//
column_thresh_name = ${COLUMN_THRESH_NAME};
column_thresh_val = ${COLUMN_THRESH_VAL};

//
// Stratify by performing string matching on non-numeric data columns.
//
column_str_name = ${COLUMN_STR_NAME};
column_str_val = ${COLUMN_STR_VAL};

//
// Similar to the column_thresh options above
//
init_thresh_name = ${INIT_THRESH_NAME};
init_thresh_val = ${INIT_THRESH_VAL};

//
// Similar to the column_str options above
//
init_str_name = ${INIT_STR_NAME};
init_str_val = ${INIT_STR_VAL};

//
// Stratify by the ADECK and BDECK distances to land.
//
water_only = ${WATER_ONLY};

//
// Specify whether only those track points for which rapid intensification
// or weakening of the maximum wind speed occurred in the previous time
// step should be retained.
//
rirw = {
   track  = NONE;
   adeck = {
      time   = "24";
      exact  = TRUE;
      thresh = >=30.0;
   }
   bdeck = adeck;
}

//
// Specify whether only those track points occurring near landfall should be
// retained, and define the landfall retention window in HH[MMSS] format
// around the landfall time.
//
landfall = ${LANDFALL};
landfall_beg = "${LANDFALL_BEG}";
landfall_end = "${LANDFALL_END}";


//
// Specify whether only those track points common to both the ADECK and BDECK
// tracks should be retained.  May modify using the "-match_points" job command
// option.
//
match_points = ${MATCH_POINTS};

//
// Specify whether only those cases common to all models in the dataset should
// be retained.
//
event_equal = FALSE;

//
// Specify lead times that must be present for a track to be included in the
// event equalization logic.
//
event_equal_lead = [];

//
// Apply polyline masking logic to the location of the ADECK track at the
// initialization time.
//
out_init_mask = "";

//
// Apply polyline masking logic to the location of the ADECK track at the
// valid time.
//
out_valid_mask = "";

//
// Array of TCStat analysis jobs to be performed on the filtered data
//
jobs = ${JOBS};

//
// Indicate a version number for the contents of this configuration file.
// The value should generally not be modified.
//
//version = "V9.0";

////////////////////////////////////////////////////////////////////////////////
//
// Series-Analysis configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
model = "WRF";

//
// Output description to be written
//
desc = "NA";

//
// Output observation type to be written
//
obtype = "ANALYS";

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
//
regrid = {
   to_grid    = ${REGRID_TO_GRID};
   method     = FORCE;
   width      = 1;
   vld_thresh = 0.5;
   shape      = SQUARE;
}

////////////////////////////////////////////////////////////////////////////////

censor_thresh = [];
censor_val    = [];
cat_thresh  = [ NA ];
cnt_thresh  = [ NA ];
cnt_logic   = UNION;

//
// Forecast and observation fields to be verified
//
fcst = {

   field = [
      {
        name  = "${NAME}";
        level = [ "${LEVEL}" ];
      }
   ];

}
obs = fcst;

////////////////////////////////////////////////////////////////////////////////

//
// Climatology data
//
climo_mean = {

   file_name = [];
   field     = [];

   regrid = {
      method     = NEAREST;
      width      = 1;
      vld_thresh = 0.5;
      shape      = SQUARE;
   }

   time_interp_method = DW_MEAN;
   day_interval       = 31;
   hour_interval      = 6;
}

climo_stdev = climo_mean;
climo_stdev = {
   file_name = [];
}

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
//
ci_alpha  = [ 0.05 ];

boot = {
   interval = PCTILE;
   rep_prop = 1.0;
   n_rep    = 0;
   rng      = "mt19937";
   seed     = "";
}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
//
mask = {
   grid = "";
   poly = "";
}

//
// Number of grid points to be processed concurrently.  Set smaller to use
// less memory but increase the number of passes through the data.
//
block_size = 4000;

//
// Ratio of valid matched pairs to compute statistics for a grid point
//
vld_thresh = 1.0;

////////////////////////////////////////////////////////////////////////////////

//
// Statistical output types
//
output_stats = {
   fho    = [];
   ctc    = [];
   cts    = [];
   mctc   = [];
   mcts   = [];
   cnt    = ${STAT_LIST};
   sl1l2  = [];
   sal1l2 = [];
   pct    = [];
   pstd   = [];
   pjc    = [];
   prc    = [];
}

////////////////////////////////////////////////////////////////////////////////

rank_corr_flag = FALSE;
tmp_dir        = "/tmp";
//version        = "V9.0";

////////////////////////////////////////////////////////////////////////////////

See the following files for more information about the environment variables set in these configuration files.

parm/use_cases/met_tool_wrapper/TCPairs/TCPairs.py

Running METplus

This use case can be run two ways:

1. Passing in TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.conf, then a user-specific system configuration file:

   master_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/medium_range/TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.conf
   -c /path/to/user_system.conf
   

2. Modifying the configurations in parm/metplus_config, then passing in TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.conf:

   master_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/medium_range/TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.conf
   

The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly:

• INPUT_BASE - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases

• OUTPUT_BASE - Path where METplus output will be written. This must be in a location where you have write permissions

• MET_INSTALL_DIR - Path to location where MET is installed locally

and for the [exe] section, you will need to define the location of NON-MET executables. If the executable is in the user’s path, METplus will find it from the name. If the executable is not in the path, specify the full path to the executable here (i.e. RM = /bin/rm) The following executables are required for performing series analysis use cases:

If the executables are in the path:

• RM = rm

• CUT = cut

• TR = tr

• NCAP2 = ncap2

• CONVERT = convert

• NCDUMP = ncdump

NOTE: All of these executable items must be located under the [exe] section.

If the executables are not in the path, they need to be defined:

• RM = /path/to/rm

• CUT = /path/to/cut

• TR = /path/to/tr

• NCAP2 = /path/to/ncap2

• CONVERT = /path/to/convert

• NCDUMP= /path/to/ncdump

NOTE: All of these executable items must be located under the [exe] section. Example User Configuration File:

[dir]
INPUT_BASE = /path/to/sample/input/data
OUTPUT_BASE = /path/to/output/dir
MET_INSTALL_DIR = /path/to/met-X.Y

[exe]
RM = /path/to/rm
CUT = /path/to/cut
TR = /path/to/tr
NCAP2 = /path/to/ncap2
CONVERT = /path/to/convert
NCDUMP = /path/to/ncdump

NOTE: The INPUT_BASE, OUTPUT_BASE, and MET_INSTALL_DIR must be located under the [dir] section, while the RM, CUT, TR, NCAP2, CONVERT, and NCDUMP must be located under the [exe] section.

Expected Output

A successful run will output the following both to the screen and to the logfile:

INFO: METplus has successfully finished running.

Refer to the value set for OUTPUT_BASE to find where the output data was generated. Output for this use case will be found in the following directories (relative to OUTPUT_BASE):

• Day1

• series_animate

The Day1 subdirectory will contain files that have the following format:

ANLY_FILES_Fhhh_to_FHHH

FCST_ASCII_FILES_Fhhh_to_FHHH

series_<varname>_<level>_<stat>.png

series_<varname>_<level>_<stat>.ps

series_<varname>_<level>_<stat>.nc

Where:

hhh is the starting forecast hour/lead time in hours

HHH is the ending forecast hour/lead time in hours

varname is the variable of interest, as specified in the METplus series_by_lead_all_fhrs config file

level is the level of interest, as specified in the METplus series_by_lead_all_fhrs config file

stat is the statistic of interest, as specified in the METplus series_by_lead_all_fhrs config file.

NOTE:

There is only a Day1 subdirectory, and the expected Day2 and Day3 subdirectories are absent, even though these are requested in the METplus series_by_lead_by_fhr_grouping.conf file. This is because we have been provided with only data for 20141214_00 to 20141214_18, which constitutes only one days’ worth of results.

The series_animate directory contains the animations of the series analysis in .gif format for all variable, level, and statistics combinations:

series_animate_<varname>_<level>_<stat>.gif

Keywords

Note

TCPairsToolUseCase, SeriesByLeadUseCase, TCStatToolUseCase, RegridDataPlaneToolUseCase, MediumRangeAppUseCase, SeriesAnalysisUseCase, GRIB2FileUseCase, FeatureRelativeUseCase, SBUOrgUseCase DiagnosticsUseCase

sphinx_gallery_thumbnail_path = ‘_static/medium_range-TCStat_SeriesAnalysis_fcstGFS_obsGFS_FeatureRelative_SeriesByLead.png’