Note

Click here to download the full example code

5.2.4.1. Grid-Stat and MODE: Sea Ice Validation

model_applications/cryosphere/GridStat_MODE_fcstIMS _obsNCEP_sea_ice.conf

Scientific Objective

Run Grid-Stat and MODE to compare the National Ice Center (NIC) Interactive Multisensor Snow and Ice Mapping System (IMS) and the National Centers for Environmental Prediction (NCEP) sea ice analysis. This is a validation and diagnostics use case because it is limited to a comparison between IMS analysis to NCEP analysis.

Datasets

Both IMS and NCEP sea ice analyses are observation datasets. For the purposes of MET, IMS is referred to as “forecast” and NCEP is referred to as “observation”.

  • Forecast dataset: IMS Sea Ice Concentration

    • Variable of interest: ICEC; ICEC is a binary field where “1” means a sea ice concentration of >=0.40 and “0” means a sea ice concentration of <0.40.

    • Level: Z0 (surface)

    • Dates: 20190201 - 20190228

    • Valid time: 22 UTC

    • Format: Grib2

    • Projection: 4-km Polar Stereographic

  • Observation dataset: NCEP Sea Ice Concentration

    • Variable of interest: ICEC; ICEC is the sea ice concentration with values from 0.0 - 1.0. Values >1.0 && <=1.28 indicate flagged data to be included and should be set to ==1.0 when running MET. Values >1.28 should be ignored as that indicates an invalid observation.

    • Level: Z0 (surface)

    • Dates: 20190201 - 20190228

    • Valid time: 00 UTC

    • Format: Grib2

    • Projection: 12.7-km Polar Stereographic

  • Data source: Received from Robert Grumbine at EMC. IMS data is originally from the NIC. NCEP data is originally from NCEP.

  • Location: IMS: https://www.natice.noaa.gov/ims/index.html; IMS - (https://polar.ncep.noaa.gov/seaice/Analyses.shtml)

METplus Components

This use case runs the MET GridStat and MODE tools.

METplus Workflow

The workflow processes the data by valid time, meaning that each tool will be run for each time before moving onto the next valid time. The GridStat tool is called first followed by the MODE tool. It processes analysis times from 2019-02-01 to 2019-02-05. The valid times for each analysis are different from one another (please see Datasets section for more information).

METplus Configuration

METplus first loads all of the configuration files found in parm/metplus_config. Then, it loads any configuration files passed to METplus by the command line with the -c option.

# IMS Ice Concentration (fcst) vs. NCEP Ice Concentration (obs)
# IMS and NCEP are both observation analyses. For the purpose of running MET, IMS is referred to as
# the forecast and NCEP as the obs.
# Written by Lindsay Blank, NCAR. January 2020
####################################################################################################
[config]
# Loop by analysis time
LOOP_BY = VALID

# Format of VALID_BEG and VALID_END
VALID_TIME_FMT = %Y%m%d

# L: Available dates are 20190201 - 20190228
# Start time for METplus run
VALID_BEG=20190201

# End time for METplus run
VALID_END=20190201

# Increment between METplus runs in seconds. Must be >= 60
VALID_INCREMENT=86400

# Options are times, processes
# times = run all items in the PROCESS_LIST for a single initialization
# time, then repeat until all times have been evaluated.
# processes = run each item in the PROCESS_LIST for all times
# specified, then repeat for the next item in the PROCESS_LIST.
LOOP_ORDER = times

# List of applications to run
PROCESS_LIST = GridStat, Mode

# Description of data to be processed
# used in output file path
MODEL = IMS
OBTYPE = NCEP

# Set to true if forecast data is probabilistic
FCST_IS_PROB = false


###################################################################################################
# GridStat Configurations
###################################################################################################
# List of variables to compare
# "THRESH" refers to "cat_thresh"
FCST_VAR1_NAME = ICEC
FCST_VAR1_LEVELS = Z0
FCST_VAR1_THRESH = ==1.0 

OBS_VAR1_NAME = ICEC
OBS_VAR1_LEVELS = Z0
OBS_VAR1_THRESH = >=0.40
OBS_VAR1_OPTIONS = censor_thresh = [ >1.00 && <=1.28, >1.28 ]; censor_val    = [ 1.00 , -9999 ];

GRID_STAT_NEIGHBORHOOD_WIDTH = 3, 5, 7, 9

# regridding domain for GridStat
GRID_STAT_REGRID_TO_GRID = OBS

# Location of grid_stat MET config file
GRID_STAT_CONFIG_FILE = {PARM_BASE}/met_config/GridStatConfig_wrapped

# prefix to add to GridStat output filenames
GRID_STAT_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_vs_{OBTYPE}_{CURRENT_OBS_NAME}_{CURRENT_FCST_LEVEL}

GRID_STAT_MASK_GRID = 

#GRID_STAT_MASK_POLY =

GRID_STAT_OUTPUT_FLAG_CTC = STAT
GRID_STAT_OUTPUT_FLAG_CTS = STAT
GRID_STAT_OUTPUT_FLAG_FHO = STAT
GRID_STAT_OUTPUT_FLAG_CNT = STAT
GRID_STAT_OUTPUT_FLAG_SL1L2 = STAT
GRID_STAT_OUTPUT_FLAG_PCT = STAT
GRID_STAT_OUTPUT_FLAG_PSTD = STAT
GRID_STAT_OUTPUT_FLAG_NBRCNT = STAT

GRID_STAT_NC_PAIRS_FLAG_NBRHD = TRUE

####################################################################################################
# MODE Configurations
####################################################################################################
# regridding domain for MODE
MODE_REGRID_TO_GRID = OBS

# Turn on quilting
MODE_QUILT = False

# Convolution radius list
MODE_CONV_RADIUS = 50

# Convolution threshold list
# L: IMS is a binary field where a value of "1" is equivalent to >=0.40 sea ice concentration.
FCST_MODE_CONV_THRESH = ==1.00
OBS_MODE_CONV_THRESH = >=0.40

# Location of mode MET config file
MODE_CONFIG_FILE = {PARM_BASE}/met_config/MODEConfig_wrapped

# Merge flag: options are NONE, THRESH, ENGINE, or BOTH
MODE_MERGE_FLAG = NONE

# Merge threshold list
MODE_MERGE_THRESH = >=1.25

MODE_GRID_RES = 12.7

MODE_OBS_CENSOR_THRESH = >1.00 && <=1.28, >1.28
MODE_OBS_CENSOR_VAL = 1.00 , -9999

MODE_MATCH_FLAG = NO_MERGE

MODE_MASK_POLY_FLAG = BOTH

MODE_TOTAL_INTEREST_THRESH = 0.8

# prefix to add to MODE output filenames
MODE_OUTPUT_PREFIX = {MODEL}_{CURRENT_FCST_NAME}_vs_{OBTYPE}_{CURRENT_OBS_NAME}

[dir]
# input and output data directories for each application in PROCESS_LIST
OBS_GRID_STAT_INPUT_DIR = {INPUT_BASE}/model_applications/cryosphere/sea_ice/NCEP_data
FCST_GRID_STAT_INPUT_DIR = {INPUT_BASE}/model_applications/cryosphere/sea_ice/IMS_data

OBS_MODE_INPUT_DIR = {INPUT_BASE}/model_applications/cryosphere/sea_ice/NCEP_data
FCST_MODE_INPUT_DIR = {INPUT_BASE}/model_applications/cryosphere/sea_ice/IMS_data

GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE}/model_applications/cryosphere/sea_ice/GridStat
MODE_OUTPUT_DIR = {OUTPUT_BASE}/model_applications/cryosphere/sea_ice/MODE

[filename_templates]
# format of filenames

# IMS
FCST_GRID_STAT_INPUT_TEMPLATE = imssnow96.{valid?fmt=%Y%m%d}.grb.grib2
FCST_MODE_INPUT_TEMPLATE = imssnow96.{valid?fmt=%Y%m%d}.grb.grib2

# NCEP
OBS_GRID_STAT_INPUT_TEMPLATE = seaice.t00z.north12psg.grib2.{valid?fmt=%Y%m%d}
OBS_MODE_INPUT_TEMPLATE = seaice.t00z.north12psg.grib2.{valid?fmt=%Y%m%d}

GRID_STAT_VERIFICATION_MASK_TEMPLATE = {INPUT_BASE}/model_applications/cryosphere/sea_ice/seaice_nland127.nc

MODE_VERIFICATION_MASK_TEMPLATE = {INPUT_BASE}/model_applications/cryosphere/sea_ice/seaice_nland127.nc

GRID_STAT_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d}/grid_stat
MODE_OUTPUT_TEMPLATE = {valid?fmt=%Y%m%d}/mode

MET Configuration

METplus sets environment variables based on user settings in the METplus configuration file. See How METplus controls MET config file settings for more details.

YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!

If there is a setting in the MET configuration file that is currently not supported by METplus you’d like to control, please refer to: Overriding Unsupported MET config file settings

GridStatConfig_wrapped

Note

See the GridStat MET Configuration section of the User’s Guide for more information on the environment variables used in the file below:

////////////////////////////////////////////////////////////////////////////////
//
// Grid-Stat configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
// model =
${METPLUS_MODEL}

//
// Output description to be written
// May be set separately in each "obs.field" entry
//
// desc =
${METPLUS_DESC}

//
// Output observation type to be written
//
// obtype =
${METPLUS_OBTYPE}

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
//
// regrid = {
${METPLUS_REGRID_DICT}

////////////////////////////////////////////////////////////////////////////////

censor_thresh    = [];
censor_val       = [];
cat_thresh  	 = [];
cnt_thresh  	 = [ NA ];
cnt_logic   	 = UNION;
wind_thresh 	 = [ NA ];
wind_logic  	 = UNION;
eclv_points      = 0.05;
nc_pairs_var_suffix = "";
//nc_pairs_var_name =
${METPLUS_NC_PAIRS_VAR_NAME}
rank_corr_flag   = FALSE;

//
// Forecast and observation fields to be verified
//
fcst = {
  ${METPLUS_FCST_FILE_TYPE}
  ${METPLUS_FCST_FIELD}
}
obs = {
  ${METPLUS_OBS_FILE_TYPE}
  ${METPLUS_OBS_FIELD}
}

////////////////////////////////////////////////////////////////////////////////

//
// Climatology mean data
//
//climo_mean = {
${METPLUS_CLIMO_MEAN_DICT}


//climo_stdev = {
${METPLUS_CLIMO_STDEV_DICT}

//
// May be set separately in each "obs.field" entry
//
//climo_cdf = {
${METPLUS_CLIMO_CDF_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
//
// mask = {
${METPLUS_MASK_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
//
ci_alpha  = [ 0.05 ];

boot = {
   interval = PCTILE;
   rep_prop = 1.0;
   n_rep    = 0;
   rng      = "mt19937";
   seed     = "";
}

////////////////////////////////////////////////////////////////////////////////

//
// Data smoothing methods
//
//interp = {
${METPLUS_INTERP_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Neighborhood methods
//
nbrhd = {
   field      = BOTH;
   // shape =
   ${METPLUS_NBRHD_SHAPE}
   // width =
   ${METPLUS_NBRHD_WIDTH}
   // cov_thresh =
   ${METPLUS_NBRHD_COV_THRESH}
   vld_thresh = 1.0;
}

////////////////////////////////////////////////////////////////////////////////

//
// Fourier decomposition
// May be set separately in each "obs.field" entry
//
fourier = {
   wave_1d_beg = [];
   wave_1d_end = [];
}

////////////////////////////////////////////////////////////////////////////////

//
// Gradient statistics
// May be set separately in each "obs.field" entry
//
gradient = {
   dx = [ 1 ];
   dy = [ 1 ];
}

////////////////////////////////////////////////////////////////////////////////

//
// Distance Map statistics
// May be set separately in each "obs.field" entry
//
distance_map = {
   baddeley_p        = 2;
   baddeley_max_dist = NA;
   fom_alpha         = 0.1;
   zhu_weight        = 0.5;
}

////////////////////////////////////////////////////////////////////////////////

//
// Statistical output types
//
//output_flag = {
${METPLUS_OUTPUT_FLAG_DICT}

//
// NetCDF matched pairs output file
// May be set separately in each "obs.field" entry
//
// nc_pairs_flag = {
${METPLUS_NC_PAIRS_FLAG_DICT}

////////////////////////////////////////////////////////////////////////////////

//grid_weight_flag =
${METPLUS_GRID_WEIGHT_FLAG}
tmp_dir          = "/tmp";
// output_prefix =
${METPLUS_OUTPUT_PREFIX}

////////////////////////////////////////////////////////////////////////////////

${METPLUS_MET_CONFIG_OVERRIDES}

MODEConfig_wrapped

Note

See the MODE MET Configuration section of the User’s Guide for more information on the environment variables used in the file below:

////////////////////////////////////////////////////////////////////////////////
//
// MODE configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
${METPLUS_MODEL}

//
// Output description to be written
//
${METPLUS_DESC}

//
// Output observation type to be written
//
${METPLUS_OBTYPE}

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
//
${METPLUS_REGRID_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Approximate grid resolution (km)
//
// grid_res =
${METPLUS_GRID_RES}

////////////////////////////////////////////////////////////////////////////////

//
// Run all permutations of radius and threshold
//
${METPLUS_QUILT}

//
// Forecast and observation fields to be verified
//
fcst = {
   ${METPLUS_FCST_FIELD}

   ${METPLUS_FCST_CENSOR_THRESH}
   ${METPLUS_FCST_CENSOR_VAL}
   ${METPLUS_FCST_CONV_RADIUS}
   ${METPLUS_FCST_CONV_THRESH}
   ${METPLUS_FCST_VLD_THRESH}
   ${METPLUS_FCST_FILTER_ATTR_NAME}
   ${METPLUS_FCST_FILTER_ATTR_THRESH}
   ${METPLUS_FCST_MERGE_THRESH}
   ${METPLUS_FCST_MERGE_FLAG}
}

obs = {
   ${METPLUS_OBS_FIELD}

   ${METPLUS_OBS_CENSOR_THRESH}
   ${METPLUS_OBS_CENSOR_VAL}
   ${METPLUS_OBS_CONV_RADIUS}
   ${METPLUS_OBS_CONV_THRESH}
   ${METPLUS_OBS_VLD_THRESH}
   ${METPLUS_OBS_FILTER_ATTR_NAME}
   ${METPLUS_OBS_FILTER_ATTR_THRESH}
   ${METPLUS_OBS_MERGE_THRESH}
   ${METPLUS_OBS_MERGE_FLAG}
}

////////////////////////////////////////////////////////////////////////////////

//
// Handle missing data
//
mask_missing_flag = BOTH;

//
// Match objects between the forecast and observation fields
//
//match_flag =
${METPLUS_MATCH_FLAG}

//
// Maximum centroid distance for objects to be compared
//
//max_centroid_dist =
${METPLUS_MAX_CENTROID_DIST}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
//
//mask = {
${METPLUS_MASK_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Fuzzy engine weights
//
//weight = {
${METPLUS_WEIGHT_DICT}

////////////////////////////////////////////////////////////////////////////////

//
// Fuzzy engine interest functions
//
interest_function = {

   ${METPLUS_INTEREST_FUNCTION_CENTROID_DIST}

   ${METPLUS_INTEREST_FUNCTION_BOUNDARY_DIST}

   ${METPLUS_INTEREST_FUNCTION_CONVEX_HULL_DIST}

   angle_diff = (
      (  0.0, 1.0 )
      ( 30.0, 1.0 )
      ( 90.0, 0.0 )
   );

   aspect_diff = (
      (  0.00, 1.0 )
      (  0.10, 1.0 )
      (  0.75, 0.0 )
   );

   corner   = 0.8;
   ratio_if = (
      (    0.0, 0.0 )
      ( corner, 1.0 )
      (    1.0, 1.0 )
   );

   area_ratio = ratio_if;

   int_area_ratio = (
      ( 0.00, 0.00 )
      ( 0.10, 0.50 )
      ( 0.25, 1.00 )
      ( 1.00, 1.00 )
   );

   curvature_ratio = ratio_if;

   complexity_ratio = ratio_if;

   inten_perc_ratio = ratio_if;
}

////////////////////////////////////////////////////////////////////////////////

//
// Total interest threshold for determining matches
//
//total_interest_thresh =
${METPLUS_TOTAL_INTEREST_THRESH}

//
// Interest threshold for printing output pair information
//
print_interest_thresh = 0.0;

////////////////////////////////////////////////////////////////////////////////

//
// Plotting information
//
met_data_dir = "MET_BASE";

fcst_raw_plot = {
   color_table      = "MET_BASE/colortables/met_default.ctable";
   plot_min         = 0.0;
   plot_max         = 0.0;
   colorbar_spacing = 1;
}

obs_raw_plot = {
   color_table      = "MET_BASE/colortables/met_default.ctable";
   plot_min         = 0.0;
   plot_max         = 0.0;
   colorbar_spacing = 1;
}

object_plot = {
   color_table      = "MET_BASE/colortables/mode_obj.ctable";
}

//
// Boolean for plotting on the region of valid data within the domain
//
plot_valid_flag = FALSE;

//
// Plot polyline edges using great circle arcs instead of straight lines
//
plot_gcarc_flag = FALSE;

////////////////////////////////////////////////////////////////////////////////

//
// NetCDF matched pairs, PostScript, and contingency table output files
//
ps_plot_flag    = TRUE;

//nc_pairs_flag = {
${METPLUS_NC_PAIRS_FLAG_DICT}

ct_stats_flag   = TRUE;

////////////////////////////////////////////////////////////////////////////////

shift_right = 0;   //  grid squares

////////////////////////////////////////////////////////////////////////////////

${METPLUS_OUTPUT_PREFIX}
//version        = "V10.0";

////////////////////////////////////////////////////////////////////////////////

${METPLUS_MET_CONFIG_OVERRIDES}

Running METplus

This use case can be run two ways:

  1. Passing in GridStat_MODE_fcstIMS_obsNCEP_sea_ice.conf then a user-specific system configuration file:

    run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/cryosphere/GridStat_MODE_fcstIMS_obsNCEP_sea_ice.conf -c /path/to/user_system.conf
  2. Modifying the configurations in parm/metplus_config, then passing in GridStat_MODE_fcstIMS_obsNCEP_sea_ice.conf::

    run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/cryosphere/GridStat_MODE_fcstIMS_obsNCEP_sea_ice.conf

The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly:

  • INPUT_BASE - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs).

  • OUTPUT_BASE - Path where METplus output will be written. This must be in a location where you have write permissions

  • MET_INSTALL_DIR - PAth to location where MET is installed locally

Example User Configuration File:

[dir]
INPUT_BASE = /path/to/sample/input/data
OUTPUT_BASE = /path/to/output/dir
MET_INSTALL_DIR = /path/to/met-X.Y

NOTE All of these items must be found under the [dir] section.

Expected Output

A successful run of this use case will output the following to the screen and logfile:

INFO: METplus has successfully finished running.

A successful run will have the following output files in the location defined by {OUTPUT_BASE}, which is located in the metplus_system.conf configuration file located in /path/to/METplus/parm/metplus_config. This list of files should be found for every time run through METplus. GridStat output will be in model_applications/cryosphere/sea_ice/GridStat relative to the {OUTPUT_BASE}. MODE output will be in model_applications/cryosphere/sea_ice/MODE relative to the {OUTPUT_BASE}. Using the output for 20190201 as an example:

GridStat output:

  • grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V_pairs.nc

  • grid_stat_IMS_ICEC_vs_NCEP_ICEC_Z0_000000L_20190201_220000V.stat

MODE output:

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R1_T1_cts.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R1_T1_obj.nc

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R1_T1_obj.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R1_T1.ps

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R2_T1_cts.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R2_T1_obj.nc

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R2_T1_obj.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R2_T1.ps

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R3_T1_cts.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R3_T1_obj.nc

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R3_T1_obj.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R3_T1.ps

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R4_T1_cts.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R4_T1_obj.nc

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R4_T1_obj.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R4_T1.ps

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R5_T1_cts.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R5_T1_obj.nc

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R5_T1_obj.txt

  • mode_IMS_ICEC_vs_NCEP_ICEC_000000L_20190201_220000V_000000A_R5_T1.ps

Keywords

sphinx_gallery_thumbnail_path = ‘_static/cryosphere_GridStat_MODE_fcstIMS_obsNCEP_Sea_Ice.png’

Note

GridStatToolUseCase, MODEToolUseCase, MarineAndCryoAppUseCase, ValidationUseCase, S2SAppUseCase, NOAAEMCOrgUseCase DiagnosticsUseCase

Total running time of the script: ( 0 minutes 0.000 seconds)

Gallery generated by Sphinx-Gallery