Note
Click here to download the full example code
5.2.9.3. TCGen: Genesis Density Function (GDF) and Track Density Function (TDF)
model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF.conf
Scientific Objective
Tropocal cyclone (TC) genesis density function (GDF) and track density function (TDF) are designed to quantitatively evaluate geographic distributions of TC activities including TC genesis frequency and subsequent TC tracks. Spatial patterns of long-term averaged GDF or TDF on the regional or global scale are particularly useful to evaluate TC forecasts against those derived from an observational best-track dataset, such as IBTrACS or ATCF B-decks, from a climate perspective. The metrics can help assess the forecast biases (under- or over-prediction) of TC formations or TC vortices around particular locations in a numerical model.
For demonstration purposes, only cyclone tracker output and b-decks data for 2016 are used.
The following settings are used in the use case, all of which are configurable in the METplus configuration file (see below).
Forecast genesis event criteria:
Minimum forecast lead: 48h
Maximum forecast lead: 120h
Maximum velocity threshold: >= 16.5 m/s
Minimum TC duration: 24h
Observed genesis event criteria:
Minimum TC duration: 24h
Maximum velocity threshold: >= 17.0 m/s
Minimum TC Category: TD
Matching settings:
Genesis matching window: +/- 24h
Early genesis matching window: -120h
Late genesis matching window: +120h
Genesis hit scoring window: +/- 24h
Early genesis hit scoring window: -120h
Late genesis hit scoring window: +120h
Matching and Scoring radius: 555 km
In addition to the above settings, normalization is performed on the metrics by the number of years included in the dataset (in this example, just one), and the total number of model forecasts valid at the time of an observed genesis event. The latter can also be thought of as the total number of chances that the model had to forecast a genesis event.
Datasets
Both forecast and observation datasets for this use case must adhere to the ATCF format.
Forecast data: GFDL Cyclone Tracker output configured for “genesis mode” for the FV3GFS model. This configuration used an experimental GFSv15 physics package, and had a horizontal grid spacing of ~25 km with 64 vertical levels.
Observation data: Global ATCF B-decks files from the National Hurricane Center (NHC) and Joint Typhoon Warning Center (JTWC)
Location: All of the input data required for this use case can be found in the met_test sample data tarball. Click here to the METplus releases page and download sample data for the appropriate release: https://github.com/dtcenter/METplus/releases This tarball should be unpacked into the directory that you will set the value of INPUT_BASE. See ‘Running METplus’ section for more information.
The MET TCGen tool requires forecast data to be provided from the GFDL cyclone tracker. More information about the GFDL cyclone tracker can be found here: https://dtcenter.org/community-code/gfdl-vortex-tracker
Archives of ATCF B-decks files can be found at these locations:
Software Versions
This use case was developed with specific versions of various software and Python packages. Any deviation from these versions may require re-configuration or adaptation to reproduce the results shown.
Names and version numbers:
python-3.6.3
cartopy-0.18.0
matplotlib-3.1.2
MET-10.0.0
METplus-4.0.0
METplotpy-1.0.0
METplus Components
This use case utilizes the MET TCGen tool to generate matched pairs of TC genesis, and then uses Python Embedding to compute the TDF and GDF metrics and create graphics for the year 2016.
METplus Workflow
The following tools are used for each run time: TCGen, Python
The TCGen tool is designed to be provided a single file pair or a directory containing a list of files, rather than loop over valid or initialization times. Thus, a single year is used in the METplus configuration file and wildcard symbols are provided to gather all the tracker and genesis input files at each input directory.
METplus Configuration
[config]
# Looping by times: steps through each 'task' in the PROCESS_LIST for each
# defined time, and repeats until all times have been evaluated.
LOOP_ORDER = times
# 'Tasks' to be run
PROCESS_LIST = TCGen, UserScript
LOOP_BY = INIT
# The init time
INIT_TIME_FMT = %Y
INIT_BEG = 2016
LOG_TC_GEN_VERBOSITY = 5
LOG_LEVEL=INFO
# optional list of strings to loop over and call the tool multiple times
# value of each item can be referenced in filename templates with {custom?fmt=%s}
TC_GEN_CUSTOM_LOOP_LIST =
# I/O Configurations
# Location of input data directory for track data
TC_GEN_TRACK_INPUT_DIR = {INPUT_BASE}/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF/obs/bdecks/{INIT_BEG}
TC_GEN_TRACK_INPUT_TEMPLATE = *.dat
# Location of input data directory for genesis data
TC_GEN_GENESIS_INPUT_DIR = {INPUT_BASE}/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF/fcst/tracker/reformat/{INIT_BEG}
TC_GEN_GENESIS_INPUT_TEMPLATE = *.fort.66
# directory to write output files generated by tc_gen
TC_GEN_OUTPUT_DIR = {OUTPUT_BASE}/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF/TCGen
TC_GEN_OUTPUT_TEMPLATE = tc_gen_{init?fmt=%Y}
# MET Configurations
TC_GEN_CONFIG_FILE = {PARM_BASE}/met_config/TCGenConfig_wrapped
# The following variables set values in the MET configuration file used by this example
# Leaving these values commented will use the value found in the default MET configuration file
# See the MET documentation for this tool for more information on the settings
TC_GEN_INIT_FREQ = 6
TC_GEN_VALID_FREQ = 6
TC_GEN_FCST_HR_WINDOW_BEGIN = 48
TC_GEN_FCST_HR_WINDOW_END = 120
TC_GEN_MIN_DURATION = 24
TC_GEN_FCST_GENESIS_VMAX_THRESH = >=16.5
TC_GEN_FCST_GENESIS_MSLP_THRESH = NA
TC_GEN_BEST_GENESIS_TECHNIQUE = BEST
TC_GEN_BEST_GENESIS_CATEGORY = TD
TC_GEN_BEST_GENESIS_VMAX_THRESH = >=17.0
TC_GEN_BEST_GENESIS_MSLP_THRESH = NA
TC_GEN_OPER_TECHNIQUE =
TC_GEN_FILTER_MODEL = GFSO
TC_GEN_GDF_FILTER_DESC = GDF
TC_GEN_EARLY_FILTER_DESC = GDF_EARLY
TC_GEN_LATE_FILTER_DESC = GDF_LATE
# TC_GEN_FILTER_<n> sets filter items in the MET configuration file
# quotation marks within quotation marks must be preceeded with \
TC_GEN_FILTER_1 = model = "{TC_GEN_FILTER_MODEL}"; desc = "{TC_GEN_GDF_FILTER_DESC}"; dev_hit_window = { beg = -24; end = 24; }; dev_hit_radius = 555; genesis_match_window = { beg = -24; end = 24;};
TC_GEN_FILTER_2 = model = "{TC_GEN_FILTER_MODEL}"; desc = "{TC_GEN_EARLY_FILTER_DESC}"; dev_hit_window = { beg = -120; end = 0; }; dev_hit_radius = 555; genesis_match_window = { beg = -120; end = 0;};
TC_GEN_FILTER_3 = model = "{TC_GEN_FILTER_MODEL}"; desc = "{TC_GEN_LATE_FILTER_DESC}"; dev_hit_window = { beg = 0; end = 120; }; dev_hit_radius = 555; genesis_match_window = { beg = 0; end = 120;};
TC_GEN_DESC = ALL
MODEL =
TC_GEN_STORM_ID =
TC_GEN_STORM_NAME =
TC_GEN_INIT_BEG =
TC_GEN_INIT_END =
TC_GEN_INIT_INC =
TC_GEN_INIT_EXC =
TC_GEN_VALID_BEG =
TC_GEN_VALID_END =
TC_GEN_INIT_HOUR =
# sets METPLUS_LEAD in the wrapped MET config file
LEAD_SEQ =
TC_GEN_VX_MASK =
TC_GEN_BASIN_MASK =
TC_GEN_DLAND_THRESH = NA
TC_GEN_GENESIS_MATCH_RADIUS = 555
TC_GEN_GENESIS_MATCH_POINT_TO_TRACK = False
TC_GEN_GENESIS_MATCH_WINDOW_BEG = 0
TC_GEN_GENESIS_MATCH_WINDOW_END = 0
TC_GEN_OPS_HIT_WINDOW_BEG = 0
TC_GEN_OPS_HIT_WINDOW_END = 48
TC_GEN_DEV_HIT_RADIUS = 500
TC_GEN_DEV_HIT_WINDOW_BEGIN = -24
TC_GEN_DEV_HIT_WINDOW_END = 24
TC_GEN_OPS_HIT_TDIFF = 48
TC_GEN_DISCARD_INIT_POST_GENESIS_FLAG = True
TC_GEN_DEV_METHOD_FLAG = True
TC_GEN_OPS_METHOD_FLAG = False
TC_GEN_CI_ALPHA = 0.05
TC_GEN_OUTPUT_FLAG_FHO = NONE
TC_GEN_OUTPUT_FLAG_CTC = BOTH
TC_GEN_OUTPUT_FLAG_CTS = BOTH
TC_GEN_OUTPUT_FLAG_GENMPR = BOTH
TC_GEN_NC_PAIRS_FLAG_LATLON = TRUE
TC_GEN_NC_PAIRS_FLAG_FCST_GENESIS = TRUE
TC_GEN_NC_PAIRS_FLAG_FCST_TRACKS = TRUE
TC_GEN_NC_PAIRS_FLAG_FCST_FY_OY = TRUE
TC_GEN_NC_PAIRS_FLAG_FCST_FY_ON = TRUE
TC_GEN_NC_PAIRS_FLAG_BEST_GENESIS = TRUE
TC_GEN_NC_PAIRS_FLAG_BEST_TRACKS = TRUE
TC_GEN_NC_PAIRS_FLAG_BEST_FY_OY = TRUE
TC_GEN_NC_PAIRS_FLAG_BEST_FN_OY = TRUE
TC_GEN_VALID_MINUS_GENESIS_DIFF_THRESH = >0
TC_GEN_BEST_UNIQUE_FLAG = TRUE
TC_GEN_DLAND_FILE = MET_BASE/tc_data/dland_global_tenth_degree.nc
TC_GEN_BASIN_FILE = MET_BASE/tc_data/basin_global_tenth_degree.nc
TC_GEN_NC_PAIRS_GRID = G003
USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE_FOR_EACH
USER_SCRIPT_INPUT_TEMPLATE = {TC_GEN_OUTPUT_DIR}/tc_gen_{init?fmt=%Y}_pairs.nc
USER_SCRIPT_COMMAND = {PARM_BASE}/use_cases/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF/UserScript_fcstGFSO_obsBDECKS_GDF_TDF.py {USER_SCRIPT_INPUT_TEMPLATE}
[user_env_vars]
TCGEN_INIT_FREQ = {TC_GEN_INIT_FREQ}
TCGEN_MIN_LEAD = {TC_GEN_FCST_HR_WINDOW_BEGIN}
TCGEN_MAX_LEAD = {TC_GEN_FCST_HR_WINDOW_END}
GDF_LAT_HALF_DELTA = 5.0
GDF_LON_HALF_DELTA = 5.0
GDF_NORM_YEARS = 1.0
GDF_PLOT_OUTDIR = {OUTPUT_BASE}/images
GDF_MODEL_STRING = {TC_GEN_FILTER_MODEL}
GDF_OBS_STRING = BEST
GDF_DESC_STRING = {TC_GEN_GDF_FILTER_DESC}
GDF_EARLY_STRING = {TC_GEN_EARLY_FILTER_DESC}
GDF_LATE_STRING = {TC_GEN_LATE_FILTER_DESC}
MET Configuration
METplus sets environment variables based on user settings in the METplus configuration file. See How METplus controls MET config file settings for more details.
YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!
If there is a setting in the MET configuration file that is currently not supported by METplus you’d like to control, please refer to: Overriding Unsupported MET config file settings
TCGenConfig_wrapped
Note
See the TCGen MET Configuration section of the User’s Guide for more information on the environment variables used in the file below:
////////////////////////////////////////////////////////////////////////////////
//
// TC-Gen configuration file.
//
// For additional information, see the MET_BASE/config/README_TC file.
//
////////////////////////////////////////////////////////////////////////////////
//
// ATCF file format reference:
// http://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
//
////////////////////////////////////////////////////////////////////////////////
//
// Genesis event definition criteria.
//
////////////////////////////////////////////////////////////////////////////////
//
// Model initialization frequency in hours, starting at 0.
//
// init_freq =
${METPLUS_INIT_FREQ}
//
// Valid hour frequency to be analyzed in hours, starting at 0
//
// valid_freq =
${METPLUS_VALID_FREQ}
//
// Forecast hours to be searched for genesis events
//
// fcst_hr_window =
${METPLUS_FCST_HR_WINDOW_DICT}
//
// Minimum track duration for genesis event in hours.
//
// min_duration =
${METPLUS_MIN_DURATION}
//
// Forecast genesis event criteria. Defined as tracks reaching the specified
// intensity category, maximum wind speed threshold, and minimum sea-level
// pressure threshold. The forecast genesis time is the valid time of the first
// track point where all of these criteria are met.
//
// fcst_genesis =
${METPLUS_FCST_GENESIS_DICT}
//
// BEST track genesis event criteria. Defined as tracks reaching the specified
// intensity category, maximum wind speed threshold, and minimum sea-level
// pressure threshold. The BEST track genesis time is the valid time of the
// first track point where all of these criteria are met.
//
// best_genesis =
${METPLUS_BEST_GENESIS_DICT}
//
// Operational track technique name
//
// oper_technique =
${METPLUS_OPER_TECHNIQUE}
////////////////////////////////////////////////////////////////////////////////
//
// Track filtering options
// May be specified separately in each filter array entry.
//
////////////////////////////////////////////////////////////////////////////////
//
// Array of dictionaries containing the track filtering options
// If empty, a single filter is defined using the top-level settings.
//
// filter =
${METPLUS_FILTER}
//
// Description written to output DESC column
//
// desc =
${METPLUS_DESC}
//
// Forecast ATCF ID's
// If empty, all ATCF ID's found will be processed.
// Statistics will be generated separately for each ATCF ID.
//
// model =
${METPLUS_MODEL}
//
// BEST and operational track storm identifiers
//
// storm_id =
${METPLUS_STORM_ID}
//
// BEST and operational track storm names
//
// storm_name =
${METPLUS_STORM_NAME}
//
// Forecast and operational initialization times to include or exclude
//
// init_beg =
${METPLUS_INIT_BEG}
// init_end =
${METPLUS_INIT_END}
// init_inc =
${METPLUS_INIT_INC}
// init_exc =
${METPLUS_INIT_EXC}
//
// Forecast, BEST, and operational valid time window
//
// valid_beg =
${METPLUS_VALID_BEG}
// valid_end =
${METPLUS_VALID_END}
//
// Forecast and operational initialization hours
//
// init_hour =
${METPLUS_INIT_HOUR}
//
// Forecast and operational lead times in hours
//
// lead =
${METPLUS_LEAD}
//
// Spatial masking region (path to gridded data file or polyline file)
//
// vx_mask =
${METPLUS_VX_MASK}
//
// Spatial masking of hurricane basin names from the basin_file
//
// basin_mask =
${METPLUS_BASIN_MASK}
//
// Distance to land threshold
//
//dland_thresh =
${METPLUS_DLAND_THRESH}
////////////////////////////////////////////////////////////////////////////////
//
// Matching and scoring options
// May be specified separately in each filter array entry.
//
////////////////////////////////////////////////////////////////////////////////
//
// Genesis matching logic. Compare the forecast genesis point to all points in
// the Best track (TRUE) or the single Best track genesis point (FALSE).
//
//genesis_match_point_to_track =
${METPLUS_GENESIS_MATCH_POINT_TO_TRACK}
//
// Radius in km to search for a matching genesis event
//
// genesis_match_radius =
${METPLUS_GENESIS_MATCH_RADIUS}
//
// Time window in hours, relative to the model genesis time, to search for a
// matching Best track point
//
//genesis_match_window = {
${METPLUS_GENESIS_MATCH_WINDOW_DICT}
//
// Radius in km for a development scoring method hit
//
// dev_hit_radius =
${METPLUS_DEV_HIT_RADIUS}
//
// Time window in hours for a development scoring method hit
//
// dev_hit_window =
${METPLUS_DEV_HIT_WINDOW_DICT}
// Time window in hours for the Best track genesis minus model initialization
// time difference for an operational scoring method hit
//
//ops_hit_window = {
${METPLUS_OPS_HIT_WINDOW_DICT}
//
// Discard genesis forecasts for initializations at or after the matching
// BEST track genesis time
//
// discard_init_post_genesis_flag =
${METPLUS_DISCARD_INIT_POST_GENESIS_FLAG}
//
// Scoring methods to be applied
//
//dev_method_flag =
${METPLUS_DEV_METHOD_FLAG}
// ops_method_flag =
${METPLUS_OPS_METHOD_FLAG}
////////////////////////////////////////////////////////////////////////////////
//
// Output options
// May be specified separately in each filter array entry.
//
////////////////////////////////////////////////////////////////////////////////
//
// Confidence interval alpha value
//
// ci_alpha =
${METPLUS_CI_ALPHA}
//
// Statistical output types
//
// output_flag =
${METPLUS_OUTPUT_FLAG_DICT}
//
// NetCDF genesis pair counts
//
// nc_pairs_flag =
${METPLUS_NC_PAIRS_FLAG_DICT}
//
// Specify which track points should be counted by thresholding the track point
// valid time minus genesis time difference.
//
// valid_minus_genesis_diff_thresh =
${METPLUS_VALID_MINUS_GENESIS_DIFF_THRESH}
//
// Count unique BEST track genesis event locations (TRUE) versus counting the
// location for all pairs (FALSE).
//
// best_unique_flag =
${METPLUS_BEST_UNIQUE_FLAG}
////////////////////////////////////////////////////////////////////////////////
//
// Global settings
// May only be specified once.
//
////////////////////////////////////////////////////////////////////////////////
//
// Specify the NetCDF output of the gen_dland tool containing a gridded
// representation of the minimum distance to land.
//
// dland_file =
${METPLUS_DLAND_FILE}
//
// Specify the NetCDF file containing a gridded representation of the
// global basins.
//
// basin_file =
${METPLUS_BASIN_FILE}
//
// NetCDF genesis pairs grid
//
// nc_pairs_grid =
${METPLUS_NC_PAIRS_GRID}
//
// Indicate a version number for the contents of this configuration file.
// The value should generally not be modified.
//
//version = "V10.0.0";
${METPLUS_MET_CONFIG_OVERRIDES}
Python Embedding
This use case uses a Python embedding script to create output graphics
parm/use_cases/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF/UserScript_fcstGFSO_obsBDECKS_GDF_TDF.py
#!/usr/bin/env python3
"""UserScript to compute density variables for the METplus S2S TDF/GDF use case
This script is used to read in netCDF output from the MET TCGen tool and compute
various density variables related to the Genesis Density Function (GDF) and
Track Density Function (TDF) metrics for subseasonal-to-seasonal applications.
Contains the following functions:
* as_density()
Author: Daniel R. Adriaansen
Date: 24 March 2021
"""
import xarray as xr
import os
import sys
import datetime
import multiprocessing
# Import METplotpy
from metplotpy.contributed.tc_s2s_panel import plot_tc_s2s_panel as tc_s2s_panel
# Enviromnment variables for use case
GDF_INPUT_FILENAME = sys.argv[1]
GDF_LAT_HALF_DELTA = float(str(os.environ.get('GDF_LAT_HALF_DELTA',5.0)))
GDF_LON_HALF_DELTA = float(str(os.environ.get('GDF_LON_HALF_DELTA',5.0)))
GDF_MODEL_STRING = str(os.environ.get('GDF_MODEL_STRING','TESTMODEL'))
GDF_OBS_STRING = str(os.environ.get('GDF_OBS_STRING','TESTOBS'))
GDF_DESC_STRING = str(os.environ.get('GDF_DESC_STRING','GDF'))
GDF_EARLY_STRING = str(os.environ.get('GDF_EARLY_STRING','GDF_EARLY'))
GDF_LATE_STRING = str(os.environ.get('GDF_LATE_STRING','GDF_LATE'))
GDF_NORM_YEARS = float(str(os.environ.get('GDF_NORM_YEARS',1.0)))
# Compute the total number of model forecasts that could have forecasted a hypothetical genesis event
# within the user defined lead window
lead_step = int(str(os.environ.get('TCGEN_INIT_FREQ')))
shortest_lead = int(str(os.environ.get('TCGEN_MIN_LEAD')))
longest_lead = int(str(os.environ.get('TCGEN_MAX_LEAD')))
num_forecasts = float(len([shortest_lead + x for x in range(0,longest_lead,lead_step) if shortest_lead+x <= longest_lead]))
# Local variables
DEBUG = False
# Create some netCDF varname strings to use when referencing the data
fcstgenvarname = f"{GDF_DESC_STRING}_{GDF_MODEL_STRING}_GENESIS"
fcsthitvarname = f"{GDF_DESC_STRING}_{GDF_MODEL_STRING}_DEV_FY_OY"
fcstfalmvarname = f"{GDF_DESC_STRING}_{GDF_MODEL_STRING}_DEV_FY_ON"
obsmissvarname = f"{GDF_DESC_STRING}_{GDF_MODEL_STRING}_BEST_DEV_FN_OY"
obsgenvarname = f"{GDF_DESC_STRING}_BEST_GENESIS"
fcsttrackvarname = f"{GDF_DESC_STRING}_{GDF_MODEL_STRING}_TRACKS"
obstrackvarname = f"{GDF_DESC_STRING}_BEST_TRACKS"
fcstearlygenvarname = f"{GDF_EARLY_STRING}_{GDF_MODEL_STRING}_GENESIS"
fcstearlyhitvarname = f"{GDF_EARLY_STRING}_{GDF_MODEL_STRING}_DEV_FY_OY"
fcstlategenvarname = f"{GDF_LATE_STRING}_{GDF_MODEL_STRING}_GENESIS"
fcstlatehitvarname = f"{GDF_LATE_STRING}_{GDF_MODEL_STRING}_DEV_FY_OY"
if DEBUG:
print("\nUSING VARIABLE NAMES:")
print(f"Forecast genesis events varname: {fcstgenvarname}")
print(f"Hits (fy_oy) varname:: {fcsthitvarname}")
print(f"False alarms (fy_on) varname: {fcstfalmvarname}")
print(f"Miss (fn_oy) varname: {obsmissvarname}")
print(f"Observed genesis events varname: {obsgenvarname}")
print(f"Forecast track points varname: {fcsttrackvarname}")
print(f"Observed track points varname: {obstrackvarname}")
print(f"Early genesis event varname: {fcstearlygenvarname}")
print(f"Early genesis hit varname: {fcstearlyhitvarname}")
print(f"Late genesis event varname: {fcstlategenvarname}")
print(f"Late genesis hit varname: {fcstlatehitvarname}")
# Open the TCGen output file
tcgendata = xr.open_dataset(GDF_INPUT_FILENAME)
# Create 1D data to find locations of events
tcgendata1d = tcgendata.stack(adim=('lat','lon'))
# Get the lat/lon of EARLY FCST genesis events
earl_lat = tcgendata1d[fcstearlygenvarname].where(tcgendata1d[fcstearlygenvarname]>0.0,drop=True)['lat'].values
earl_lon = tcgendata1d[fcstearlygenvarname].where(tcgendata1d[fcstearlygenvarname]>0.0,drop=True)['lon'].values
# Get the lat/lon of LATE FCST genesis events
late_lat = tcgendata1d[fcstlategenvarname].where(tcgendata1d[fcstlategenvarname]>0.0,drop=True)['lat'].values
late_lon = tcgendata1d[fcstlategenvarname].where(tcgendata1d[fcstlategenvarname]>0.0,drop=True)['lon'].values
# Get the lat/lon of FCST genesis events
fcst_lat = tcgendata1d[fcstgenvarname].where(tcgendata1d[fcstgenvarname]>0.0,drop=True)['lat'].values
fcst_lon = tcgendata1d[fcstgenvarname].where(tcgendata1d[fcstgenvarname]>0.0,drop=True)['lon'].values
# Get the lat/lon of the OBS (BEST) genesis events
obs_lat = tcgendata1d[obsgenvarname].where(tcgendata1d[obsgenvarname]>0.0,drop=True)['lat'].values
obs_lon = tcgendata1d[obsgenvarname].where(tcgendata1d[obsgenvarname]>0.0,drop=True)['lon'].values
# Get the lat/lon of the FCST track points (based on genesis)
ftrk_lat = tcgendata1d[fcsttrackvarname].where(tcgendata1d[fcsttrackvarname]>0.0,drop=True)['lat'].values
ftrk_lon = tcgendata1d[fcsttrackvarname].where(tcgendata1d[fcsttrackvarname]>0.0,drop=True)['lon'].values
# Get the lat/lon of the OBS (BEST) track points (based on genesis)
otrk_lat = tcgendata1d[obstrackvarname].where(tcgendata1d[obstrackvarname]>0.0,drop=True)['lat'].values
otrk_lon = tcgendata1d[obstrackvarname].where(tcgendata1d[obstrackvarname]>0.0,drop=True)['lon'].values
# Function to take gridded counts of data and create a density plot given a lat/lon region defined by GDF_LAT/LON_HALF_DELTA around these counts.
# Input are the individual lats/lons of each location where there are any events, as well as the actual gridded variable of counts
def as_density(elats,elons,grid_var,fcst):
"""Computes the density of an event based on a gridded count variable and the lat/lon of each event
Parameters
----------
elats: list of floating point latitude values of individual events
elons: list of floating point longitude values of individual events
grid_var: Xarray DataArray containing the count at each event location
fcst: a boolean to denote whether the grid_var is a forecast (True) or observation (False) variable
Returns
-------
Xarray DataArray object the with the same likeness as grid_var
"""
# Create a DataArray that looks like the input grid_var
dens_var = xr.zeros_like(grid_var,dtype='float32')
# Try to re-write the while loop as a for loop
#for clat,clon in tuple(zip(elats,elons)):
llcnt = 0
while llcnt < len(elats):
clat = elats[llcnt]
clon = elons[llcnt]
# Latitude and longitude of subdomain around the point lat/lon
glat = grid_var.lat[(grid_var.lat>=clat-GDF_LAT_HALF_DELTA) & (grid_var.lat<=clat+GDF_LAT_HALF_DELTA)]
glon = grid_var.lon[(grid_var.lon>=clon-GDF_LON_HALF_DELTA) & (grid_var.lon<=clon+GDF_LON_HALF_DELTA)]
# Get the number of events at the current point lat/lon
nevent = grid_var.sel(lat=clat,lon=clon).values
# Increment the dens_var where we want
dens_var.loc[dict(lat=glat,lon=glon)] += nevent
llcnt += 1
# Return the dens_var
if fcst:
return(dens_var/(GDF_NORM_YEARS*num_forecasts))
else:
return(dens_var/(GDF_NORM_YEARS))
# Create some lists of function arguments to as_density() to run in parallel
varlist = [fcstgenvarname,fcsthitvarname,fcstfalmvarname,fcsttrackvarname,obsgenvarname,obsmissvarname,obstrackvarname,fcstlatehitvarname,fcstearlyhitvarname]
varlats = [fcst_lat,fcst_lat,fcst_lat,ftrk_lat,obs_lat,obs_lat,otrk_lat,late_lat,earl_lat]
varlons = [fcst_lon,fcst_lon,fcst_lon,ftrk_lon,obs_lon,obs_lon,otrk_lon,late_lon,earl_lon]
fcstobs = [True,True,True,True,False,True,False,True,True]
denvars = ['FCST_DENS','FYOY_DENS','FYON_DENS','FTRK_DENS','OBS_DENS','FNOY_DENS','OTRK_DENS','LHIT_DENS','EHIT_DENS']
# Use multiprocessing to run in parallel
# Results is a list of DataArray objects
mp = multiprocessing.Pool(max(multiprocessing.cpu_count()-2, 1))
results = mp.starmap(as_density,[(x,y,tcgendata[z],f) for x,y,z,f in tuple(zip(varlats,varlons,varlist,fcstobs))])
# Unpack the results
for r,n in tuple(zip(results,denvars)):
tcgendata[n] = r
if DEBUG:
print("\nOBS_DENS")
print(tcgendata['OBS_DENS'].min().values)
print(tcgendata['OBS_DENS'].max().values)
print("\nFCST_DENS")
print(tcgendata['FCST_DENS'].min().values)
print(tcgendata['FCST_DENS'].max().values)
print("\nFYOY_DENS")
print(tcgendata['FYOY_DENS'].min().values)
print(tcgendata['FYOY_DENS'].max().values)
print("\nFYON_DENS")
print(tcgendata['FYON_DENS'].min().values)
print(tcgendata['FYON_DENS'].max().values)
print("\nFNOY_DENS")
print(tcgendata['FNOY_DENS'].min().values)
print(tcgendata['FNOY_DENS'].max().values)
print("\nFTRK_DENS")
print(tcgendata['FTRK_DENS'].min().values)
print(tcgendata['FTRK_DENS'].max().values)
print("\nOTRK_DENS")
print(tcgendata['OTRK_DENS'].min().values)
print(tcgendata['OTRK_DENS'].max().values)
print("\nEHIT_DENS")
print(tcgendata['EHIT_DENS'].min().values)
print(tcgendata['EHIT_DENS'].max().values)
print("\nLHIT_DENS")
print(tcgendata['LHIT_DENS'].min().values)
print(tcgendata['LHIT_DENS'].max().values)
# Call plotting for GDF. tc_s2s_panel.plot_gdf() requires just the Xarray Dataset object
# Panel order for GDF is:
# 1. Total BEST (observed) genesis density
# 2. Total MODEL (forecast) genesis density
# 3. Difference 2-1
gdf_varlist = ['OBS_DENS','FCST_DENS']
tc_s2s_panel.plot_gdf(tcgendata[gdf_varlist],os.environ.get('GDF_PLOT_OUTDIR'))
# Call plotting for TDF. tc_s2s_panel.plot_tdf() requires just the Xarray Dataset object
# Panel order for TDF is:
# 1. Total BEST (observed) track points
# 2. Total FCST (hour 24-120) track points
# 3. FCST-BEST
tdf_varlist = ['FTRK_DENS','OTRK_DENS']
tc_s2s_panel.plot_tdf(tcgendata[tdf_varlist],os.environ.get('GDF_PLOT_OUTDIR'))
# Call plotting for GDF category. tc_s2s_panel.plot_gdf_cat() requires just the Xarray Dataset object
# Panel order for GDF category is:
# 1. Total HITS density
# 2. Total EARLY HITS density
# 3. Total LATE HITS density
# 4. Total FALSE_ALARMS density
gdf_cat_varlist = ['FYOY_DENS','EHIT_DENS','LHIT_DENS','FYON_DENS']
tc_s2s_panel.plot_gdf_cat(tcgendata[gdf_cat_varlist],os.environ.get('GDF_PLOT_OUTDIR'))
# Call plotting for GDF UFS. tc_s2s_panel.plot_gdf_ufs() requires just the Xarray Dataset object
# Panel order for GDF UFS is:
# 1. Total BEST (observed) genesis density (scatter is BEST genesis locations)
# 2. Total HITS density (scatter is BEST genesis locations)
# 3. Total FALSE_ALARMS density (scatter is FCST genesis locations, but only false?)
# 4. Total HITS+FALSE_ALARMS density (scatter is FCST genesis locations, but only hits+false?)
gdf_ufs_varlist = ['OBS_DENS','FYOY_DENS','FYON_DENS']
tc_s2s_panel.plot_gdf_ufs(tcgendata[gdf_ufs_varlist],os.environ.get('GDF_PLOT_OUTDIR'))
Running METplus
This use case can be run two ways:
Passing in TCGen_fcstGFSO_obsBDECKS_GDF_TDF.conf then a user-specific system configuration file:
run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF.conf -c /path/to/user_system.conf
Modifying the configurations in parm/metplus_config, then passing in TCGen_fcstGFSO_obsBDECKS_GDF_TDF.conf:
run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF.conf
The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly:
INPUT_BASE - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases
OUTPUT_BASE - Path where METplus output will be written. This must be in a location where you have write permissions
MET_INSTALL_DIR - Path to location where MET is installed locally
Example User Configuration File:
[config]
INPUT_BASE = /path/to/sample/input/data
OUTPUT_BASE = /path/to/output/dir
MET_INSTALL_DIR = /path/to/met-X.Y
Expected Output
A successful run will output the following both to the screen and to the logfile:
INFO: METplus has successfully finished running.
Refer to the value set for OUTPUT_BASE to find where the output data was generated.
Output from TCGen for this use case will be found in model_applications/s2s/TCGen_fcstGFSO_obsBDECKS_GDF_TDF/TCGen (relative to OUTPUT_BASE)
For each month and year there will be five files written:
* tc_gen_2016_pairs.nc
* tc_gen_2016_genmpr.txt
* tc_gen_2016_ctc.txt
* tc_gen_2016_cts.txt
* tc_gen_2016.stat
Keywords
sphinx_gallery_thumbnail_path = ‘_static/s2s-TCGen_fcstGFSO_obsBDECKS_GDF_TDF.png’
Total running time of the script: ( 0 minutes 0.000 seconds)