Note

Go to the end to download the full example code

QBO Phase plots and QBO Index: UserScript, Stat-Analysis

model_applications/ s2s_stratosphere/ UserScript_fcstGFS_obsERA_StratosphereQBO.py

Scientific Objective

Many common modes of variability in the troposphere have stratospheric teloconnection pathways. This use case performs evaluation of the Quasi-biennial Oscillation (QBO), one of the key players of stratosphereic varability, using several different calculations and plots. Specifically, phase diagrams can be used to compare the QBO plhase progression between the model and observations. Additionally, timeseries of U at 30 and 50mb are also plotted to compare the speed of propagation of QBO in the model versus observations. Continuous statistics (ME, RMSE, etc.) are calculated for U at 30 and 50mb, and are also computed separately to evaluate QBO in the easterly phase versus the westerly phase.

Datasets

  • GFS: 0-24 hour forecasts for 10/2017 - 2/2018

  • ERA: 30 year climatology for EOFs and 10/2017 - 2/2018

Data for this use case is not contained in the sample data tar files due to its size. Rather, it is stored as additional data in a separate tar file, named additional_data_UserScript_fcstGFS_obsERA_StratosphereQBO.tar.gz at https://dtcenter.ucar.edu/dfiles/code/METplus/METplus_Data/v6.0/.

METplus Components

This use case calls UserScript and StatAnalysis. The UserScript accesses calculation in METcalcpy, METplotpy, and METdataio. For it to run, the following versions of those repositories are needed, METcalcpy 3.0.0, METplotpy 3.0.0, and METdataio 2.1.

METplus Workflow

This use case does not loop but UserScript and StatAnalysis are each run once. The UserScript call runs the driver script stratosphere_qbo_driver.py which first computes zonal and meridional means using directional_means.py in METcalcpy on U from -10 S to 10N latitude. Then, an EOF analysis is performed on this zonal and meridional mean data, and two phase diagrams of QBO are created using the plot_qbo_phase_circuits and plot_qbo_phase_space functions from stratosphere_plots.py in METplotpy. Additionally the zonal and meridional mean at 30 and 50mb are output as time series to matched pair (MPR) files using write_mpr.py in METcalcpy and are also plotted as timeseries using the plot_u_timeseries function from stratosphere_plots.py in METplotpy. Finally StatAnalysis is run on the 30 and 50mb U mpr files to compute the bias (ME).

METplus Configuration

METplus first loads all of the configuration files found in parm/metplus_config, then it loads any configuration files passed to METplus via the command line, i.e parm/use_cases/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO.conf

[config]

# Documentation for this use case can be found at
# https://metplus.readthedocs.io/en/latest/generated/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereBias.html

# For additional information, please see the METplus Users Guide.
# https://metplus.readthedocs.io/en/latest/Users_Guide

###
# Processes to run
# https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-list
###
#PROCESS_LIST = UserScript(create_obs_eofs_filelist), UserScript(compute_plot_qbo), StatAnalysis(qbo_bias)
PROCESS_LIST = UserScript(compute_plot_qbo), StatAnalysis(qbo_bias)


###
# Time Info
# LOOP_BY options are INIT, VALID, RETRO, and REALTIME
# If set to INIT or RETRO:
#   INIT_TIME_FMT, INIT_BEG, INIT_END, and INIT_INCREMENT must also be set
# If set to VALID or REALTIME:
#   VALID_TIME_FMT, VALID_BEG, VALID_END, and VALID_INCREMENT must also be set
# LEAD_SEQ is the list of forecast leads to process
# https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#timing-control
###

LOOP_BY = INIT
INIT_TIME_FMT = %Y%m%d%H
INIT_BEG = 2017100100
INIT_END = 2018022821
INIT_INCREMENT = 86400
LEAD_SEQ = begin_end_incr(0,21,3)


USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE

LOOP_ORDER = processes

###
# User Environment Variables
###
[user_env_vars]
# Variable to determine if Zonal and Meridional means are need to be computed on the data used
# to create EOFS.  Set to True to compute zonal and meridional means.  If set to False, a file 
# should be provided in EOF_DATA_FILE_NAME that specifies the location of a dataset with zonal
# and meridional means.  The data should have the coordinates (time, pres), and the variable 
# names should be the same as what are set in OBS_TIME_VAR, OBS_LAT_VAR, and OBS_U_VAR below.
# Note, this is only for EOF calculation, and not for the data used to create plots.
COMPUTE_EOF_ZONAL_MERIDIONAL_MEAN = False
# Ignored if the above is set to True
EOF_DATA_FILE_NAME = {INPUT_BASE}/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/erai_zonal_mean_u.nc

# Option to save out the Zonal/Meridional Mean data that was computed for the EOF calculation
SAVE_EOF_ZONAL_MERIDIONAL_MEAN = False
# Name of the output Zonal/meridional mean file.  Ignored if The above is set to False
ZONAL_MERIDIONAL_MEAN_EOF_FILE_NAME = {INPUT_BASE}/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/erai_zonal_mean_u.nc

# Name of forecast variables for time, latitude, longitude, and U, and
# the dimension names for latitude and longitude
FCST_TIME_VAR = time 
FCST_LAT_VAR = latitude
FCST_U_VAR = u
FCST_LON_DIM = lon
FCST_LAT_DIM = lat

# Name of observation variables for time, latitude, longitude, and U and
# the dimension names for latitude and longitude
OBS_TIME_VAR = time
OBS_LAT_VAR = latitude
OBS_U_VAR = u
OBS_LON_DIM = lon
OBS_LAT_DIM = lat

# Minimum and maximum pressure levels to use in the QBO calculation
PRES_LEV_MIN = 10
PRES_LEV_MAX = 100

# Minimum and maximim latitude to use in the QBO calculation
LAT_MIN = -10
LAT_MAX = 10

# Name of the model used
MODEL_NAME = GFS

# Directory where output plots and statistics should be sent
OUTPUT_DIR = {OUTPUT_BASE}/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO

# Plot Settings
# Settings for the Circut Phase Diagram(s)
# List or single value of start times, each of which will have a phase diagram created 
PLOT_START_DATES = 10-1-2017
# Number of days for each diagram.  If the PLOT_START_DATES contains more than one value and
# PLOT_STAT_DATES[0]+PLOT_PERIODS != PLOT_START_DATES[1], the data between 
# PLOT_STAT_DATES[0]+PLOT_PERIODS and PLOT_START_DATES[1] will be omitted
PLOT_PERIODS = 151
# Full path to the output name of the phase circuits plot
PLOT_PHASE_CIRCUTS_OUTPUT_NAME = ERA_GFS_QBO_circuits.png

# Title and output name of the phase space plot
PLOT_PHASE_SPACE_TITLE = QBO Phase Space from ERA5 (10/2017 - 2/2018)
PLOT_PHASE_SPACE_OUTPUT_NAME = ERA5_QBO_PhaseSpace.png

# Time Series plot titles and output names for 30mb and 50mb
PLOT_TIME_SERIES_TITLE_30 = ERA5 and GFS Zonal Mean U at 30mb (10/2017 - 2/2018)
PLOT_TIME_SERIES_OUTPUT_NAME_30 = ERA_GFS_timeseries_30mb_u_201710_201802.png
PLOT_TIME_SERIES_TITLE_50 = ERA5 and GFS Zonal Mean U at 50mb (10/2017 - 2/2018)
PLOT_TIME_SERIES_OUTPUT_NAME_50 = ERA_GFS_timeseries_50mb_u_201710_201802.png


###
# Creates the file listing for the EOF creation UserScript Settings
# https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
###
[create_obs_eofs_filelist]
INIT_BEG = 1991010100
INIT_END = 2020123121
INIT_INCREMENT = 86400
LEAD_SEQ = 0

# Template of filenames to input to the user-script
USER_SCRIPT_INPUT_TEMPLATE = {INPUT_BASE}/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/ERA/{valid?fmt=%Y%m}/ERA_{valid?fmt=%Y%m%d%H}.nc

# Name of the file containing the listing of input files
# The option is OBS_EOF_INPUT; FCST_EOF_INPUT is not supported
# *** Make sure the order is the same as the order of templates listed in USER_SCRIPT_INPUT_TEMPLATE
USER_SCRIPT_INPUT_TEMPLATE_LABELS = OBS_EOF_INPUT

USER_SCRIPT_COMMAND = echo Populated file list for obs EOF Input


###
# QBO Calculation and plotting UserScript Settings
# https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
###
[compute_plot_qbo]
USER_SCRIPT_RUNTIME_FREQUENCY = RUN_ONCE

# Template of filenames to input to the user-script
USER_SCRIPT_INPUT_TEMPLATE = {INPUT_BASE}/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/ERA/{valid?fmt=%Y%m}/ERA_{valid?fmt=%Y%m%d%H}.nc, {INPUT_BASE}/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/GFS/GFS_{init?fmt=%Y%m%d%H}_f{lead?fmt=%HHH}h.nc

# Name of the file containing the listing of input files
# The options are OBS_INPUT and FCST_INPUT
# *** Make sure the order is the same as the order of templates listed in USER_SCRIPT_INPUT_TEMPLATE
USER_SCRIPT_INPUT_TEMPLATE_LABELS = OBS_INPUT, FCST_INPUT

USER_SCRIPT_COMMAND = {PARM_BASE}/use_cases/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/stratosphere_qbo_driver.py


[qbo_bias]
MODEL1 = GFS
MODEL1_OBTYPE = ADPUPA

STAT_ANALYSIS_CONFIG_FILE = {PARM_BASE}/met_config/STATAnalysisConfig_wrapped

STAT_ANALYSIS_JOB1 = -job aggregate_stat -line_type MPR -out_line_type CNT -fcst_var QBO_U -by FCST_LEV -out_stat [out_stat_file]_zonal_wind_CNT.stat
STAT_ANALYSIS_JOB2 = -job aggregate_stat -line_type MPR -out_line_type CNT -fcst_var QBO_U -by FCST_LEV,DESC -out_stat [out_stat_file]_zonal_wind_byphase_CNT.stat

MODEL_LIST = {MODEL1}
FCST_LEAD_LIST = 21

GROUP_LIST_ITEMS = MODEL_LIST
LOOP_LIST_ITEMS = 

MODEL1_STAT_ANALYSIS_LOOKIN_DIR = {OUTPUT_BASE}/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/mpr 

STAT_ANALYSIS_OUTPUT_DIR = {OUTPUT_BASE}/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO/StatAnalysis

MODEL1_STAT_ANALYSIS_OUT_STAT_TEMPLATE = {model?fmt=%s}_ERA_{obs_valid_beg?fmt=%Y%m%d}_{obs_valid_end?fmt=%Y%m%d}_{lead?fmt=%H%M%S}L

MET Configuration

METplus sets environment variables based on user settings in the METplus configuration file. See How METplus controls MET config file settings for more details.

YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!

If there is a setting in the MET configuration file that is currently not supported by METplus you’d like to control, please refer to: Overriding Unsupported MET config file settings

StatAnalysisConfig_wrapped

Note

See the Stat-Analysis MET Configuration section of the User’s Guide for more information on the environment variables used in the file below:

////////////////////////////////////////////////////////////////////////////////
//
// STAT-Analysis configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Filtering input STAT lines by the contents of each column
//
//model = [
${METPLUS_MODEL}

//desc  = [
${METPLUS_DESC}

//fcst_lead = [
${METPLUS_FCST_LEAD}

//obs_lead  = [
${METPLUS_OBS_LEAD}

//fcst_valid_beg  =
${METPLUS_FCST_VALID_BEG}

//fcst_valid_end  =
${METPLUS_FCST_VALID_END}

fcst_valid_inc  = [];
fcst_valid_exc  = [];

//fcst_valid_hour = [
${METPLUS_FCST_VALID_HOUR}


//obs_valid_beg   =
${METPLUS_OBS_VALID_BEG}

//obs_valid_end   =
${METPLUS_OBS_VALID_END}

obs_valid_inc   = [];
obs_valid_exc   = [];

//obs_valid_hour  = [
${METPLUS_OBS_VALID_HOUR}


//fcst_init_beg   =
${METPLUS_FCST_INIT_BEG}

//fcst_init_end   =
${METPLUS_FCST_INIT_END}

fcst_init_inc   = [];
fcst_init_exc   = [];

//fcst_init_hour  = [
${METPLUS_FCST_INIT_HOUR}


//obs_init_beg    =
${METPLUS_OBS_INIT_BEG}

//obs_init_end    =
${METPLUS_OBS_INIT_END}

obs_init_inc    = [];
obs_init_exc    = [];

//obs_init_hour   = [
${METPLUS_OBS_INIT_HOUR}


//fcst_var = [
${METPLUS_FCST_VAR}
//obs_var  = [
${METPLUS_OBS_VAR}

//fcst_units = [
${METPLUS_FCST_UNITS}
//obs_units  = [
${METPLUS_OBS_UNITS}

//fcst_lev = [
${METPLUS_FCST_LEVEL}
//obs_lev  = [
${METPLUS_OBS_LEVEL}

//obtype = [
${METPLUS_OBTYPE}

//vx_mask = [
${METPLUS_VX_MASK}

//interp_mthd = [
${METPLUS_INTERP_MTHD}

//interp_pnts = [
${METPLUS_INTERP_PNTS}

//fcst_thresh = [
${METPLUS_FCST_THRESH}
//obs_thresh = [
${METPLUS_OBS_THRESH}
//cov_thresh = [
${METPLUS_COV_THRESH}

//alpha = [
${METPLUS_ALPHA}

//line_type = [
${METPLUS_LINE_TYPE}

column = [];

weight = [];

////////////////////////////////////////////////////////////////////////////////

//
// Array of STAT-Analysis jobs to be performed on the filtered data
//
//jobs = [
${METPLUS_JOBS}

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
//
out_alpha = 0.05;

boot = {
   interval = PCTILE;
   rep_prop = 1.0;
   n_rep    = 0;
   rng      = "mt19937";
   seed     = "";
}

////////////////////////////////////////////////////////////////////////////////

//
// WMO mean computation logic
//
wmo_sqrt_stats   = [ "CNT:FSTDEV",  "CNT:OSTDEV",  "CNT:ESTDEV",
                     "CNT:RMSE",    "CNT:RMSFA",   "CNT:RMSOA",
                     "VCNT:FS_RMS", "VCNT:OS_RMS", "VCNT:RMSVE",
                     "VCNT:FSTDEV", "VCNT:OSTDEV" ];

wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",
                     "CNT:KT_CORR", "CNT:ANOM_CORR" ];

////////////////////////////////////////////////////////////////////////////////

//hss_ec_value =
${METPLUS_HSS_EC_VALUE}
rank_corr_flag = FALSE;
vif_flag       = FALSE;

tmp_dir = "${MET_TMP_DIR}";

//version        = "V10.0";

${METPLUS_MET_CONFIG_OVERRIDES}

Python Embedding

This use case does not use python embedding

Python Scripting

This use case runs the stratospher_qbo_driver.py python script. The processing performed by the script are detailed in the METplus Workflow section above.

Running METplus

Pass the use case configuration file to the run_metplus.py script along with any user-specific system configuration files if desired:

run_metplus.py /path/to/METplus/parm/use_cases/model_applications/s2s_stratosphere/UserScript_fcstGFS_obsERA_StratosphereQBO.conf /path/to/user_system.conf

See Running METplus for more information.

Expected Output

A successful run will output the following both to the screen and to the logfile:

INFO: METplus has successfully finished running.

There should be 4 graphics output to the plot directory in the location set as OUTPUT_DIR in the [user_env_vars] section of the configuration file:

  • ERA_GFS_QBO_circuits.png

  • ERA5_QBO_PhaseSpace.png

  • ERA_GFS_timeseries_30mb_u_201710_201802.png

  • ERA_GFS_timeseries_50mb_u_201710_201802.png

The name of the output graphics can be changed using PLOT_PHASE_CIRCUITS_OUTPUT_NAME, PLOT_PHASE_SPACE_OUTPUT_NAME, PLOT_TIME_SERIES_OUTPUT_NAME_30, and PLOT_TIME_SERIES_OUTPUT_NAME_50 also in the [user_env_vars] section. Additionally many matched pair .stat files will be output to OUTPUT_DIR/mpr, and tow computed continuous statistics will be output to OUTPUT_DIR/StatAnalysis:

  • GFS_ERA_20171001_20180228_210000L_zonal_wind_byphase_CNT.stat

  • GFS_ERA_20171001_20180228_210000L_zonal_wind_CNT.stat

Keywords

Note

  • UserScriptUseCase

  • S2SAppUseCase

  • S2SStratosphereAppUseCase

  • StatAnalysisUseCase

  • METcalcpyUseCase

  • METplotpyUseCase

Navigate to the METplus Quick Search for Use Cases page to discover other similar use cases.

sphinx_gallery_thumbnail_path = ‘_static/s2s_stratosphere-UserScript_fcstGFS_obsERA_StratosphereQBO.png’

Total running time of the script: (0 minutes 0.000 seconds)

Gallery generated by Sphinx-Gallery