Note
Go to the end to download the full example code
UserScript: Make a Phase Diagram plot from input RMM or OMI
model_applications/ s2s_mjo/ UserScript_obsERA_obsOnly_PhaseDiagram.py
Scientific Objective
To produce a phase diagram using either OLR based MJO Index (OMI) or the Real-time Multivariate MJO index (RMM)
Datasets
Forecast dataset: None.
Observation dataset: ERA Reanlaysis Outgoing Longwave Radiation.
External Dependencies
You will need to use a version of Python 3.6+ that has the following packages installed:
* numpy
* netCDF4
* datetime
* xarray
* matplotlib
* scipy
* pandas
If the version of Python used to compile MET did not have these libraries at the time of compilation, you will need to add these packages or create a new Python environment with these packages.
If this is the case, you will need to set the MET_PYTHON_EXE environment variable to the path of the version of Python you want to use. If you want this version of Python to only apply to this use case, set it in the [user_env_vars] section of a METplus configuration file.:
[user_env_vars] MET_PYTHON_EXE = /path/to/python/with/required/packages/bin/python
METplus Components
This use case runs the Phase Diagram driver which and creates a phase diagram. Inputs to the driver are a text file containing the following columns, yyyy,mm,dd,hh,pc1,pc2,amp for OMI, or yyyy,mm,dd,pc1,pc2,phase,amp,source for RMM.
METplus Workflow
The Phase diagram driver script python code is run for each lead time on the forecast and observations data. This example loops by valid time for the model pre-processing, and valid time for the other steps. It creates the phase diagram plot and a text file listing of the valid times to use in creating the plots.
METplus Configuration
METplus first loads all of the configuration files found in parm/metplus_config, then it loads any configuration files passed to METplus via the command line i.e. parm/use_cases/model_applications/s2s_mjo/UserScript_obsERA_obsERA_OMI.conf. The file UserScript_obsERA_obsOnly_PhaseDiagram/PhaseDiagram_driver.py runs the python program and UserScript_obsERA_obsOnly_PhaseDiagram.conf sets the variables for all steps of the use case.
[config]
# Documentation for this use case can be found at
# https://metplus.readthedocs.io/en/latest/generated/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram.html
# For additional information, please see the METplus Users Guide.
# https://metplus.readthedocs.io/en/latest/Users_Guide
###
# Processes to run
# https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-list
###
PROCESS_LIST = UserScript(obs_time_filelist), UserScript(script_PhaseDiagram)
###
# Time Info
# LOOP_BY options are INIT, VALID, RETRO, and REALTIME
# If set to INIT or RETRO:
# INIT_TIME_FMT, INIT_BEG, INIT_END, and INIT_INCREMENT must also be set
# If set to VALID or REALTIME:
# VALID_TIME_FMT, VALID_BEG, VALID_END, and VALID_INCREMENT must also be set
# LEAD_SEQ is the list of forecast leads to process
# https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#timing-control
###
LOOP_BY = VALID
VALID_TIME_FMT = %Y%m%d%H
VALID_BEG = 2012010100
VALID_END = 2012033100
VALID_INCREMENT = 86400
LEAD_SEQ = 0
# variables referenced in other sections
# Run the obs for these cases
OBS_RUN = True
FCST_RUN = False
# Input and Output Directories for the OBS OLR Files and output text file containing the file list
OBS_PDTIME_FMT = %Y%m%d-%H%M%S
OBS_PDTIME_INPUT_TEMPLATE = {valid?fmt=%Y%m%d-%H%M%S}
OBS_PDTIME_OUTPUT_DIR = {OUTPUT_BASE}/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram/
OBS_PDTIME_OUTPUT_TEMPLATE = time_list_lead{lead?fmt=%HHH}.txt
###
# UserScript(obs_time_filelist) Settings
# https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
###
# Create a time file that contains the times we want to filter for plotting
[obs_time_filelist]
# Find the files for each time
USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE_FOR_EACH
USER_SCRIPT_COMMAND = {METPLUS_BASE}/parm/use_cases/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram/save_input_files_txt.py {OBS_PDTIME_INPUT_TEMPLATE} {OBS_PDTIME_OUTPUT_DIR}/{OBS_PDTIME_OUTPUT_TEMPLATE}
# Configurations for the Phase Diagram Plotting Script
[user_env_vars]
# Whether to Run the model or obs
RUN_OBS = {OBS_RUN}
RUN_FCST = {FCST_RUN}
# Make OUTPUT_BASE Available to the script
SCRIPT_OUTPUT_BASE = {OUTPUT_BASE}
# Index to Plot
PLOT_INDEX = RMM
# Input Directories
OBS_PHASE_DIAGRAM_INPUT_DIR = {INPUT_BASE}/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram
# Input filename template
OBS_PHASE_DIAGRAM_INPUT_FILE = rmm.1x.txt
# Input Time file
OBS_PHASE_DIAGRAM_INPUT_TIMELIST_TEXTFILE = {OBS_PDTIME_OUTPUT_DIR}/{OBS_PDTIME_OUTPUT_TEMPLATE}
OBS_PHASE_DIAGRAM_INPUT_TIME_FMT = {OBS_PDTIME_FMT}
# Plot Output Directory
PHASE_DIAGRAM_PLOT_OUTPUT_DIR = {OUTPUT_BASE}/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram/plots
# Plot Ouptut Name
OBS_PHASE_PLOT_OUTPUT_NAME = RMM_phase_diagram
###
# UserScript(script_PhaseDiagram) Settings
# https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
###
# Configurations for UserScript: Run the RMM Analysis driver
[script_PhaseDiagram]
# list of strings to loop over for each run time.
# Run the user script once per lead
USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE_PER_LEAD
# Command to run the user script with input configuration file
USER_SCRIPT_COMMAND = {METPLUS_BASE}/parm/use_cases/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram/PhaseDiagram_driver.py
MET Configuration
METplus sets environment variables based on the values in the METplus configuration file. These variables are referenced in the MET configuration file. YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS! If there is a setting in the MET configuration file that is not controlled by an environment variable, you can add additional environment variables to be set only within the METplus environment using the [user_env_vars] section of the METplus configuration files. See the ‘User Defined Config’ section on the ‘System Configuration’ page of the METplus User’s Guide for more information.
Python Scripts
The phase diagram driver script orchestrates the generation of a phase diagram plot: parm/use_cases/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_OMI/PhaseDiagram_driver.py:
#!/usr/bin/env python3
"""
Driver Script to read in OMI or RMM indices and plot phase diagram for specified dates.
OMI values can be obtained from https://psl.noaa.gov/mjo/, RMM values can be obtained from
http://www.bom.gov.au/climate/mjo/graphics/rmm.74toRealtime.txt
"""
import os
import atexit
import numpy as np
import pandas as pd
import datetime
import warnings
import metplotpy.contributed.mjo_rmm_omi.plot_mjo_indices as pmi
def handle_exit(obs_timefile,fcst_timefile):
try:
os.remove(obs_timefile)
except:
pass
try:
os.remove(fcst_timefile)
except:
pass
def run_phasediagram_steps(inlabel, alldata_timefile, oplot_dir):
# which index are we plotting
indexname = os.environ['PLOT_INDEX']
pltfile = os.path.join(os.environ[inlabel+'_PHASE_DIAGRAM_INPUT_DIR'],
os.environ[inlabel+'_PHASE_DIAGRAM_INPUT_FILE'])
# read data from text file
if indexname=='OMI':
data = pd.read_csv(pltfile, header=None, delim_whitespace=True, names=['yyyy','mm','dd','hh','pc1','pc2','amp'],
parse_dates={'dtime':['yyyy','mm','dd','hh']})
elif indexname=='RMM':
data = pd.read_csv(pltfile, header=None, delim_whitespace=True,
names=['yyyy','mm','dd', 'pc1','pc2','phase','amp','source'], parse_dates={'dtime':['yyyy','mm','dd']})
# Get the file with the listing of times and format of this file
alldata_timefmt = os.environ[inlabel+'_PHASE_DIAGRAM_INPUT_TIME_FMT']
# Read the file
with open(alldata_timefile) as at:
alldata_time = at.read().splitlines()
keepdata = []
for dd in alldata_time:
timeloc = np.where(data.dtime == datetime.datetime.strptime(dd,alldata_timefmt))
if len(timeloc[0]) > 0:
for l in timeloc[0]:
keepdata.append(l)
pltdata = data.iloc[keepdata]
dates = np.array(pltdata.dtime.dt.strftime('%Y%m%d').values,dtype=int)
months = np.array(pltdata.dtime.dt.strftime('%m').values,dtype=int)
days = np.array(pltdata.dtime.dt.strftime('%d').values,dtype=int)
PC1 = np.array(pltdata.pc1.values)
PC2 = np.array(pltdata.pc2.values)
# plot the phase diagram
phase_plot_name = os.path.join(oplot_dir,os.environ.get(inlabel+'_PHASE_PLOT_OUTPUT_NAME',inlabel+'_phase'))
phase_plot_format = os.environ.get(inlabel+'_PHASE_PLOT_OUTPUT_FORMAT','png')
# plot the phase diagram
pmi.phase_diagram(indexname,PC1,PC2,dates,months,days,phase_plot_name,'png')
def main():
obs_timelist = os.path.join(os.environ.get('OBS_PHASE_DIAGRAM_INPUT_DIR',''),
os.environ.get('OBS_PHASE_DIAGRAM_INPUT_TIMELIST_TEXTFILE',''))
fcst_timelist = os.path.join(os.environ.get('FCST_PHASE_DIAGRAM_INPUT_DIR',''),
os.environ.get('FCST_PHASE_DIAGRAM_INPUT_TIMELIST_TEXTFILE',''))
atexit.register(handle_exit,obs_timelist,fcst_timelist)
# Check for an output plot directory in the configs. Create one if it does not exist
oplot_dir = os.environ.get('PHASE_DIAGRAM_PLOT_OUTPUT_DIR','')
if not oplot_dir:
obase = os.environ['OUTPUT_BASE']
oplot_dir = os.path.join(obase,'plots')
if not os.path.exists(oplot_dir):
os.makedirs(oplot_dir)
# Determine if doing forecast or obs
run_obs_phasediagram = os.environ.get('RUN_OBS','False').lower()
run_fcst_phasediagram = os.environ.get('FCST_RUN_FCST','False').lower()
# Run the steps to compute OMM
# Observations
if (run_obs_phasediagram == 'true'):
run_phasediagram_steps('OBS', obs_timelist, oplot_dir)
# Forecast
if (run_fcst_phasediagram == 'true'):
run_phasediagram_steps('FCST', fcst_timelist, oplot_dir)
# nothing selected
if (run_obs_phasediagram == 'false') and (run_fcst_phasediagram == 'false'):
warnings.warn('Forecast and Obs runs not selected, no plots will be created')
warnings.warn('Set RUN_FCST or RUN_OBS in the [user_en_vars] section to generate output')
if __name__ == "__main__":
main()
#! /usr/bin/env python
import os
import sys
input_file = sys.argv[1]
output_file = sys.argv[2]
output_dir = os.path.dirname(output_file)
if not os.path.exists(output_dir):
print(f'Creating output dir: {output_dir}')
os.makedirs(output_dir)
filelist = open(output_file,'a+')
filelist.write(input_file + '\n')
filelist.close()
Running METplus
This use case is run in the following ways:
Passing in UserScript_obsERA_obsOnly_PhaseDiagram.conf then a user-specific system configuration file:
run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram.conf -c /path/to/user_system.conf
Modifying the configurations in parm/metplus_config, then passing in UserScript_obsERA_obsOnly_PhaseDiagram.py:
run_metplus.py -c /path/to/METplus/parm/use_cases/model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram.conf
The following variables must be set correctly:
INPUT_BASE - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases
OUTPUT_BASE - Path where METplus output will be written. This must be in a location where you have write permissions
MET_INSTALL_DIR - Path to location where MET is installed locally
Example User Configuration File:
[dir]
INPUT_BASE = /path/to/sample/input/data
OUTPUT_BASE = /path/to/output/dir
MET_INSTALL_DIR = /path/to/met-X.Y
Expected Output
Refer to the value set for OUTPUT_BASE to find where the output data was generated. Output for this use case will be found in model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram. This may include the regridded data and daily averaged files. In addition, the phase diagram plots will be generated and the output location can be specified as PHASE_DIAGRAM_PLOT_OUTPUT_DIR. If it is not specified, plots will be sent to model_applications/s2s_mjo/UserScript_obsERA_obsOnly_PhaseDiagram/plots (relative to OUTPUT_BASE).
Keywords
Note
S2SAppUseCase
S2SMJOAppUseCase
METplotpyUseCase
Navigate to METplus Quick Search for Use Cases to discover other similar use cases.
sphinx_gallery_thumbnail_path = ‘_static/s2s_mjo-UserScript_obsERA_obsOnly_PhaseDiagram.png’
Total running time of the script: (0 minutes 0.000 seconds)