1. Overview

1.1. Purpose and organization of the User’s Guide

The goal of this User’s Guide is to equip users with the information needed to use the Model Evaluation Tools (MET) and its companion package METplus Wrappers. MET is a set of verification tools developed and supported to community via the Developmental Testbed Center (DTC) for use by the numerical weather prediction community. METplus Wrappers is a suite of Python wrappers and ancillary scripts to enhance the user’s ability to quickly set-up and run MET. Over the next few years, METplus Wrappers will become the authoritative repository for verification of the Unified Forecast System.

The METplus Wrappers User’s Guide is organized as follows. An overview of METplus Wrappers can be found below. Installation contains basic information about how to get started with METplus Wrappers - including system requirements, required software, and how to download METplus Wrappers. Configuration provides information about configuring your environment and METplus Wrappers installation.

1.2. The Developmental Testbed Center (DTC)

METplus Wrappers has been developed, and will be maintained and enhanced, by the Developmental Testbed Center (DTC; http://www.dtcenter.org/ ). The main goal of the DTC is to serve as a bridge between operations and research and to facilitate the activities of these two important components of the numerical weather prediction (NWP) community. The DTC provides an environment that is functionally equivalent to the operational environment in which the research community can test model enhancements; the operational community benefits from DTC testing and evaluation of models before new models are implemented operationally. METplus Wrappers serves both the research and operational communities in this way - offering capabilities for researchers to test their own enhancements to models and providing a capability for the DTC to evaluate the strengths and weaknesses of advances in NWP prior to operational implementation.

METplus Wrappers will also be available to DTC visitors and the NOAA Unified Forecast System (UFS) and NCAR System for Integrated Modeling of the Atmosphere (SIMA) modeling communities for testing and evaluation of new model capabilities, applications in new environments, and so on. The METplus Wrappers release schedule is coincident with the MET release schedule and the METplus Wrappers major release number is six less than the MET major release number (e.g. MET 8.X is released with METplus Wrappers 2.X).

1.3. METplus Wrappers goals and design philosophy

METplus Wrappers is a Python scripting infrastructure for the MET tools. The primary goal of METplus Wrappers development is to provide MET users with a highly configurable and simple means to perform model verification using the MET tools. Prior to the availability of METplus Wrappers, users who had more complex verifications that required the use of more than one MET tool were faced with setting up multiple MET config files and creating some automation scripts to perform the verification. METplus Wrappers provides the user with the infrastructure to modularly create the necessary steps to perform such verifications.

METplus Wrappers has been designed to be modular and adaptable. This is accomplished through wrapping the MET tools with Python and the use of hierarchical configuration files to enable users to readily customize their verification environments. Wrappers can be run individually, or as a group of wrappers that represent a sequence of MET processes. New wrappers can readily be added to the METplus Wrappers package due to this modular design. Currently, METplus Wrappers can easily be applied by any user on their own computer platform that supports Python 3.8. We have deprecated support to Python 2.7.

The METplus Wrappers code and documentation is maintained by the DTC in Boulder, Colorado. METplus Wrappers is freely available to the modeling, verification, and operational communities, including universities, governments, the private sector, and operational modeling and prediction centers through a publicly accessible GitHub repository. Refer to Getting the METplus Wrappers source code for simple examples of obtaining METplus Wrappers.

1.4. METplus Wrappers Components

The major components of the METplus Wrappers package are METplus Python wrappers to the MET tools, MET configuration files and a hierarchy of METplus Wrappers configuration files. Some Python wrappers do not correspond to a particular MET tool, but wrap utilities to extend METplus functionality.

1.5. METplus Components Python Requirements

METplus Components Python Requirements

Name

Version

METplus Component

Source

Description

Use Cases (only applicable for METplus wrappers component)(followed by python package name)

Python 3.10.4+

METplus wrappers,
METcalcpy,
METplotpy,
METdataio

cartopy

>=0.21.1

METplus wrappers,
METcalcpy,
METplotpy

https://scitools.org.uk/cartopy/docs/latest/

Designed for
geospatial data
processing in
order to produce
maps and other
geospatial data
analyses

TCGen: Genesis Density Function (GDF) and Track Density Function (TDF) (cartopy)
CyclonePlotter: Extra-TC Tracker and Plotting Capabilities (cartopy)

cfgrib

METplus wrappers

https://pypi.org/project/cfgrib/

map GRIB files
to the NetCDF
Common Data Model
following the
CF Convention
using ecCodes

Multi_Tool: Feature Relative by Lead using Multiple User-Defined Fields (cfgrib)

cmocean

METcalcpy,
METplotpy

https://pypi.org/project/cmocean/

contains colormaps
for commonly-used
oceanographic variables

dateutil

>=2.8.2

METplus wrappers

https://github.com/dateutil/dateutil/releases

provides powerful
extensions to the
standard datetime
module

Most

eofs

METplus wrappers,
METcalcpy,
METplotpy

https://pypi.org/project/eofs/

empirical orthogonal
functions analysis of
spatial-temporal data

WeatherRegime Calculation: RegridDataPlane, PcpCombine, and WeatherRegime python code (eofs)
WeatherRegime Calculation: GFS and ERA RegridDataPlane, PcpCombine, and WeatherRegime python code (eofs)

h5py

METplus wrappers

https://github.com/h5py/h5py

Pythonic interface
to the HDF5
binary data format

PCPCombine: Python Embedding Use Case (h5py)

imutils

>=0.5.4

METplotpy

https://pypi.org/project/imutils/

A series of convenience
functions to make basic
image processing
functions such as
translation, rotation,
resizing, skeletonization,
displaying Matplotlib
images, sorting contours,
detecting edges,
and much more easier

imageio

METcalcpy,
METplotpy

https://pypi.org/project/imageio/

provides an easy
interface to read
and write a wide
range of image data,
including animated
images, volumetric data,
and scientific formats

lxml

>=4.9.1

METcalcpy,
METplotpy,
METdataio

https://pypi.org/project/lxml/

a Pythonic binding for
the C libraries
libxml2 and libxslt

matplotlib

>=3.6.3

METplus wrappers,
METcalcpy,
METplotpy

https://matplotlib.org/stable/users/installing/index.html

a comprehensive library
for creating static,
animated, and
interactive visualizations

UserScript: Make OMI plot from calculated MJO indices (obs only) (matplotlib)
TCGen: Genesis Density Function (GDF) and Track Density Function (TDF) (matplotlib)
UserScript: Make a Phase Diagram plot from input RMM or OMI (matplotlib)
UserScript: Make OMI plot from calculated MJO indices (matplotlib)
UserScript: Make RMM plots from calculated MJO indices (matplotlib)
CyclonePlotter: Extra-TC Tracker and Plotting Capabilities (matplotlib)

metcalcpy

METplus wrappers,
METcalcpy,
METplotpy

https://github.com/dtcenter/METcalcpy/releases

a Python version of the
statistics calculation
functionality of
METviewer, METexpress,
plotting packages in
METplotpy and is a
stand-alone package for
any other application

UserScript: Make a Hovmoeller plot (metcalcpy)
UserScript: Make a Cross Spectra plot (metcalcpy)
Grid-Stat: Verification of TC forecasts against merged TDR data (metcalcpy)
UserScript: Calculate the Difficulty Index (metcalcpy)
UserScript: Make zonal and meridonial means (metcalcpy)

metplotpy

METplus wrappers

https://github.com/dtcenter/METplotpy/releases

contains packages for
plotting in METplus as
stand-alone, or part of
METplus use case,
METplus wrappers,
METexpress,
or METviewer

UserScript: Make a Hovmoeller plot (metplotpy)
UserScript: Make a Cross Spectra plot (metplotpy)
UserScript: Calculate the Difficulty Index (metplotpy)
TCGen: Genesis Density Function (GDF) and Track Density Function (TDF) (metplotpy)

metpy

>=1.4.0

METplus wrappers

https://www.unidata.ucar.edu/software/metpy/

a collection of tools
in Python for reading,
visualizing, and
performing calculations
with weather data

Multi_Tool: Feature Relative by Lead using Multiple User-Defined Fields (metpy)

nc-time-axis

1.4

METplotpy
stratosphere_diagnostics

https://github.com/SciTools/nc-time-axis

extension to cftime
**REQUIRES Python 3.7

netCDF4

>=1.6.2

METplus wrappers,
METcalcpy,
METplotpy

https://unidata.github.io/netcdf4-python/

a Python interface to
the netCDF C library

For using MET Python embedding functionality in use cases

numpy

>=1.24.2

METplus wrappers,
METcalcpy,
METplotpy,
METdataio

https://numpy.org/

NumPy offers
comprehensive
mathematical functions,
random number generators,
linear algebra routines,
Fourier transforms, and more.

For using MET Python embedding functionality in use cases

pandas

>=1.5.2

METplus wrappers,
METcalcpy,
METplotpy,
METdataio

https://pypi.org/project/pandas

a fast, powerful,
flexible and easy to use
open source data analysis
and manipulation tool,
built on top of the
Python programming
language

For using MET Python embedding functionality in use cases

pint

>=0.20.1

METcalcpy

https://github.com/hgrecco/pint

Python package to define,
operate and manipulate
physical quantities

plotly

>=5.13.0

METcalcpy,
METplotpy

https://github.com/plotly/plotly.py

makes interactive,
publication-quality graphs

pygrib

METplus wrappers

https://github.com/jswhit/pygrib

for reading/writing
GRIB files

Multi_Tool: Feature Relative by Lead using Multiple User-Defined Fields (pygrib)

pylab

METplus wrappers

https://pypi.org/project/matplotlib/

a convenience module
that bulk imports
matplotlib.pyplot (for
plotting) and NumPy (for
Mathematics and working
with arrays) in a
single name space

WeatherRegime Calculation: RegridDataPlane, PcpCombine, and WeatherRegime python code (pylab)
WeatherRegime Calculation: GFS and ERA RegridDataPlane, PcpCombine, and WeatherRegime python code (pylab)

pymysql

>=1.0.2

METcalcpy,
METplotpy,
METdataio

https://pypi.org/project/psutil/

a pure-Python MySQL
client library,
based on PEP 249

pyproj

>=2.3.1

METplus wrappers

https://github.com/pyproj4/pyproj/archive/v2.3.1rel.tar.gz

Python interface to PROJ
(cartographic projections and
coordinate transformations library)

GridStat: Python Embedding to read and process ice cover (pyproj)

pyresample

METplus wrappers

https://github.com/pytroll/pyresample

for resampling geospatial
image data

GridStat: Python Embedding to read and process SST (pyresample)
GridStat: Python Embedding to read and process ice cover (pyresample)
GridStat: Python Embedding for sea surface salinity using level 3, 1 day composite obs (pyresample)
GridStat: Python Embedding for sea surface salinity using level 3, 8 day mean obs (pyresample)

pytest

>=7.2.1

METcalcpy,
METplotpy,
METdataio

https://github.com/pytest-dev/pytest/archive/5.2.1.tar.gz

a mature full-featured
Python testing tool that
helps you write better
programs

python-kaleido

>=0.2.1

METcalcpy,
METplotpy

https://pypi.org/project/kaleido/

provides a low-level
Python API that is
designed to be used by
high-level plotting
libraries like Plotly

pyyaml

>=6.0

METcalcpy,
METplotpy,
METdataio

https://github.com/yaml/pyyaml

a full-featured YAML
framework for the Python
programming language

scikit-image

>=0.19.3

METcalcpy,
METplotpy

https://scikit-image.org

a collection of
algorithms for image
processing

scikit-learn

>=1.2.2

METplus wrappers,
METcalcpy,
METplotpy

https://github.com/scikit-learn/scikit-learn/releases

Open Source library for
Machine Learning in Python

GridStat: Python Embedding to read and process SST (scikit-learn)
GridStat: Python Embedding to read and process ice cover (scikit-learn)
GridStat: Python Embedding for sea surface salinity using level 3, 1 day composite obs (scikit-learn)
GridStat: Python Embedding for sea surface salinity using level 3, 8 day mean obs (scikit-learn)

scipy

>=1.9.3

METplus wrappers,
METcalcpy,
METplotpy

https://www.scipy.org/

wraps highly-optimized
implementations written
in low-level languages
like Fortran, C, and C++

Blocking Calculation: ERA RegridDataPlane, PcpCombine, and Blocking python code (scipy)
WeatherRegime Calculation: RegridDataPlane, PcpCombine, and WeatherRegime python code (scipy)
UserScript: Make OMI plot from calculated MJO indices (obs only) (scipy)
WeatherRegime Calculation: GFS and ERA RegridDataPlane, PcpCombine, and WeatherRegime python code (scipy)
Blocking Calculation: GFS and ERA RegridDataPlane, PcpCombine, and Blocking python code (scipy)
UserScript: Make a Phase Diagram plot from input RMM or OMI (scipy)
UserScript: Make OMI plot from calculated MJO indices (scipy)
UserScript: Make RMM plots from calculated MJO indices (scipy)

sklearn

METplus wrappers

https://www.kite.com/python/docs/sklearn

Simple and efficient
tools for predictive
data analysis

WeatherRegime Calculation: RegridDataPlane, PcpCombine, and WeatherRegime python code (sklearn)
WeatherRegime Calculation: GFS and ERA RegridDataPlane, PcpCombine, and WeatherRegime python code (sklearn)

xarray

>=2023.1.0

METplus wrappers,
METcalcpy,
METplotpy

https://xarray.pydata.org/en/v0.17.0/

makes working with
labelled
multi-dimensional arrays
simple, efficient,
and fun

For using MET Python embedding functionality in use cases

xesmf

METplus wrappers

NOTE: The xesmf package
will not be installed
on WCOSS2 and there is
an open GitHub issue to
replace the package in
the use case that uses it:
https://github.com/dtcenter/METplus/issues/1314

for regridding

PlotDataPlane: Python Embedding of tripolar coordinate file (xesmf)

yaml

METcalcpy,
METplotpy

https://pypi.org/project/PyYAML/

to load, read, and write
YAML files with PyYAML

1.6. Future development plans

METplus Wrappers is an evolving application. New capabilities are planned in controlled, successive version releases that are synchronized with MET releases. Software bugs and user-identified problems will be documented using GitHub issues and fixed either in the next bugfix or official release. Future METplus Wrappers development plans are based on several contributing factors, including the needs of both the operational and research community. Issues that are in the development queue detailed in the “Issues” section of the GitHub repository. Please create a post in the METplus GitHub Discussions Forum with any questions.

1.7. Code support

Support for METplus Wrappers is provided through the METplus GitHub Discussions Forum. We will endeavor to respond to requests for help in a timely fashion. In addition, information about METplus Wrappers and tools that can be used with MET are provided on the MET Users web page.

We welcome comments and suggestions for improvements to METplus Wrappers, especially information regarding errors. Comments may be submitted using the MET Feedback form available on the MET website. In addition, comments on this document would be greatly appreciated. While we cannot promise to incorporate all suggested changes, we will certainly take all suggestions into consideration.

METplus Wrappers is a “living” set of wrappers and configuration files. Our goal is to continually enhance it and add to its capabilities. Because our time, resources, and talents can at times be limited, we welcome contributed code for future versions of METplus. These contributions may represent new use cases or new plotting functions. For more information on contributing code to METplus Wrappers, please create a post in the METplus GitHub Discussions Forum.