1. Overview

1.1. Purpose and organization of the User’s Guide

The goal of this User’s Guide is to equip users with the information needed to use the Model Evaluation Tools (MET) and its companion package METplus Wrappers. MET is a set of verification tools developed and supported to community via the Developmental Testbed Center (DTC) for use by the numerical weather prediction community. METplus Wrappers is a suite of Python wrappers and ancillary scripts to enhance the user’s ability to quickly set-up and run MET. Over the next few years, METplus Wrappers will become the authoritative repository for verification of the Unified Forecast System.

The METplus Wrappers User’s Guide is organized as follows. An overview of METplus Wrappers can be found below. Software Installation/Getting Started contains basic information about how to get started with METplus Wrappers - including system requirements, required software, and how to download METplus Wrappers. System Configuration provides information about configuring your environment and METplus Wrappers installation.

1.2. The Developmental Testbed Center (DTC)

METplus Wrappers has been developed, and will be maintained and enhanced, by the Developmental Testbed Center (DTC; http://www.dtcenter.org/ ). The main goal of the DTC is to serve as a bridge between operations and research and to facilitate the activities of these two important components of the numerical weather prediction (NWP) community. The DTC provides an environment that is functionally equivalent to the operational environment in which the research community can test model enhancements; the operational community benefits from DTC testing and evaluation of models before new models are implemented operationally. METplus Wrappers serves both the research and operational communities in this way - offering capabilities for researchers to test their own enhancements to models and providing a capability for the DTC to evaluate the strengths and weaknesses of advances in NWP prior to operational implementation.

METplus Wrappers will also be available to DTC visitors and the NOAA Unified Forecast System (UFS) and NCAR System for Integrated Modeling of the Atmosphere (SIMA) modeling communities for testing and evaluation of new model capabilities, applications in new environments, and so on. The METplus Wrappers release schedule is coincident with the MET release schedule and the METplus Wrappers major release number is six less than the MET major release number (e.g. MET 8.X is released with METplus Wrappers 2.X).

1.3. METplus Wrappers goals and design philosophy

METplus Wrappers is a Python scripting infrastructure for the MET tools. The primary goal of METplus Wrappers development is to provide MET users with a highly configurable and simple means to perform model verification using the MET tools. Prior to the availability of METplus Wrappers, users who had more complex verifications that required the use of more than one MET tool were faced with setting up multiple MET config files and creating some automation scripts to perform the verification. METplus Wrappers provides the user with the infrastructure to modularly create the necessary steps to perform such verifications.

METplus Wrappers has been designed to be modular and adaptable. This is accomplished through wrapping the MET tools with Python and the use of hierarchical configuration files to enable users to readily customize their verification environments. Wrappers can be run individually, or as a group of wrappers that represent a sequence of MET processes. New wrappers can readily be added to the METplus Wrappers package due to this modular design. Currently, METplus Wrappers can easily be applied by any user on their own computer platform that supports Python 3.6. We have deprecated support to Python 2.7.

The METplus Wrappers code and documentation is maintained by the DTC in Boulder, Colorado. METplus Wrappers is freely available to the modeling, verification, and operational communities, including universities, governments, the private sector, and operational modeling and prediction centers through a publicly accessible GitHub repository. Refer to Getting the METplus Wrappers source code for simple examples of obtaining METplus Wrappers.

1.4. METplus Wrappers Components

The major components of the METplus Wrappers package are METplus Python wrappers to the MET tools, MET configuration files and a hierarchy of METplus Wrappers configuration files. Some Python wrappers do not correspond to a particular MET tool, but wrap utilities to extend METplus functionality.

1.5. METplus Release Notes

When applicable, release notes are followed by the GitHub issue number which describes the bugfix, enhancement, or new feature: https://github.com/dtcenter/METplus/issues

New in v3.1.1 (20201222)

Bugfixes:

  • Missing dependencies for CyclonePlotter wrapper no longer prevent others wrappers from being used (#586)

  • Fixed corrupted time information error when running using custom loop list (#740)

  • Fixed logic preventing StatAnalysis to run with MakePlots (#743)

New in v3.1 (20200810)

Bugfixes:

  • Running EnsembleStat then GridStat fails when PCPCombine is also run (#509)

  • All changes included in 3.0.1 and 3.0.2 bugfix releases

New Wrappers:

New Use Cases:

  • StatAnalysis use case to demonstrate using Python Embedding (#492)

  • Develop new SWPC use case using_gen_vx_mask (#427)

  • Create a Feature Relative Use Case with User Diagnostics (#522)

  • Develop Surrogate Severe Calculation use-case (#413)

Enhancements:

  • GenVxMask wrapper doesn’t handle commas within command line arguments properly (#454)

  • Enhance PointStat to process one field at a time (#451)

  • GridStat and other wrappers set input dir to OUTPUT_BASE if not set (#446)

  • Add curl possibility to build_components build MET script (#513)

  • Change the shebang lines from “#!/usr/bin/env python” to “#!/usr/bin/env python3” (#503)

  • Add variable MET_BIN_DIR to replace {MET_INSTALL_DIR}/bin in the code (#502)

  • File window functionality gives useful message if not enough information provided in filename template (#517)

  • Enable METplus to only process certain months of a year (#512)

  • Enhance StatAnalysis/MakePlots to support use defined templates in plotting scripts (#500)

  • Create Docker image for METplus release (#498)

  • StatAnalysis wrapper no longer silently fails when no field information is provided (#422)

  • Allow regrid_data_plane wrapper to input multiple fields (#421)

  • Expand support for begin_end_incr syntax (#404)

  • Clean up StringSub/StringExtract calls (#343)

  • Rearrange ush with subdirs (#311)

Internal:

  • Change mouse over text for use cases to include config file name (#400)

  • Setup Initial Integration Test Framework - Travis CI (#185)

  • Setup new location to house INPUT DATA for testing (#461)

  • Split up use case tests so it can be run on Travis (#460)

  • Update Tutorial Chapter 4 for MET 9.0, METplus 3.0 (#428)

  • Reorganize sphinx documentation files (#418)

New in v3.0 (20200316)

  • Moved to using Python 3.6.3

  • User environment variables ([user_env_vars]) and [FCST/OBS]_VAR<n>_[NAME/LEVEL/OPTIONS] now support filename template syntax, i.e. {valid?fmt=%Y%m%d%H}

  • Added support for python embedding to supply gridded input data to MET tools (PCPCombine, GridStat, PointStat (gridded data only), RegridDataPlane…

  • PCPCombine now supports custom user-defined commands to build atypical use case calls

  • Improved logging to help debugging by listing expected file path

  • PyEmbedIngester wrapper added to allow python embedding for multiple data sources

  • Added support for month and year intervals for [INIT/VALID]_INCREMENT and LEAD_SEQ

  • Addition of contributor/developer guide as part of documentation

  • Documentation moved online using GitHub Pages and completely renamed, PDF option TBD.

  • Bugfix: PCPCombine subtract mode will call add method with 1 file if processing accumulation data and the lead time is equal to the desired accumulation

  • Bugfix: PCPCombine add mode forecast GRIB input

  • Bugfix: PCPCombine sum mode no longer fails when input level is not explicitly specified

1.6. Future development plans

METplus Wrappers is an evolving application. New capabilities are planned in controlled, successive version releases that are synchronized with MET releases. Bug fixes and user-identified problems will be addressed as they are found and posted to the known issues section of the METplus Wrappers Users web page (https://dtcenter.org/community-code/model-evaluation-tools-met). Future METplus Wrappers development plans are based on several contributing factors, including the needs of both the operational and research community. Issues that are in the development queue detailed in the “Issues” section of the GitHub repository. Please send questions to met_help@ucar.edu.

1.7. Code support

Support for METplus Wrappers is provided through a MET-help e-mail address: met_help@ucar.edu. We will endeavor to respond to requests for help in a timely fashion. In addition, information about METplus Wrappers and tools that can be used with MET are provided on the MET Users web page (https://dtcenter.org/community-code/model-evaluation-tools-met).

We welcome comments and suggestions for improvements to METplus Wrappers, especially information regarding errors. Comments may be submitted using the MET Feedback form available on the MET website. In addition, comments on this document would be greatly appreciated. While we cannot promise to incorporate all suggested changes, we will certainly take all suggestions into consideration.

METplus Wrappers is a “living” set of wrappers and configuration files. Our goal is to continually enhance it and add to its capabilities. Because our time, resources, and talents can at times be limited, we welcome contributed code for future versions of METplus. These contributions may represent new use cases or new plotting functions. For more information on contributing code to METplus Wrappers, please contact met_help@ucar.edu.