"""
PointStat: CESM and FLUXNET2015 Terrestrial Coupling Index (TCI) 
================================================================

model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf

"""
##############################################################################
# .. contents::
#   :depth: 1
#   :local:
#   :backlinks: none

##############################################################################
# Scientific Objective
# --------------------
# This use case ingests two CESM (CAM and CLM) files and raw FLUXNET2015 data.
# The use case calculates the Terrestrial Coupling Index (TCI) from the
# CESM forecasts and FLUXNET observations.
# Utilizing Python embedding, this use case taps into a new vital observation
# dataset and compares it to CESM forecasts of TCI.
# Finally, it will generate plots of model forecast TCI overlaid with
# TCI observations at FLUXNET sites.
#
# The reference for the Terrestrial Coupling Index calculation is as follows:
#
# Dirmeyer, P. A., 2011: The terrestrial segment of soil moisture-climate
# coupling. *Geophys. Res. Lett.*, **38**, L16702, doi: 10.1029/2011GL048268.
#

##############################################################################
# Version Added
# -------------
#
# METplus version 5.1

##############################################################################
# Datasets
# --------
#
# **Forecast:** CESM 1979-1983 Simulations
#
# * Community Land Model (CLM) file
# * Community Atmosphere Model (CAM) file
#
# .. note::
#
#    The CESM data used here are in NetCDF format which span several years and
#    have a time dimension. In the event a user has GRIB or other data that does not
#    contain a time dimension, it is advised that they use Python or other tools to 
#    create a data object or NetCDF file such that a time dimension is available to use
#    for computing TCI. The user may also need to adjust various settings in the use
#    case configuration file to match their resulting dataset.
#
# **Observations:** Raw FLUXNET2015 observations
#
# **Data Source:** CESM - NSF NCAR Climate & Global Dynamics (CGD);
# FLUXNET2015 "SUBSET" Data Product:
# https://fluxnet.org/data/fluxnet2015-dataset/subset-data-product/
#
# **Location:** All of the input data required for this use case can be
# found in a sample data tarball. Each use case category will have
# one or more sample data tarballs. It is only necessary to download
# the tarball with the use case’s dataset and not the entire collection
# of sample data. Click here to access the METplus releases page and download sample data
# for the appropriate release: https://github.com/dtcenter/METplus/releases
# This tarball should be unpacked into the directory that you will
# set the value of INPUT_BASE. See :ref:`running-metplus` section for more information.
#


##############################################################################
# METplus Components
# ------------------
#
# This use case utilizes the PyEmbedIngest to read the CESM files and calculate
# TCI using python embedding and a NETCDF file of the TCI is generated.
# Then PointStat processes the output of PyEmbedIngest and FLUXNET2015 dataset
# (using Python embedding), and outputs the requested line types.
# Then the PlotPointObs tool reads the output of PyEmbedIngest and
# FLUXNET2015 dataset and produce plots of TCI from CESM and point observations.
# A custom loop runs through all the pre-defined seasons (DJF, MAM, JJA, SON)
# and runs PyEmbedIngest, PointStat, and PlotPointObs.
#

##############################################################################
# METplus Workflow
# ----------------
#
# The PyEmbedIngest tool reads 2 CESM files containing Soil Moisture (CLM file)
# and Sensible Heat Flux (CAM file), each composed of daily forecasts from
# 1979 to 1983 and calculates TCI and generates a NETCDF file of the TCI.
# Raw CSV files containing FLUXNET station observations of latent heat flux
# (LE_F_MDS) and soil water content at the shallowest level (SWC_F_MDS_1)
# are read using Python embedding, and TCI is computed.
#
# **Beginning time (VALID_BEG):** 1979-01-01 at 00z
#
# **End time (VALID_END):** 1979-01-01 at 00z
#
# **Increment between beginning and end times (VALID_INCREMENT):** None
#
# **Sequence of forecast leads to process (LEAD_SEQ):** 0
#
# PointStat is used to compare the two new fields
# (TCI calculated from CESM dataset and FLUXNET2015).
# Finally, PlotPointObs is run to plot the CESM TCI overlaying the
# FLUXNET2015 point observations.
#
# .. note::
# 
#   The CESM forecasts cover a time period prior to the availability of FLUXNET
#   observations. Thus, this use case should be considered a demonstration of
#   the capability to read CESM forecast data, raw FLUXNET observation data,
#   and compute TCI, rather than a bonafide scientific application.
#   The use case is designed to enforce seasonal alignment, but it is not
#   designed to enforce date/time alignment. In this case, the CESM data cover
#   1979-1983, whereas the sample FLUXNET observations cover varying time
#   ranges depending on the site.
#

##############################################################################
# METplus Configuration
# ---------------------
#
# METplus first loads all of the configuration files found in parm/metplus_config,
# then it loads any configuration files passed to METplus via the command line
# i.e. parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf
#
# .. highlight:: bash
# .. literalinclude:: ../../../../parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf

##############################################################################
# MET Configuration
# -----------------
#
# METplus sets environment variables based on user settings in the METplus
# configuration file. See :ref:`How METplus controls MET config file settings<metplus-control-met>` for more details.
#
# **YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!**
#
# If there is a setting in the MET configuration file that is currently
# not supported by METplus you’d like to control, please refer to:
# :ref:`Overriding Unsupported MET config file settings<met-config-overrides>`
#
# .. dropdown:: PointStatConfig_wrapped
#
#   .. highlight:: bash
#   .. literalinclude:: ../../../../parm/met_config/PointStatConfig_wrapped

##############################################################################
# Python Embedding
# ----------------
#
# This use case requires the following Python dependencies:
#
# * Xarray
# * Pandas
# * METcalcpy 3.0.0+
#
# This use case uses a Python embedding script to read both the forecast and
# observation data, in order to compute TCI,
# which is the diagnostic that is being verified by MET using PointStat.
# The forecast data (CESM in this case) is read using:
#
# .. dropdown:: parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/tci_fcst.py
#
#    .. highlight:: python
#    .. literalinclude:: ../../../../parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/tci_fcst.py
#
# The user can control all arguments to this script via the METplus use case configuration file using the following config entries:
#
# .. glossary::
# 
#    TCI_FCST_LHFLUX_VAR
#      The latent heat flux variable from the forecast data to use for computing TCI
#
#      | *Default:* LHFLX
#
#    TCI_FCST_SOILM_VAR
#      The soil moisture variable from the forecast data to use for computing TCI
#
#      | *Default:* SOILWATER_10CM
#
#    TCI_FCST_LHFLUX_FILE_PATH
#      The path to the model netcdf file that contains the latent heat flux field
#
#    TCI_FCST_SOILM_FILE_PATH
#      The path to the model netcdf file that contains the soil moisture field
#
# The raw FLUXNET2015 SUBSET data are read using:
#
# .. dropdown:: parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/fluxnet2015_tci.py
# 
#   .. highlight:: python
#   .. literalinclude:: ../../../../parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/fluxnet2015_tci.py
#
# The user can control all command line arguments to this script via METplus config entries:
#
# .. glossary::
#
#    FLUXNET_DATA_DIR
#      The directory containing raw FLUXNET CSV files
#
#    FLUXNET_LAT_HEAT_VAR
#      The FLUXNET surface latent heat flux variable name to use for computing TCI
#
#      | *Default:* LE_F_MDS
#
#    FLUXNET_SOIL_MOIST_VAR
#      The FLUXNET soil moisture variable name to use for computing TCI
#
#      | *Default:* SWC_F_MDS_1
#
#    FLUXNET_OBS_METADATA_FILE
#      The absolute path to the fluxnetstations.csv metadata file included with the use case
#
# and for data filtering options, via METplus config entries:
#
# .. glossary::
#
#    FLUXNET_SKIP_LEAP_DAYS
#      Skip FLUXNET observations from 29 February days
#
#      | *Default:* True
#
#    FLUXNET_HIGHRES_QC_THRESH
#      The fraction of higher temporal resolution FLUXNET data required to have
#      passed quality control in order to use the daily data.
#
#      | *Default:* 0.8
#
#    FLUXNET_MIN_DAYS_PER_SEASON
#      The minimum number of days to include in individual seasons at each site
#
#      | *Default:* 1
#
#    FLUXNET_MIN_DAYS_PER_SITE
#      The minimum number of days for all seasons at each site
#
#      | *Default:* 10
#
#    FLUXNET_RAW_FILENAME_PATTERN
#      A filename pattern matching the template of the raw FLUXNET CSV files
#
#      | *Default:* FLX_*_DD_*.csv
#
#    FLUXNET_DEBUG
#      Turn on additional print statements from the Python embedding script
#
#      | *Default:* False
#
# Both of the above Python embedding scripts compute TCI using the ``calc_tci()`` function in METcalcpy.
# See the METcalcpy documentation for more information: https://metcalcpy.readthedocs.io/en/latest/index.html.
#
# For more information on the basic requirements to utilize Python Embedding in METplus, 
# please refer to the MET User’s Guide section on `Python embedding <https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#appendix-f-python-embedding>`_ 

##############################################################################
# User Scripting
# --------------
#
# This use case does not use additional scripts.


##############################################################################
# Running METplus
# ---------------
#
# Pass the use case configuration file to the run_metplus.py script
# along with any user-specific system configuration files if desired::
#
#   run_metplus.py /path/to/METplus/parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf /path/to/user_system.conf
#
# See :ref:`running-metplus` for more information.

##############################################################################
# Expected Output
# ---------------
#
# A successful run will output the following both to the screen and to the logfile::
#
#   INFO: METplus has successfully finished running.
#
# Refer to the value set for **OUTPUT_BASE** to find where the output data was generated.
# Output for the use case will be found in 3 folders (relative to **OUTPUT_BASE**).
# Those folders are:
#
# * PyEmbedIngest
#
# The **OUTPUT_BASE** folder contains all of the TCI output calculated using CESM files in NETCDF format:
#
# * regrid_data_plane_DJF.nc
# * regrid_data_plane_JJA.nc
# * regrid_data_plane_MAM.nc
# * regrid_data_plane_SON.nc
#
# * PointStat
#
# The final folder, PointStat, contains all of the following output from the PointStat call:
#
# * point_stat_DJF_000000L_19790101_000000V_cnt.txt
# * point_stat_DJF_000000L_19790101_000000V_ctc.txt
# * point_stat_DJF_000000L_19790101_000000V_mpr.txt
# * point_stat_DJF_000000L_19790101_000000V.stat
# * point_stat_JJA_000000L_19790101_000000V_cnt.txt
# * point_stat_JJA_000000L_19790101_000000V_ctc.txt
# * point_stat_JJA_000000L_19790101_000000V_mpr.txt
# * point_stat_JJA_000000L_19790101_000000V.stat
# * point_stat_MAM_000000L_19790101_000000V_cnt.txt
# * point_stat_MAM_000000L_19790101_000000V_ctc.txt
# * point_stat_MAM_000000L_19790101_000000V_mpr.txt
# * point_stat_MAM_000000L_19790101_000000V.stat
# * point_stat_SON_000000L_19790101_000000V_cnt.txt
# * point_stat_SON_000000L_19790101_000000V_ctc.txt
# * point_stat_SON_000000L_19790101_000000V_mpr.txt
# * point_stat_SON_000000L_19790101_000000V.stat
#
# * PlotPointObs
#
# The final folder plot_point_obs, contains all of the plots from the PlotPointObs call:
#
# * cesm_fluxnet2015_DJF.ps
# * cesm_fluxnet2015_JJA.ps
# * cesm_fluxnet2015_MAM.ps
# * cesm_fluxnet2015_SON.ps
#

##############################################################################
# Keywords
# --------
#
# .. note::
#
#   * PyEmbedIngestToolUseCase
#   * PointStatToolUseCase
#   * PlotPointObsToolUseCase
#   * PythonEmbeddingFileUseCase
#   * NETCDFFileUseCase
#   * LandSurfaceAppUseCase
#
#   Navigate to the :ref:`quick-search` page to discover other similar use cases.
#
#
#
# sphinx_gallery_thumbnail_path = '_static/land_surface-PointStat_fcstCESM_obsFLUXNET2015_TCI.png'