.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "generated/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_generated_model_applications_land_surface_PointStat_fcstCESM_obsFLUXNET2015_TCI.py: PointStat: CESM and FLUXNET2015 Terrestrial Coupling Index (TCI) ================================================================ model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf .. GENERATED FROM PYTHON SOURCE LINES 9-13 .. contents:: :depth: 1 :local: :backlinks: none .. GENERATED FROM PYTHON SOURCE LINES 15-30 Scientific Objective -------------------- This use case ingests two CESM (CAM and CLM) files and raw FLUXNET2015 data. The use case calculates the Terrestrial Coupling Index (TCI) from the CESM forecasts and FLUXNET observations. Utilizing Python embedding, this use case taps into a new vital observation dataset and compares it to CESM forecasts of TCI. Finally, it will generate plots of model forecast TCI overlaid with TCI observations at FLUXNET sites. The reference for the Terrestrial Coupling Index calculation is as follows: Dirmeyer, P. A., 2011: The terrestrial segment of soil moisture-climate coupling. *Geophys. Res. Lett.*, **38**, L16702, doi: 10.1029/2011GL048268. .. GENERATED FROM PYTHON SOURCE LINES 32-36 Version Added ------------- METplus version 5.1 .. GENERATED FROM PYTHON SOURCE LINES 38-70 Datasets -------- **Forecast:** CESM 1979-1983 Simulations * Community Land Model (CLM) file * Community Atmosphere Model (CAM) file .. note:: The CESM data used here are in NetCDF format which span several years and have a time dimension. In the event a user has GRIB or other data that does not contain a time dimension, it is advised that they use Python or other tools to create a data object or NetCDF file such that a time dimension is available to use for computing TCI. The user may also need to adjust various settings in the use case configuration file to match their resulting dataset. **Observations:** Raw FLUXNET2015 observations **Data Source:** CESM - NSF NCAR Climate & Global Dynamics (CGD); FLUXNET2015 "SUBSET" Data Product: https://fluxnet.org/data/fluxnet2015-dataset/subset-data-product/ **Location:** All of the input data required for this use case can be found in a sample data tarball. Each use case category will have one or more sample data tarballs. It is only necessary to download the tarball with the use case’s dataset and not the entire collection of sample data. Click here to access the METplus releases page and download sample data for the appropriate release: https://github.com/dtcenter/METplus/releases This tarball should be unpacked into the directory that you will set the value of INPUT_BASE. See :ref:`running-metplus` section for more information. .. GENERATED FROM PYTHON SOURCE LINES 73-85 METplus Components ------------------ This use case utilizes the PyEmbedIngest to read the CESM files and calculate TCI using python embedding and a NETCDF file of the TCI is generated. Then PointStat processes the output of PyEmbedIngest and FLUXNET2015 dataset (using Python embedding), and outputs the requested line types. Then the PlotPointObs tool reads the output of PyEmbedIngest and FLUXNET2015 dataset and produce plots of TCI from CESM and point observations. A custom loop runs through all the pre-defined seasons (DJF, MAM, JJA, SON) and runs PyEmbedIngest, PointStat, and PlotPointObs. .. GENERATED FROM PYTHON SOURCE LINES 87-121 METplus Workflow ---------------- The PyEmbedIngest tool reads 2 CESM files containing Soil Moisture (CLM file) and Sensible Heat Flux (CAM file), each composed of daily forecasts from 1979 to 1983 and calculates TCI and generates a NETCDF file of the TCI. Raw CSV files containing FLUXNET station observations of latent heat flux (LE_F_MDS) and soil water content at the shallowest level (SWC_F_MDS_1) are read using Python embedding, and TCI is computed. **Beginning time (VALID_BEG):** 1979-01-01 at 00z **End time (VALID_END):** 1979-01-01 at 00z **Increment between beginning and end times (VALID_INCREMENT):** None **Sequence of forecast leads to process (LEAD_SEQ):** 0 PointStat is used to compare the two new fields (TCI calculated from CESM dataset and FLUXNET2015). Finally, PlotPointObs is run to plot the CESM TCI overlaying the FLUXNET2015 point observations. .. note:: The CESM forecasts cover a time period prior to the availability of FLUXNET observations. Thus, this use case should be considered a demonstration of the capability to read CESM forecast data, raw FLUXNET observation data, and compute TCI, rather than a bonafide scientific application. The use case is designed to enforce seasonal alignment, but it is not designed to enforce date/time alignment. In this case, the CESM data cover 1979-1983, whereas the sample FLUXNET observations cover varying time ranges depending on the site. .. GENERATED FROM PYTHON SOURCE LINES 123-132 METplus Configuration --------------------- METplus first loads all of the configuration files found in parm/metplus_config, then it loads any configuration files passed to METplus via the command line i.e. parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf .. highlight:: bash .. literalinclude:: ../../../../parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf .. GENERATED FROM PYTHON SOURCE LINES 134-150 MET Configuration ----------------- METplus sets environment variables based on user settings in the METplus configuration file. See :ref:`How METplus controls MET config file settings` for more details. **YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!** If there is a setting in the MET configuration file that is currently not supported by METplus you’d like to control, please refer to: :ref:`Overriding Unsupported MET config file settings` .. dropdown:: PointStatConfig_wrapped .. highlight:: bash .. literalinclude:: ../../../../parm/met_config/PointStatConfig_wrapped .. GENERATED FROM PYTHON SOURCE LINES 152-258 Python Embedding ---------------- This use case requires the following Python dependencies: * Xarray * Pandas * METcalcpy 3.0.0+ This use case uses a Python embedding script to read both the forecast and observation data, in order to compute TCI, which is the diagnostic that is being verified by MET using PointStat. The forecast data (CESM in this case) is read using: .. dropdown:: parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/tci_fcst.py .. highlight:: python .. literalinclude:: ../../../../parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/tci_fcst.py The user can control all arguments to this script via the METplus use case configuration file using the following config entries: .. glossary:: TCI_FCST_LHFLUX_VAR The latent heat flux variable from the forecast data to use for computing TCI | *Default:* LHFLX TCI_FCST_SOILM_VAR The soil moisture variable from the forecast data to use for computing TCI | *Default:* SOILWATER_10CM TCI_FCST_LHFLUX_FILE_PATH The path to the model netcdf file that contains the latent heat flux field TCI_FCST_SOILM_FILE_PATH The path to the model netcdf file that contains the soil moisture field The raw FLUXNET2015 SUBSET data are read using: .. dropdown:: parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/fluxnet2015_tci.py .. highlight:: python .. literalinclude:: ../../../../parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI/fluxnet2015_tci.py The user can control all command line arguments to this script via METplus config entries: .. glossary:: FLUXNET_DATA_DIR The directory containing raw FLUXNET CSV files FLUXNET_LAT_HEAT_VAR The FLUXNET surface latent heat flux variable name to use for computing TCI | *Default:* LE_F_MDS FLUXNET_SOIL_MOIST_VAR The FLUXNET soil moisture variable name to use for computing TCI | *Default:* SWC_F_MDS_1 FLUXNET_OBS_METADATA_FILE The absolute path to the fluxnetstations.csv metadata file included with the use case and for data filtering options, via METplus config entries: .. glossary:: FLUXNET_SKIP_LEAP_DAYS Skip FLUXNET observations from 29 February days | *Default:* True FLUXNET_HIGHRES_QC_THRESH The fraction of higher temporal resolution FLUXNET data required to have passed quality control in order to use the daily data. | *Default:* 0.8 FLUXNET_MIN_DAYS_PER_SEASON The minimum number of days to include in individual seasons at each site | *Default:* 1 FLUXNET_MIN_DAYS_PER_SITE The minimum number of days for all seasons at each site | *Default:* 10 FLUXNET_RAW_FILENAME_PATTERN A filename pattern matching the template of the raw FLUXNET CSV files | *Default:* FLX_*_DD_*.csv FLUXNET_DEBUG Turn on additional print statements from the Python embedding script | *Default:* False Both of the above Python embedding scripts compute TCI using the ``calc_tci()`` function in METcalcpy. See the METcalcpy documentation for more information: https://metcalcpy.readthedocs.io/en/latest/index.html. For more information on the basic requirements to utilize Python Embedding in METplus, please refer to the MET User’s Guide section on `Python embedding `_ .. GENERATED FROM PYTHON SOURCE LINES 260-264 User Scripting -------------- This use case does not use additional scripts. .. GENERATED FROM PYTHON SOURCE LINES 267-276 Running METplus --------------- Pass the use case configuration file to the run_metplus.py script along with any user-specific system configuration files if desired:: run_metplus.py /path/to/METplus/parm/use_cases/model_applications/land_surface/PointStat_fcstCESM_obsFLUXNET2015_TCI.conf /path/to/user_system.conf See :ref:`running-metplus` for more information. .. GENERATED FROM PYTHON SOURCE LINES 278-328 Expected Output --------------- A successful run will output the following both to the screen and to the logfile:: INFO: METplus has successfully finished running. Refer to the value set for **OUTPUT_BASE** to find where the output data was generated. Output for the use case will be found in 3 folders (relative to **OUTPUT_BASE**). Those folders are: * PyEmbedIngest The **OUTPUT_BASE** folder contains all of the TCI output calculated using CESM files in NETCDF format: * regrid_data_plane_DJF.nc * regrid_data_plane_JJA.nc * regrid_data_plane_MAM.nc * regrid_data_plane_SON.nc * PointStat The final folder, PointStat, contains all of the following output from the PointStat call: * point_stat_DJF_000000L_19790101_000000V_cnt.txt * point_stat_DJF_000000L_19790101_000000V_ctc.txt * point_stat_DJF_000000L_19790101_000000V_mpr.txt * point_stat_DJF_000000L_19790101_000000V.stat * point_stat_JJA_000000L_19790101_000000V_cnt.txt * point_stat_JJA_000000L_19790101_000000V_ctc.txt * point_stat_JJA_000000L_19790101_000000V_mpr.txt * point_stat_JJA_000000L_19790101_000000V.stat * point_stat_MAM_000000L_19790101_000000V_cnt.txt * point_stat_MAM_000000L_19790101_000000V_ctc.txt * point_stat_MAM_000000L_19790101_000000V_mpr.txt * point_stat_MAM_000000L_19790101_000000V.stat * point_stat_SON_000000L_19790101_000000V_cnt.txt * point_stat_SON_000000L_19790101_000000V_ctc.txt * point_stat_SON_000000L_19790101_000000V_mpr.txt * point_stat_SON_000000L_19790101_000000V.stat * PlotPointObs The final folder plot_point_obs, contains all of the plots from the PlotPointObs call: * cesm_fluxnet2015_DJF.ps * cesm_fluxnet2015_JJA.ps * cesm_fluxnet2015_MAM.ps * cesm_fluxnet2015_SON.ps .. GENERATED FROM PYTHON SOURCE LINES 330-347 Keywords -------- .. note:: * PyEmbedIngestToolUseCase * PointStatToolUseCase * PlotPointObsToolUseCase * PythonEmbeddingFileUseCase * NETCDFFileUseCase * LandSurfaceAppUseCase Navigate to the :ref:`quick-search` page to discover other similar use cases. sphinx_gallery_thumbnail_path = '_static/land_surface-PointStat_fcstCESM_obsFLUXNET2015_TCI.png' .. _sphx_glr_download_generated_model_applications_land_surface_PointStat_fcstCESM_obsFLUXNET2015_TCI.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: PointStat_fcstCESM_obsFLUXNET2015_TCI.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: PointStat_fcstCESM_obsFLUXNET2015_TCI.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: PointStat_fcstCESM_obsFLUXNET2015_TCI.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_