examples/api_usage_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demonstration of TORRENTpy's API usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook is an example of the use of the TORRENTpy Application Programming Interface (API) to simulate the hydrology and water solutes and particles transport in a catchment discretised in a semi-distributed manner. Example data files that can be used for this example are available in `examples/in/CatchmentSemiDistributedName_OutletName`. Files for a lumped situation are also provided in `examples/in/CatchmentLumpedName_OutletName`. At the end of this tutorial, the user will be able to check their output files against the output files provided in the out folder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step is, of course, to import the Python package `torrentpy`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torrentpy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Create an instance of the Network class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, the first object to instantiate is always from the class `Network`. It will create the node-link network itself, and collect all the information required for these, present in the `*.network`, `*.waterbodies`, and `*.descriptors` files. More information on these files can be found in the file requirements document also available on this repository. A utility is also available to make the creation of the `*.network` file easier.\n",
    "\n",
    "The created network is composed of links (that is to say the drainage basins(s) in the catchment, minimum of one, that is for lumped catchments) connected by nodes (minimum of two, that is for lumped catchments). Links only communicate through nodes. The pdf file in the example input folder shows an example of such a nodes-links network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nw = torrentpy.Network(\n",
    "    catchment='CatchmentSemiDistributedName', \n",
    "    outlet='OutletName', \n",
    "    input_folder='in/CatchmentSemiDistributedName_OutletName/', \n",
    "    output_folder='out/CatchmentSemiDistributedName_OutletName/',\n",
    "    variable_h='q_h2o',\n",
    "    variables_q=['c_no3', 'c_nh4', 'c_dph', 'c_pph', 'c_sed'],\n",
    "    water_quality=True,\n",
    "    verbose=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Create an instance of the TimeFrame class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An instance of the `TimeFrame` class is also required. Be warned, the first four arguments need to be datetime.datetime objects from the package `datetime`. The arguments containing __data__ refer to the time information about the input data, the arguments containing __save__ refer to the desired time attributes of the output data, and the attributes containing __simu__ refer to the internal time information that will be used during simulation (typically the discretisation time step). Indeed, due to numerical instabilities, some models require an hourly simulation time step from daily input series. To overcome this potential problem, TORRENTpy can disaggregate the input series to the required simulation time step. \n",
    "\n",
    "A slicing of the save period is recommended to avoid memory issues during the simulation (when data structures become too big to be stored in the flash memory of the computer). Experience using TORRENTpy shows that a safe place where to start is 2000, which corresponds to 2000 simulation time steps. There is an optimum to be found between run time and memory limits, however because each catchment contains a different number of links, each simulation period length is different, and each user has access to different hardware specifications, no golden rule can be provided. Rather, the user is encouraged to try to increase this argument until a compromise is found for their own context.\n",
    "\n",
    "Finally, a warm-up period is possible with TORRENTpy: it uses the duration in days (let it be N) given and uses the first N days in the input data to warm up the model for the catchment, resulting in reservoirs not being empty when the actual simulation starts. If the user wishes to use real warm-up input data (that is to say actual past information on climate and contamination), it is recommended to include this warm-up period in the save period, and select *a posteriori* the save period excluding the warm-up period."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "\n",
    "tf = torrentpy.TimeFrame(\n",
    "    dt_data_start=datetime.strptime('01/01/2008 09:00:00', '%d/%m/%Y %H:%M:%S'), \n",
    "    dt_data_end=datetime.strptime('31/12/2012 09:00:00', '%d/%m/%Y %H:%M:%S'), \n",
    "    dt_save_start=datetime.strptime('01/01/2009 09:00:00', '%d/%m/%Y %H:%M:%S'), \n",
    "    dt_save_end=datetime.strptime('31/12/2010 09:00:00', '%d/%m/%Y %H:%M:%S'),\n",
    "    data_increment_in_minutes=1440, \n",
    "    save_increment_in_minutes=1440, \n",
    "    simu_increment_in_minutes=60,\n",
    "    expected_simu_slice_length=2000, \n",
    "    warm_up_in_days=365\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Create an instance of the KnowledgeBase class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to get access to the models already available in TORRENTpy (and appending your own ones), an instance of `KnowledgeBase` has to be created. It gives access to all the models built-in TORRENTpy (that should grow with time). The user has the possibility to define and add its own models to KnowledgeBase using the methods `add_catchment_model`, `add_river_model`, and `add_lake_model` (whose use are not demonstrated in this tutorial)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kb = torrentpy.KnowledgeBase()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Create an instance of the DataBase class"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is finally time to bring in the input data. TORRENTpy allows for `csv` files and `netcdf` files to be used by means of the argument `in_format`. Then, __meteorological__ data are required, and __contamination__ data are also required if water quality is set to `True` in the instance of `Network`. TORRENTpy can disaggregate input datasets if required, this is why for each type of data (meteorological and contamination) it is required to distinguish between __cumulative__ and __average__ data. If disaggregation is required (*i.e.* when simulation increment is smaller than data increment in the instance of `TimeFrame`): cumulative data will be disaggregated using a simple assumption of a uniform distribution in the past; average data will be disaggregate by replicating the values in the past."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "db = torrentpy.DataBase(\n",
    "    nw, tf, kb,\n",
    "    in_format='csv',\n",
    "    meteo_cumulative=['rain', 'peva'],\n",
    "    meteo_average=['airt', 'soit'],\n",
    "    contamination_cumulative=['m_no3', 'm_nh4', 'm_p_ino', 'm_p_org'],\n",
    "    contamination_average=[]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Assign models to the links in the Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The last step is to assign models to the links in the network. Assigning models for __catchment__ and __river__ for each link is mandatory, while __lake__ models are only required if lakes are present in the network. Hydrological models are mandatory (___h__) while water quality models (___q__) are only required if water quality is set to `True` in the instance of `Network`. In this example, the rainfall-runoff model `SMART` and the water quality components of the `INCA` are used. These models come as built-in models in TORRENTpy and are therefore already known by the KnowlegdeBase. If user-defined models have been added to the KnowledgeBase, they can be used here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nw.set_links_models(\n",
    "    kb,\n",
    "    catchment_h='SMART', river_h='SMART',\n",
    "    catchment_q='INCA', river_q='INCA'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simulate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Everything is now in place for the simulation to be started, one last instruction and the simulation will be underway. Once again, TORRENTpy gives the choice for the file format for the output data, choose between `csv` and `netcdf`. The user will be informed in the console of the progress of the simulation. When the simulation is over, the output files can be consulted in the output folder defined in the instance of `Network` created earlier. All files have customised extensions intrinsic to TORRENTpy, but they are no more than disguised csv files, open them with any good text editor, or add the extension .csv to open them with any good spreadsheet software to visualise them as tables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nw.simulate(\n",
    "    db, tf, out_format='csv'\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}