Wednesday, December 7, 2016

A list of papers estimating the water budget at various scales


Since almost couple of decades I am trying to develop tools that evaluate the surface water budget components, and I look at the closure of the budget equation.  The outcomes of this research are our models GEOtop and our system JGrass-NewAGE, and some applications are listed below. My impression was that many researchers are talking of the water budget closure, since many actually have the knowledge and the tools for estimating the budget, but less are really doing it. We wrote  (more or less)  it in the introduction of one of our paper and we were asked by the reviewers to be more precise. The list below, certainly to be cleaned, improved and enriched, says that there are effectively many papers (out of around a hundred we inspected) that do it. They are distributed according to four  threads. 
  • The one of “global and continental hydrology” where the water budget of the whole earth or of the largest river basins is studied. The methods used are remote sensing data, global circulation models, large-scale hydrological models.
  • One is (but the focus is more often on evaporation) based on the use of Budyko curves, at various scales. 
  • The  use models plus in-situ data, with various levels of simplification, usually from few kilometers to thousands of kilometers scales. Models are process-based (like  CATHY,  GEOtop  or ParFlow, to cite three of them)  or more conceptualised (as our JGrass-NewAGE). Data are the most various, depending on the spatial scale of the application and the type of model. Process-based models use more data (which is a richness or a weakness, depending on the point of view), while conceptual models use less data. Larger scale applications require a coarse graining of the data set and, obviously, a limitation in the description of spatial heterogeneity. 
  • Finally there are fully experimental papers, especially in forest and agricultural areas, with accurate measurements, for some specific plant stand, or even single trees.

In the selection of the paper below, I searched for the water budget equation, with all of the terms, its minimal expression being:

$\frac{\nabla S}{\nabla t} = P - ET - R$

where $S$ is the soil/groundwater storage, $P$ precipitation rate, $ET$ evapotranspiration, $R$ runoff. Various papers present a more articulated baudget, but certainly I did not listed the paper that not deals with the equation. Many papers, having “water budget” in the title, actually deal with evapotranspiration and were excluded. As Praveen Kumar (GS) argued to me, all good models preserve mass: but they often deal only with a part of the budget, and/or their authors are  concerned with other specific topic. Also these papers (and some really very  interesting were excluded).  Finally, please find below the list. A different version of the same list (and its LaTeX editable version)  with some comments about the spatial and temporal scales of the budget and some further information can be found here ( where references can be sorted).

References

Adelana, S. M., Dresel, P. E., Hekmeijer, P., Zydor, H., Webb, J. A., Reynolds, M., & Ryan, M. (2014). A comparison of streamflow, salt and water balances in adjacent farmland and forest catchments in south-western Victoria, Australia. Hydrological Processes, 29(6), 1630–1643. http://doi.org/10.1002/hyp.10281

Arnold, J. C., & Allen, P. M. (2016). Estimating hydrologic budgets for three Illinois watersheds. Journal of Hydrology, 176, 57–77.
Azarderakhsh M,  Rossow WB, Papa F,  Norouzi H, Khanbilvardi R. Diagnosing water variations within the Amazon basin using satellite data. Journal of Geophysical Research: Atmospheres 116 (2011).

Batelaan, O., & De Smedt, F. (2007). GIS-based recharge estimation by coupling surface–subsurface water balances. Journal of Hydrology, 337(3-4), 337–355. http://doi.org/10.1016/j.jhydrol.2007.02.001

Bertoldi, G., Rigon, R., & OVER, T. M. (2005). Impact of watershed geomorphic characteristics on the energy and water budgets. Journal of Hydrometeorology, 1–29.

Brye, K. R., Norman, J. M., Bundy, L. G., & Gower, S. T. (2000). Water-Budget evaluation of Prairie and Maize Ecosystems, 64, 715–724.

Chen J,  Lee C, Tian-Chyi Yeh J, Yu J. A Water Budget Model for the Yun-Lin Plain, Taiwan. Water Resources Management 19, 483–504 (2005).

Claessens, L., Hopkinson, C., Rastetter, E., & Vallino, J. (2006). Effect of historical changes in land use and climate on the water budget of an urbanizing watershed. Water Resources Research, 42(3), n/a–n/a. http://doi.org/10.1029/2005WR004131

Cook, P. G., Hatton, T. J., Pidsley, D., Herczeg, A. L., Held, A., O'Grady, A., & Eamus, D. (2016). Water balance of a tropical woodland ecosystem, Northern Australia: a combination of micro-meteorological, soil physical and groundwater chemical approaches. Journal of Hydrology, 210, 161–177.

Dages C, Voltz M,  Bsaibes A,  Prévot L,  Huttel O,  Louchart X, Garnier F, S Negro. Estimating the role of a ditch network in groundwater recharge in a Mediterranean catchment using a water balance approach. Journal of Hydrology 375, 498–512 (2009).

Dean, J. F., Webb, J. A., Jacobsen, G. E., Chisari, R., & Dresel, P. E. (2015). A groundwater recharge perspective on locating tree plantations within low-rainfall catchments to limit water resource losses. Hydrology and Earth System Sciences, 19(2), 1107–1123. http://doi.org/10.5194/hess-19-1107-2015

Fang, Z., H.R. Bogena, S. Kollet, J. Koch and H. Vereecken (2015): Spatio-temporal validation of long-term 3D hydrological simulations of a forested catchment using empirical orthogonal functions and wavelet coherence analysis. J. Hydrol. 529: 1754-1767, doi:10.1016/j.jhydrol.2015.08.011.

Fleischbein, K., Wilcke, W., Valarezo, C., Zech, W., & Knoblich, K. (2006). Water budgets of three small catchments under montane forest in Ecuador: experimental and modelling approach. Hydrological Processes, 20(12), 2491–2507. http://doi.org/10.1002/hyp.6212

Graf, A., Bogena, H. R., Drüe, C., Hardelauf, H., Pütz, T., Heinemann, G., & Vereecken, H. (2014). Spatiotemporal relations between water budget components and soil water content in a forested tributary catchment. Water Resources Research, 50(6), 4837–4857. http://doi.org/10.1002/2013WR014516

Harder, S. V., Amatya, D. M., Callahan, T. J., Trettin, C. C., & Hakkila, J. (2007). Hydrology and water budget for a Forested atlantic coastal plain watershed, South Carolina. Journal of the American Water Resources Association, 43(7), 563–575.

Hentschel, R., Bittner, S., Janott, M., Biernath, C., Holst, J., Ferrio, J. P., et al. (2013). Simulation of stand transpiration based on a xylem water flow model for individual trees. Agricultural and Forest Meteorology, 182-183, 31–42. http://doi.org/10.1016/j.agrformet.2013.08.002

Herron, N., & Wilson, C. (2001). A water balance approach to assessing the hydrologic buffering potential of an alluvial fan. Water Resources Res., 37(2), 341–351.

Hingerl L, Kunstmann H, Wagner S, Mauder M, Bliefernicht J, Rigon R. Spatiotemporal variability of water and energy fluxes - A case study for a meso-scale catchment in pre-alpine environment. Hydrological Processes 1–20 (2016).

Högström, U. (1968). Studies on the water balance of a small natural catchment area in southern Sweden, XX(4), 623–631.


Huntington, J.L., and Niswonger, R.G., 2012, Role of surface-water and groundwater interactions on projected summertime streamflow in snow dominated regions: An integrated modeling approach : Water Resources Research, vol. 48, W11524, doi: 10.1029/2012WR012319.

Hutley, L. B., Doley, D., Yates, D. J., & Boonsaner, A. (1997). Water Balance of an Australian Subtropical Rainforest at Altitude: the Ecological and Physiological Significance of Intercepted Cloud and Fog. Australian Journal of Botany, 45(2), 311–20. http://doi.org/10.1071/BT96014

Jothityangkoon, C., Sivapalan, M., & Farmer, D. L. (2001). Process controls of water balance variability in a large semi-arid catchment: downward approach to hydrological model development. Journal of Hydrology, 254, 174–198.

Kochendorfer, J. P., & Ramirez, J. A. (2010). Modeling the monthly mean soil-water balance with a statistical-dynamical ecohydrology model as coupled to a two-component canopy model. Hydrology and Earth System Sciences, 14(10), 2099–2120. http://doi.org/10.5194/hess-14-2099-2010

Landerer, F. W., Dickey, J. O., & Güntner, A. (2010a). Terrestrial water budget of the Eurasian pan-Arctic from GRACE satellite measurements during 2003–2009. Journal of Geophysical Research, 115(D23), D23115–14. http://doi.org/10.1029/2010JD014584

Lewis, C., Albertson, J., Zi, T., Xu, X., & Kiely, G. (2012). How does afforestation affect the hydrology of a blanket peatland? A modelling study. Hydrological Processes, 27(25), 3577–3588. http://doi.org/10.1002/hyp.9486
Lewis D, Singer MJ, Dahlgren RA, Tate KW. Hydrology in a California oak woodland watershed: a 17-year study. Journal of Hydrology 240, 106–117 (2000).

Lorenz, C., & Kunstmann, H. (2012). The Hydrological Cycle in Three State-of-the-Art Reanalyses: Intercomparison and Performance Analysis. Journal of Hydrometeorology, 13(5), 1397–1420. http://doi.org/10.1175/JHM-D-11-088.1

Luxmoore, R. J. (1983). Water Budget of an Eastern Deciduous Forest Stand. Soil Science Soc. Am. J., 47, 785–791. 

Marengo, J. A. (2004). Characteristics and spatio-temporal variability of the Amazon River Basin Water Budget. Climate Dynamics, 24(1), 11–22. http://doi.org/10.1007/s00382-004-0461-6

Maxwell, R. M., & Condon, L. (2016). Connections between groundwater flow and transpiration partitioning. Science, 353(6297), 377–379. http://doi.org/10.1126/science.aaf8589

Mitchell, V. G., McMahon, T. A., & Mein, R. G. (2003). Components of the total Water Balance of an urban Catchment. Environmental Management, 32(6), 735–746.

Munier S, Aires F, Schlaffer S, Prigent C, Papa F, Maisongrande P, Pan M. Combining data sets of satellite-retrieved products for basin-scale water balance study: 2. Evaluation on the Mississippi Basin and closure correction model. Journal of Geophysical Research: Atmospheres 119 (2014).


Niedzialek, J.M., and F.L. Ogden, 2012, First-order catchment mass balance during the wet season in the Panama Canal watershed, J. Hydrol. doi: 10.1016/j.jhydrol.2010.07.044.


Obojes, N., Bahn, M., Tasser, E., Walde, J., Inauen, N., Hiltbrunner, E., et al. (2014). Vegetation effects on the water balance of mountain grasslands depend on climatic conditions. Ecohydrology, 8(4), 552–569. http://doi.org/10.1002/eco.1524

Ogden, F.L., T.D. Crouch, R.F. Stallard, and J.S. Hall, 2013. Effect of land cover and use on dry season river runoff and peak runoff in the seasonal tropics of central Panama, Water Resour. Res. 49(12):8443-8462, doi:10.1002/2013WR013956.

Oliveira PTS, Nearing MA, Moran MS, Goodrich DC, Wendland E, Gupta HV. Trends in water balance components across the Brazilian Cerrado. Water Resources Research 50, 7100–7114 (2014).

Pan, M., & Wood, E. F. (2006). Data Assimilation for Estimating the Terrestrial Water Budget Using a Constrained Ensemble Kalman Filter. Journal of Hydrometeorology, 7, 534–547.

Pan X, Helgason W, Ireson A, Wheater H. Field-scale water balance closure in seasonally frozen conditions. Hydrology and Earth System Sciences Discussions 2016, 1–37 (2016)

Qu, W., H. R. Bogena, J. A. Huisman, M. Schmidt, R. Kunkel, A. Weuthen, B. Schilling, J. Sorg and H. Vereecken (2016): The integrated water balance and soil data set of the Rollesbroich hydrological observatory. Earth Syst. Sci. Data, 8: 517–529, doi:10.5194/essd-8-1-2016.

Sahoo, A. K., Pan, M., Troy, T. J., Vinukollu, R. K., Sheffield, J., & Wood, E. F. (2011). Reconciling the global terrestrial water budget using satellite remote sensing. Remote Sensing of Environment, 115(8), 1850–1865. http://doi.org/10.1016/j.rse.2011.03.009


Schaake, J., Koren, V., Duan, Q., Mitchell, K., & Chen, F. (2007). Simple water balance model for estimating runoff at different spatial and temporal scales. Journal of Geophysical Research, 101(D3), 7461–7475.

Schreiner-McGraw, A. P., Vivoni, E. R., Mascaro, G., & Franz, T. E. (2016). Closing the water balance with cosmic-ray soil moisture measurements and assessing their relation to evapotranspiration in two semiarid watersheds. Hydrology and Earth System Sciences, 20(1), 329–345. http://doi.org/10.5194/hess-20-329-2016

Scott, R. L. (2010). Using watershed water balance to evaluate the accuracy of eddy covariance evaporation measurements for three semiarid ecosystems. Agricultural and Forest Meteorology, 150(2), 219–225. http://doi.org/10.1016/j.agrformet.2009.11.002

Sheffield, J., Ferguson, C. R., Troy, T. J., Wood, E. F., & McCabe, M. F. (2009). Closing the terrestrial water budget from satellite remote sensing. Geophysical Research Letters, 36(7), n/a–n/a. http://doi.org/10.1029/2009GL037338

Silberstein, R. P., & Sivapalan, M. (1995). MODELLING VEGETATION HETEROGENEITY EFFECTS ON TERRESTRIAL WATER AND ENERGY BALANCES. Environmental International, 21(5), 477–484.

Sottocornola, M. (2007, July). Four years of observations of carbon dioxide fluxes, water and energy budgets, and vegetation patterns in an Irish Atlantic blanket bog. Ph.D. Thesis (Chapter 6), (G. Kiely, Ed.).

Su, F., & Lettenmaier, D. P. (2009). Estimation of the Surface Water Budget of the La Plata Basin. Journal of Hydrometeorology, 10(4), 981–998. http://doi.org/10.1175/2009JHM1100.1

Tomasella, J., Hodnett, M. G., Cuartas, L. A., Nobre, A. D., Waterloo, M. J., & Oliveira, S. M. (2008). The water balance of an Amazonian micro-catchment: the effect of interannual variability of rainfall on hydrological behaviour. Hydrological Processes, 22(13), 2133–2147. http://doi.org/10.1002/hyp.6813

Vertessy, R. A., Watson, F. G. R., & Sullivan, S. K. (2001). Factors determining relations between stand age and catchment water balance in mountain ash forests. Forest Ecology and Management, 143, 13–26.

Wagner, S., Kunstmann, H., Bárdossy, A., Conrad, C., & Colditz, R. R. (2009). Water balance estimation of a poorly gauged catchment in West Africa using dynamically downscaled meteorological fields and remote sensing information. Physics and Chemistry of the Earth, Parts a/B/C, 34(4-5), 225–235. http://doi.org/10.1016/j.pce.2008.04.002

Wang H, Guan H, Gutiérrez-Jurado HA, Simmons CT. Examination of water budget using satellite products over Australia. Journal of Hydrology 511, 546–554 (2014).

Whitehead, D., & Kelliher, F. M. (1991). Modeling the water balancevof a small Pinus radiuta catchment. Tree Physiology, 9, 17–33.

Wilson, K. B., Hanson, P. J., Mulholland, P. J., Baldocchi, D. D., & Wullschleger, S. D. (2001). A comparison of methods for determining forest evapotranspiration and its components: sap-flow, soil water budget, eddy covariance and catchment water balance. Agricoltural and Forest Meteorology, 106, 153–168.

Yang, Dawen, Sun, F., Liu, Z., Cong, Z., Ni, G., & Lei, Z. (2007). Analyzing spatial and temporal variability of annual water-energy balance in nonhumid regions of China using the Budyko hypothesis. Water Resources Research, 43(4), n/a–n/a. http://doi.org/10.1029/2006WR005224

Yao, Y., Liang, S., Xie, X., Cheng, J., Jia, K., Li, Y., & Liu, R. (2014). Estimation of the terrestrial water budget over northern China by merging multiple datasets. Journal of Hydrology, 519, 50–68. http://doi.org/10.1016/j.jhydrol.2014.06.046

Yoshiyukiishii YK, Nakamura R., Water balance of a snowy watershed in Hokkaido, Japan. Northern Research Basins Water Balance 13 (2004).

Zhang, L., Potter, N., Hickel, K., Zhang, Y., & Shao, Q. (2008a). Water balance modeling over variable time scales based on the Budyko framework – Model development and testing. Journal of Hydrology, 360(1-4), 117–131. http://doi.org/10.1016/j.jhydrol.2008.07.021

Tuesday, November 29, 2016

Water, energy and carbon balance of a peatland catchment in the Alps

When I started the enterprise called GEOtop, it was at this kind of studies that I was thinking, where experimenters meet theoreticians and/or modeler. In this case, we are studying a small peatland on a plateu on top of Monte Bondone, one the mountains close to Trento. It is usually overlooked in favour of the great dolomites we have all around here. However, it has its beauty, and I can tell you that it is a pleasure to walk over there and see its flora, in summer, or skiing there in winter.
This paper deals with the water budget of the biotope of the plateau, and its carbon cycle. The paper used measurement campaign, laboratory analysis and accurate modelling that push to the limit of GEOtop capabilities.

The paper, entitled: "Water, energy and carbon balance of a peatland catchment in the Alps "by Jeroen Pullens et al.,  has this abstract: " Over millennia, peatlands have stored around 30% of the global soil organic carbon.  Peat is formed and accumulated due to the slow decomposition rate of organic matter in the waterlogged, anaerobic soils. Therefore, the understanding of the water cycle of peatlands is important in evaluating the functioning of peatlands. To be able to study  these dynamics, the process-based hydrological model GEOtop and an appropriate set of in situ measurements were used. They were functional to simulate 4 years (2012- 2015) of the water and energy dynamics of an alpine catchment in Italy, which included a peatland. The modelled energy fluxes are comparable to the fluxes  measured by the on-site eddy covariance tower. The modelled water cycle was used  to quantify the loss of dissolved organic carbon (DOC) and to calculate the carbon balance of the peatland. The model outcomes showed an overall good fit with the  measurements during the snow-free period. During snow cover, the model had difficulties simulating the soil temperature due to insulation by the snow. Based on the measured DOC and the modelled discharge, the DOC adds another source of carbon, since the presented peatland is already acting as a carbon source based on  carbon fluxes. The total amount of loss of DOC (10.2 (± 3.8) g C m-2 yr-1) is comparable to other peatlands. In total, the peatland had a carbon balance (CO2, CH4 fluxes and DOC losses combined) of 112.3, 273.8, 190.8 and 95.3 g C m-2 yr-1 for 2012, 2013, 2014 and 2015 respectively. 
"

You can find the pdf of the manuscript by clicking on the figure above.

Friday, November 25, 2016

Python resources for Hydrologists

Python is a modern object oriented language. Occasionally I wrote about it in my posts, also for remarking that I went in a different direction. However, I cannot deny the evidence that more and more people are choosing it, and there are good reasons, as their language of choice for doing research and hydrological applications.

Motivations for Python use, over other choices, can be found in this blog post or in this paper.

To understand how to start you can follow Python programming for hydrology students that starts with indicating how to install it.

There are a lot of resources to start with python, but for hydrologists, a recommendation is to use:


with a preference for the first one. Soil Physics with Python: Transport in the Soil-Plant-Atmosphere System, by Bittelli et al, is al, is a book on soil science which is quite appealing (as seen the TOC): the kindle version cost reasonably but Ido not have it. Python programs are available here.

An overview of Scientific libraries and softwares of general use in Python is given at scipy.org, from which I extract these links to documentation:
Python is especially use as a glue for existing program, either written in C or FORTRAN. We have the cases of
  • CFM is a programming library to create hydrological models. Although written in C++, it has a Python interface 
  • ESMF regridding has been interfaced with Python ESMPy
  • GRASS GIS has been interfaced with Python
  • Python is also interfaced to gvSIG, as you can see here
  • HPGL a High Performance Geostatistics Library. Written in C++ is glued together by Python 
  • MODFLOW the groundwater model is interfaced by FloPy. Documentation and other information is here
  • PcRaster - Is a collection of software targeted at the development and deployment of spatio-temporal environmental models. It has a python interface which is constantly being enhanced. 
  • OpenHydrology is a library of open source hydrological software written in Python to operate as packages under an umbrella interface 
  • PyHSPF Python extensions to the Hydrological Simulation Program in Fortran (HSPF
  • PyQGIS: A Python interface to QGIS 
  • RhessysWorkflow  RHESSysWorkflows provides Python scripts for building RHESSys models. Other Pythonic material on RHESSys can be found here.
  • UWHydro tools for connecting University of Washington hydrological models, and, in particolar, the VIC driver PythonDriver

In the reign of hydrologic applications entirely written in Python, we remind:

  • AMBHAS - hydrological library in Python 
  • ANUGA 2 - package for modelling dam breaks, riverine flooding, storm-surge or tsunamis. In Python and C. 
  • EcoHydrolib provides a series of Python scripts for performing ecohydrology data preparation workflows. 
  • evaplib: Python library containing functions for calculation of evaporation rates. Functions include Penman open water evaporation, Makkink reference evaporation, Priestley Taylor evaporation Penman Monteith (1965) evaporation and FAO's Penman Monteith ET0 reference evaporation for short, well-watered grass. In addition there is a function to calculate the sensible heat flux from temperature fluctuation measurements. View documentation of evaplib module functions. Module documentation is also available as a PDF document. Author: M.J. Waterloo. 
  • A GLUE, Generalised Likelihood Uncertainty Estimation (GLUE) developed by Framework Joost Delsman, at Deltares, 2011 
  • Groundwater flow modelling manual for Python written by Vincent post 
  • Hydro-conductor: A set of Python scripts and modules written to couple a hydrologic model with a regional glacier model
  • ODMToolsPython and ODMTools ODMTools is a python application for managing observational data using the Observations Data Model. ODMTools allows you to query, visualize, and edit data stored in an Observations Data Model (ODM) database.ODMTools was originally developed as part of the CUAHSI Hydrologic Information System. YOu can find a presentation about here
  • PyETo is a package for calculating reference/potential evapotranspiration (ETo). 
  • Python script for rectangular Piper plot (version December 2014): Python script for plotting chemical data in a rectangular python plot (see image) according to Ray and Mukherjee (2008) Groundwater 46(6): 893-896. Also download the example data file watersamples.txt. Author: B.M. van Breukelen. 
  • Python script for multiple Stiff plots (version June 2011): Python script for preparing multiple Stiff diagrammes (see image). Also download the example data file watersamples.txt. Author: B.M. van Breukelen. 
  • Haran Kiruba tools for hydrology Not really clear what he does. 
  • USEPA site contains various python (and other languages) tools, including an interface to Epanet and SWMM (a connection to swmmtoolbox is also avilable here). 
  • Also USGS has its python tools
  • sMAP 2.0 is a tutorial will cover how to retrieve data from a sMAP archiver using Python. 
  • ulmo clean, simple and fast access to public hydrology and climatology data 


Specific hydrological Hydrological Models are enumerated below.

  • EXP-HYDRO Model is a catchment scale hydrological model that operates at a daily time-step. 
  • Landlab Landlab is a python-based modeling environment that allows scientists and students to build numerical landscape models. Designed for disciplines that quantify earth surface dynamics such as geomorphology, hydrology, glaciology, and stratigraphy, it can also be used in related fields. 
  • LHMP - lumped hydrological models playground - tiny docker container with complete environment for predictions.
  • PyCatch is a component based hydrological model of catchments built within the PCRaster Python framework. The code is here. A related paper, here
  • PyTOPKAPI is a BSD licensed Python library implementing the TOPKAPI Hydrological model (Liu and Todini, 2002). The model is a physically-based and fully distributed hydrological model, which has already been successfully applied in several countries around the world 
  • SPHY. See for details on model and publications (HESS, Nature, etc) here. Just recently a new paper on climate change and mountain hydrology in PloS came out, using SPHY model, more info here.
  • Topoflow a python hydrologic model by Scott Peckham 
  • WOFpy is an implementation of CUAHSI's Water One Flow service stack in python 
  • wflow is a distributed hydrological model platform that currently includes two models: the wflow_sbm model (derived from the topog_sbm soil concept) and the wflow_hbv model which is a distributed version of the HBV model. This is actully part of a larger Deltares project called OpenStream

GIS capabilities are also present:


Also tools to deal with Meteorology:

  • meteolib: Python library containing meteorological functions for calculation of atmospheric vapour pressures, air density, latent heat of vapourisation, heat capacity at constant pressure, psychrometric constant, day length, extraterrestrial radiation input, potential temperature and wind vector. The documentation for this module is presented at here (meteolib module functions web site). Functions to convert event-based data records to equidistant time-spaced records (event2time) and to convert date values to day-of-year values (date2doy) are now in a separate meteo_util module. Documentation is presented here (meteo_util module functions web site). Module documentation is also available as a PDF document. Author: M.J. Waterloo. 
  • MetPy is An Open Source Python Toolkit for Meteorology 
  • Melodist (MEteoroLOgical observation time series DISaggregation Tool) is an open-source software package written in Python for temporally downscaling (disaggregating) daily meteorological time series to hourly data. It is documented in a GMD paper by Forster et al., 2016. 
  • Various resources for meteorology can be found in the pyaos blog
Statistical and data analysis tools are abundant
  • CUAHSI time series viewer
  • The basic cheatshit
  • NetCDF file operations are available here. However, there is also txt2netcdf which containsvarious Python functions for importing text into NetCDF data files (creating files, adding variables, listing structure, etc.), developed by Ko van Huissteden. 
  • Pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language. (A short tutorial is also here
  • Extreme distribution (from scipy.stats) is here
  • An example of use of Pandas for analysing time series 
Visualisation is well served
  • ggplot is a plotting system for Python based on R's ggplot2 and the Grammar of Graphics. It is built for making profressional looking, plots quickly with minimal code. 
  • An impressive tour of Python possibilities in this field is given by “Regress to Impress” 
  • VisTrails an open-source scientific workflow and provenance management system that supports data exploration and visualization. Its website is here
  • uvcmetrics metrics aka diagnostics for comparing models with observations or each other. This is part of the Uv-CDAT website which contains also other visualisation tools. 

A final comment

I am actually impressed by the quality of the contributions in Python. I think there is not anymore reason to use commercial program like Matlab in Universities. What Matlab does, also Python does. Compared to R, it has a much more clear sintax and is, certainly, a better language. So I would suggest to use it with students (or R). I like R but I never built program on it, because its object orientation is really poor. Python is better and, as it is known its syntax is clean. Python is great to link FORTRAN and C/C++ native libraries, so actually many uses it to assembled libraries they wrote in those more performant languages.
As you know, however, my group uses Java as its principal programming language. Java, in comparison with Python, is less immediate and more verbose, but it allows to build many framework to work with, and is usually faster than Python. Probably because, Java is supported by magnificent building tools (Maven, Gradle) that allow to manage large projects in a way that probably cannot be done in Python.

Monday, November 21, 2016

Zenodo and Joss

Here I am robbing from Living in an Ivory Basement blog. Titus Brown (GS) introduces two interesting tools. Maybe the word “tools” is an inappropriateas description.

Zenodo is a way to archive software products, under the idea that GitHUB is not an archival:

“A common point of concern, however, is that GitHub repositories are not archival. That is, you can modify, rewrite, delete, or otherwise irreversibly mess with the contents of a git repository. And, of course, GitHub could go the way of Sourceforge and Google Code at any point. … But! Never fear! The folk at Zenodo and Mozilla Science Lab (in collaboration with Figshare) have solutions for you! … Zenodo is an open-access archive that is recommended by Peter Suber (as is Figshare). … Items will be retained for the lifetime of the repository. This is currently the lifetime of the host laboratory CERN, which currently has an experimental programme defined for the next 20 years at least.“

For more information, nothing better than reading the original blog post.

JOSS stands for Journal of Open Source Software and it promises to be a vehicle to publish software, not what software is about, or a description of the software and its peculiarities, but of the software, which seems to go through a process of peer review, extremely useful and new. You can find the Joss post here.

Sunday, November 13, 2016

The Soil Water Retention Curves

When dealing with soils you are forced to implement mass conservation dependent on two variables, the dimensionless water content, usually named $\theta$ and suction, $\psi$, i.e. The energy contained in a volume of soil per unit mass. Therefore, to solve the budget, you need (at least) to get a new relationship which connects them.  This relation is called soil water retention curve. The plural in the title means that there are many. At least one for any soil type. 

In fact,  the  relationship, and precisely $\theta(\psi)$, is dependent on soil types and structure (and some other factor probably, like temperature, organic content etc). It is a statistical quantity, which averages the behavior of many pores, and an ensamble of water injecting/extracting possibilities.  
The figure below from Lu (GS) and Godt (GS) book (2013) is a clear visualisation of the problem.

The same Ning Lu, in a recent paper (2015) tried to disentangle the various forces acting on water when in pores, and obtained what is shown below.

As expected, the forces acting are not all of the same type, at varying suction values. At very high suction, adsorption forces act in which single water molecules adhere to soils. When more layers of water molecule add, water constitute  thermodynamics compounds, whose equilibrium is globally determined in between adhesion forces, bulk water weights, surface of water and air gas interactions, and which is usually known as capillarity.  
Laws governing capillarity are described by Young-Laplace and  Kelvin laws.  Some insight of the therodynamics of these phenomena (an excellent explanation, indeed) can be found in the first pages of Steudle (2001) review about plant-root suction. 
At this stage liquid water seems to, constitute a disconnected phase, while air gas is continuous inside the medium pores.  
Increasing the water content water becomes a continuos medium and usual hydrodynamics laws become valid.  A recent review of parameterizations of the soil water retention curves (not particularly deep or brilliant though) is given by Too et al. (2014) that cites other older reviews.

When pressure increase, however, we can have two effect which partially depends on how wetting happens. If wetting happens through some sort of flooding then air can stay trapped in pores and decrease the space available for water. The net effect is  associable to a decrease of porosity. However, when water fills all the space (i.e. the soil is saturated) the soil matrix cannot be considered anymore rigid. 

Assume it would be rigid. Then water content could not increase, any pressure applied to the saturated soil would transmit instantaneously through the water volume and water would be expelled where pressure is not applied or there is less pressure in a sort of piston flow.  
Instead, because the medium is not rigid, any pressure is transmitted with a certain speed, and pressure waves can be measured. This fact implies that after saturation, the system behaves as porosity increases and, at the same time pressure varies.  

From a practical point of view, soil water retention curves can be extended to positive pressure (negative suctions) adding a term which is well known in groundwater literature and is called specific specific storage
These qualitative descriptions do not end the complex phenomenology of water retention curves.  

As Nunzio Romano (GS) and coworkers noticed, and Kosugi (1994) before them, soil water retention curves shape depend directly on the pores' distribution. This, however, is not necessarily a unimodal distribution but can be multimodal because of soil structure and soil "disturbances" in form of macropores due to animal or roots decay. In this case soil water retention curves (their integral) can be more complex than expected, as shown in Figure below.

This opens to a series of generalisation, but it would be the topic of some other post (and actually was already the topic of several posts on soil freezing).


References

Kosugi, K. 1994. Three-parameter log-normal distribution model for soil water retention. Water Resour. Res. 30:891–901. 

Lu N, Godt JW. Hillslope Hydrology and Stability. Cambridge: Cambridge University Press; 2013. 

Lu, N. (2016). Generalized Soil Water Retention Equation for Adsorption and Capillarity. Journal of Geotechnical and Geoenvironmental Engineering, 142(10), 04016051–15. http://doi.org/10.1061/(ASCE)GT.1943-5606.0001524

Romano, N., Nasta, P., Severino, G., & Hopmans, J. W. (2011). Using Bimodal Lognormal Functions to Describe Soil Hydraulic Properties. Soil Science Society of America Journal, 75(2), 468. http://doi.org/10.2136/sssaj2010.0084


Steudle, E. (2001). The Cohesion-Tension Mechanism and the Acquisition of Water by Plant Roots. Annual Review of Plant Physiology-Plant Molecular Biology, 847–877.

Too, V. K., Omuto, C. T., Biamah, E. K., & Obiero, J. P. (2014). Review of Soil Water Retention Characteristic (SWRC) Models between Saturation and Oven Dryness. Open Journal of Modern Hydrology, 04(04), 173–182. http://doi.org/10.4236/ojmh.2014.44017

Monday, November 7, 2016

Reservoirology #3

This is a revision of the previous post on the same topic. There I tried to develop my own algebra of symbols to represent coarse grained (spatially integrated) hydrological system. Later on I understood that Petri networks were already there and useful to obtain the same result. The graphs obtained in such a way where, besides, studied in several places, and many contributes of literature convergent from other disciplines, can be used for hydrological scopes.
The presentation (click on the figure) completely substitutes the old one. Who liked it, will like better this.  

Sunday, November 6, 2016

About graphs, DSL and replicable research in Francesco Serafin's work

This is the summary of what Francesco Serafin (his blog) did in its first year of doctoral studies, defending for his admission to his second year Ph.D. Undoubtedly he did a lot of work and he programs to do even more. Three are the lines of his research:

  • implementing a new flexible structure based on graphs for commanding simulations of interacting systems; 
  • implementing a domain specific language for doing environmental models (and particularly to solve ordinary and partial differential equations); 
  • deploying a system that makes easier to do replicable science. 
Clicking on the Figure above you will find his presentation. Obviously the committee admitted him to the second year of his Ph.D. studies.