Contact 
Juliane Mai, Department of Civil and Environmental Engineering
James Craig, Department of Civil and Environmental Engineering
Bryan Tolson, Department of Civil and Environmental Engineering
Introduction
Hydrologic models are widely used to improve understanding of hydrologic processes and support water management decision making, including in flood prediction, drought monitoring, infrastructure design and reservoir management. Further development and improvement of models is essential to advance the understanding and ensure greater model realism.
The evaluation of models using sensitivity analysis is a well-established method to identify the most critical relationships within a system, but has challenges that limit the transferability and insight from individual analyses. Traditionally, sensitivity analysis is performed on an individual model to estimate the sensitivities of model parameters on streamflow, rather than quantifying the sensitivity of streamflow to hydrologic processes. The analysis is often based on thousands of model runs making them computationally very expensive and difficult to share due to the amount and complexity of the data.
In this study, a novel method is applied to over more than 3000 basins across North America using a blended hydrologic model structure enabling the seamless quantification of model process and parameter sensitivities across a continuous set of models. Importantly, study inputs, setups and results were made available through an interactive website.
Methodology
A blended model withing the Raven hydrologic modeling framework was used to enable the simulation of various model structures simultaneously. The analysis used the HYSETS dataset comprised of 14,425 basins filtered by size and data availability and quality. The sequence of experiments performed in the study are shown in Figure 1.
Figure 1: Flowchart of experiments and analyses
A preliminary calibration/validation experiment of the blended model was performed to demonstrate the basic adequacy of the model and to act as the basis for excluding poorly performing models from analysis. The subset of basins with more than five years of observed streamflow data between January 1991 and December 2010 was calibrated using the Dynamically Dimensioned Search (DDS) algorithm maximizing the Nash-Sutcliffe efficiency (NSE). Only basins with more than five years of observed streamflow available were assessed in validation to ensure that the overall performance during calibration was maintained during an independent time period.
The extended Sobol’ Sensitivity Analysis (xSSA) method was applied to 3316 calibrated basins to gauge their sensitivity to simulated streamflow and was illustrated at one example catchment per climate zone. The Sobol’ main sensitivity index Si, and total sensitivity index STi, were determined for model parameters, process options, and processes. Functional relationships between known basin characteristics and the importance of these hydrologic processes were inferred using regression analyses.
Outcomes
The preliminary calibration of the basins was comparable to the performance of other models applied across the continental United States (US). The results of the calibration exercise were used to exclude low quality models from further analysis and to demonstrate the basic adequacy of models for simulating streamflow over the range of simulated conditions. While a more elaborate calibration study may improve individual optimal model performances, it is unlikely to yield improved global sensitivity estimates.
The application of the Sobol’ sensitivity index showed clear spatial patterns of hydrologic process sensitivity to streamflow (Figure 2). Overall, quickflow (Figure 2b) was the most important process for streamflow simulations and exhibited large sensitivities, especially in the Eastern US which is covered mostly in temperate broadleaf and mixed forests. Infiltration, evaporation and percolation (Figure 2a, c, and g respectively) were processes of secondary importance, especially in the Eastern US. In the Rocky Mountains and in higher latitudes, the model’s potential melt, precipitation correction and snow balance components (Figure 2f, i, and d) were the most important, while on the West Coast, rain-snow partitioning (Figure 3h) was more influential.
Figure 2: Importance of hydrologic processes for streamflow simulations
The variance-weighted total Sobol' sensitivity (STwi) for simulated streamflow against the nine hydrologic model components for 3316 basins. Note that the colourbar is logarithmic and that green colours indicate a large importance of the respective process on streamflow simulations while pink colours indicate a weak impact. The location of the dots in each panel marks the location of the outlet of each basin. The histograms of sensitivity values in each panel are the same as used for the colourbar.
An assessment of the time-dependent sensitivities of hydrologic processes in specific climatic regions confirmed that quickflow was the most important process. While the overall results were consistent with the time-aggregated sensitivities, the variation in process sensitivities throughout the year yielded more detailed insight into the seasonal variability in hydrological process influence across basins. For example, evaporation was found to be important, especially during low-flow periods. Potential melt and snow balance were often important during cold months and especially during the freshet, while baseflow was only visible in some regions and only during severe low-flow periods.
Figure 3 shows the predictability of process sensitivities with each functional relationship for each process using one predictor unless the use of two predictors increased the adjusted coefficient of determination. All panels show that the predicted sensitivities are in close agreement with those derived by the xSSA analysis, indicating that a few basic variables may be sufficient for describing the process sensitivity at most sites.
Figure 3: Predicted sensitivity index estimates using functional relationships based on basin characteristics
The total Sobol' sensitivity indices of hydrologic processes on streamflow are estimated using the xSSA method, after which they are predicted through basin-specific characteristics only (y-axis). Each panel shows the results for one hydrologic process. The mean and standard deviation of the Adjusted Coefficient of Determination R2adj and the mean absolute error (MAE) are reported. Shown are the samples of the 100 validation experiments with colour indicating the density of samples.
Conclusions
This continental-scale sensitivity analysis of hydrologic model parameters and processes to identify parameters that have on strong impact on model outputs and responses addressed key challenges associated with the traditional application of the method. The traditional application of sensitivity analysis for model-specific parameters leads to difficulties in making high-level modeling decisions such as whether to prioritize model development or seek better datasets; this study demonstrated an approach that covers model components and processes at the continental scale. In addition, the large number of basins analyzed allowed one to deduce functional relationships between basin characteristics and climatic indices that estimated the sensitivity of processes without the need to perform the computationally expensive sensitivity analysis for basins not covered. While sensitivity analyses are usually based on a single model limiting the transferability of conclusions, the use of the Raven hydrologic modeling framework and the novel blended hydrologic model supported a seamless analysis of a range of model structures and reduced the risk of inferring model-specific conclusions.
While sharing the results of large-scale model analyses is difficult due to the amount and complexity of data, this was an open-source and open-data study that enables users to explore input data, model setups and study results from an interactive website. For example, users can visualize the results that generated Figure 2, display time-dependent process sensitivities, or download calibrated model setups for the more than 3000 stations used in the study.
The approach to continental-scale sensitivity analysis demonstrated in this study is transferable both to other basins and models using similar process representations and could be extended to other modular environmental and ecological models similarly challenged by high degrees of structural uncertainty.
Future studies might investigate if more distributed land-surface hydrologic models demonstrated similar patterns in process sensitivities and if the sensitivities could also be related to basin characteristics. The influence on sensitivities when applying other forcing datasets or modeling decisions, such as different numbers of soil layers, other process options or different simulation time steps, might also be investigated.
Mai, J., Craig, J.R., Tolson, B.A., & Arsenault, R. (2022). The sensitivity of simulated streamflow to individual hydrologic processes across North America. Nature Communications, 13, 455. https://doi.org/10.1038/s41467-022-28010-7
For more information about WaterResearch, contact Julie Grant.
Aerial Photo by Joshua Fuller on Unsplash
