The importance of state transformations when using the ensemble Kalman filter for unsaturated flow modeling: Dealing with strong nonlinearities

• We use EnKF together with a strongly nonlinear unsaturated zone model.
• State variables and observations are different quantities (heads and water content).
• Extreme state values (skewed distributions) could adversely impact the EnKF update.
• Normal score transformation of state variables helps mitigate the update problem.
• Also transforming state variables to observation variable helps mitigate the problem.

The ensemble Kalman filter is, due to its computational efficiency, becoming more and more popular as a method for estimating both model states and parameters in hydrologic modeling, also for nonlinear state propagation models. In the ensemble Kalman filter the calculation of the error correlations, and hence the filter update, is done based on the ensemble of model evaluations and can therefore be strongly influenced by a few ensemble members with extreme values. With nonlinear state propagation models, extreme values can be a common phenomenon that can be, especially if there are nonlinearities between the observed variable and the modeled states, problematic during the filter update. An illustrative example of this problem is shown using an unsaturated flow model where the modeled states are pressure heads and observations are water content. It is demonstrated that the ensemble Kalman filter can in this case yield a deterioration of state predictions. We discuss the normal score transform and the transform with the retention function applied to the model states in order to mitigate this problem. It is shown that both transforms improve the estimation of the model states and parameters.