Opinions Formulated

The following scientific opinions have been formulated by me as a result of my expertise, experience and as a consequence of the systematic analysis of the Applicant's models that has been conducted by my staff colleagues at Principia, and myself. The findings from such an analysis are comprehensively documented in a separate report, and are hence not repeated here. The numerical order in which my opinions are presented below as numbered paragraphs, and clarifying sub-paragraphs within them, does not imply any special significance.

  1. It is essential to apply refined mathematical models to predict the availability of water and the consequences of its development in complex ground water systems. This is particularly true of ground water systems which include: streams of varying capacities and carrying flow rates which vary from month to month and from year to year; stream-aquifer interactions involving both gains and losses that vary with reach and with time; significant water-quality variations; historical and current diversions of both surface water and ground water; diverted water applied to irrigated lands used to grow varieties of crops and temporal variations thereof; complex patterns of return flows from applied irrigation water; changes in land-use practices that occur over periods of time; diverse types of vegetation, including wetlands, consuming water from the system; geological and hydrogeological complexities in sub-surface formations and units postulated to transmit ground water; and, boundary conditions that vary both spatially and with time. The natural and man-made features of such systems involving estimations of water quantities, require mathematical models of corresponding refinement and robustness when they are intended for use in predicting impacts of proposed water development within the systems. The South Park ground water system is complex in just these ways.

  2. When properly constructed, developed, calibrated and verified, mathematical models can be used reliably to predict quantities of water available in a complex ground water system as well as the consequences of its development. Such models require to be supplied with system-specific data characterized by necessary degrees of appropriateness, completeness and accuracy. Data supplied to such models has to be kept distinctly separate from data sets set aside for purposes of calibrations and verifications. Spatial and temporal completeness of data used in these three distinct respects are important requirements of reliable models. Based upon this criterion alone, the Applicant's modelling cannot be considered as reliable.

  3. A predictive mathematical model must fulfill certain essential requirements of reliability if it is to serve as valid support for a water-rights application. Specifically, a model used as scientific support for a water-rights application involving ground water systems with surface water interactions, must demonstrate reliability both in properly representing the actual conditions that have prevailed prior to implementing the proposed water development, and in predicting the quantitative consequences of its implementation. Examples include the following.
    1. When an application makes an underground water claim, the model it is based on must make a valid, complete and accurate representation of relevant sub-surface recharge and discharge conditions as they are, and as they have changed with time, due to conditions that existed prior to implementing the proposed application.
    2. When an application relies upon a claim for water to be stored underground, the model it is based on must make a valid, complete and accurate representation of the quantity of water proposed to be recharged or of the sub-surface space created to accommodate such recharge, as well as the valid changes to sub-surface and surface water conditions that are created as a consequence of doing so, however temporary or permanent they may be.
    3. When an application submits a plan for augmentation designed to compensate for quantitative impacts caused by implementing the proposed application, the model it is based on must make valid, complete and accurate predictions that properly quantify all depletions, their locations and their timing.
    In summary, a predictive model used in the context of the proposed application is reliable when it can demonstrably: verify that unappropriated ground water is indeed available in the quantities that are claimed; establish the sub-surface storage capability and capacity upon which the proposed application relies; identify whether, where, when and how much of the sub-surface water that is otherwise consumed is proposed to be salvaged; quantify stream depletions caused by implementing the application and the timing of each; and, quantify impacts to other relevant existing entities. Furthermore, these requirements must be applied even more stringently, if the application is based upon novel concepts which have not been demonstrated as valid for the ground water system in question. Based upon these criteria alone, the Applicant's model cannot be considered as reliable.

  4. Quantitative reliability of a mathematical model that is used as support for a water-rights application is essential and should be demonstrated by an Applicant. That is why close attention to relevant details involved in the development, adjustments and implementation of valid modelling procedures by an Applicant matters to the outcome of model predictions. For the same reason, the completeness, correctness and accuracy of appropriate on-site information used for modelling matters a great deal. Examples include the following.
    1. The appropriateness of a modelling framework chosen for the specific task at hand in an application does influence the predicted quantitative outcome. When the chosen framework ignores or otherwise neglects valid components of a ground water system, the outcome calculated by the model when evaluating impacts such as depletions, may not really be predictions but become forgone predeterminations. In such instances, models cannot be applied to predict depletions caused by proposed applications in any reliable fashion.
    2. A model which remains faithful to known and relevant real-life facts concerning a ground water system is much more likely to predict depletions caused by an application that are valid, realistic and accurate, than one which does not.
    In summary, mathematical models become unreliable for use in quantitative decision making when those factors which really matter quantitatively are either ignored or only partially or incorrectly treated. This is precisely the status of the Applicant's modelling.

  5. A proper scientific peer review of mathematical models and their predictive capabilities must be based upon three essential requirements if its outcome is to be trusted as valid and objective. They are identified as follows.
    1. Complete independence of the review process from the project team's own efforts of model development, calibration, verification and applications, must be preserved.
    2. The review process should not only consider complete documentation of modelling procedures adopted by the project team, but also evaluate the complete computer program or programs embodying the model, its input and output data sets and the sensitivity analyses conducted with it. It should also evaluate the calibration and verification procedures actually implemented, in such a computational manner as to lead to total familiarity with the model's workings.
    3. The peer review process should itself be documented to include complete details of all modelling-related materials reviewed, any questions posed to the project team responsible for model development and answers received. It should also present the scientific findings and opinions formulated by the peer reviewers regarding the model and its predictive reliability.
    Obviously, a peer review body should be composed of appropriately qualified scientific and/or engineering peers, i.e. one or more eminent practitioners of the very type of mathematical modelling which is under review. Their review should be conducted independently and without constraints. While the review process itself may or may not be conducted in secret, i.e. without the knowledge of the project team, the review body should have unfettered access to those who have developed and applied the model in question such that any questions of clarification may be posed and answers received. Objectivity in a review process can be trusted only when the process and its findings are completely documented for scrutiny by any third party interested in it. When conducted in this manner, whatever the outcome, the findings from the peer review may be trusted as objective and relied upon for purposes of decision making. Reviews of the Applicant's modelling by members of the Applicant's team of consultants were not made in this way.

  6. On occasion, the term "peer review" is misapplied to modelling actions and, consequently, the very objective of conducting the review is either misunderstood or misrepresented. For instance, incorporating the suggestions offered by a scientific peer, inducted into the project team, with regard to some component of modelling, but not its entirety, is sometimes represented as constituting a peer review. Or, allowing such a peer to read the draft of a report describing portions of the project team's modelling efforts, is similarly labelled. Interpreting such actions as the proper modelling peer review process identified above is clearly erroneous. It should not mask the fact that the underlying data sets, the embedded hypotheses and assumptions, the process and status of model calibrations as well as the success or otherwise of model verifications may have never received the close and independent scrutiny they deserve. Only such a scrutiny will allow objective assessments of the quantitative reliability of the model in question to be made. Such assessments alone can either support or reject the use of the model as a valid and reliable basis for purposes of decision making. The Applicant's modelling did not receive such scrutiny.

  7. The conceptual framework of the Applicant's model of the South Park ground water system is flawed because the choices made predetermine the modelling outcome regarding impacts within geological formations, such as the Reinecker Ridge and Laramie-Fox Hills, forming part of the South Park ground water system. Such flaws do not allow quantitative impacts to them to be properly evaluated.

  8. The conceptual framework chosen by the Applicant for the model is additionally flawed because it is based upon a mis-characterization of the manner, timing and rate of ground water recharges to and discharges from the deeper, artesian water-bearing units of the South Park ground water system.

  9. Data available to characterize the complex South Park ground water system, with respect to appropriateness, spatial and temporal completeness as well as accuracy, are insufficient to develop a proper model, to calibrate it, to verify it and to apply it in order to make reliable predictions.

  10. The numerical grid cell arrangement chosen by the Applicant for modelling is not suitable for the predictive purposes for which the model has been applied. For instance, the coarseness of the chosen arrangement does not allow proper representations of stream-aquifer interactions, vegetative consumptive use, and flows at springs and seeps, to be made.

  11. The discretization of time periods chosen by the Applicant for modelling is not suitable for the predictive purposes for which the model has been applied. For instance, the coarseness of the chosen one-month stress periods does not allow rapid variations in stream flows to be properly accounted for.

  12. The model layers chosen by the Applicant for modelling do not represent the South Park ground water system as is presently known. For instance, the prescribed model layering contains implausible gaps and impossible overlaps between adjacent layers. Also, for instance, the grouping of different materials into single model layers prevents the true characteristics of geologic materials in the South Park ground water system, from being properly represented.

  13. The framework chosen by the Applicant for modelling the South Park ground water system contains a significant number of computational errors. For instance, the assigned number of active grid cells differs from one model run made by the Applicant to the next.

  14. The framework chosen by the Applicant for modelling the South Park ground water system contains errors which have serious implications for model reliability. For instance, the assigned spatial distributions of the top and bottom elevations of model layers differ from one model run made by the Applicant to the next.

  15. The selection of aquifer material property values based upon a few non-representative tests and their spatial assignment by the Applicant to large numbers of model grid cells, are improper. For instance, values of specific yield and storage coefficient have been erroneously assigned to numerical grid cells. Also, for instance, assignments of other property values such as vertical conductance, are based entirely upon assumptions whose validity for the South Park ground water system has not been demonstrated.

  16. The representation of the Elkhorn Thrust Fault zone in the model and the assignment of material property values to numerical grid cells chosen to represent it, are improper. For instance, the spatial location represented in the Applicant's own report differs significantly from that represented by grid cells of the Applicant's model. Also, for instance, assignments of property values to large segments of this zone are based entirely upon assumptions whose validity for the South Park ground water system has not been demonstrated.

  17. The selection and assignment of general head boundary condition values to boundaries in model layers 1 and 2, are improper. For instance, the assigned values are based entirely upon assumptions whose validity for the South Park ground water system has not been demonstrated.

  18. The assignment of pumping rates to existing wells in order to represent historical well pumping in the model, is improper. For instance, assignment of constant values to historical well pumping mis-characterizes practices that may have actually occurred and consequent responses of the South Park ground water system to pumping stresses.

  19. The spatial assignment of aquifer recharge rates, attributed to precipitation, irrigation return flows, leakages of conveyance ditches, etc., to model grid cells and their time-dependent variations, is improper. For instance, the assigned values of recharge rate to certain grid cells located near streams is implausibly high due to a number of incompatible assumptions and computational errors.

  20. The representation of stream flows in the model of the South Park ground water system is improper and incorrect. For instance, the prescribed rates of stream flow at model inflow boundaries bear no resemblance to corresponding flow rates estimated by the Applicant's own surface water analyses. Also, for instance, the routing procedure implemented in the model contains a number of significant errors.

  21. The representation of stream-aquifer interactions in the model, central to calculating depletions to stream flows caused by the proposed well pumping, is improper and incorrect. For instance, the assignment of constant stream stage values to grid cells irrespective of the rate of stream flow and changes to it with time, is incorrect and causes stream gains and losses to be incorrectly calculated.

  22. The calculations of ground water recharge and discharge in connection with stream flows are incompatible with calculations of stream gains and losses both within the ground water model and as compared with the surface water calculations made by the Applicant. These calculations have not been performed iteratively as necessary to ensure compatibility and consistency.

  23. The representation of vegetative consumptive use, through evapotranspiration, in the model is improper. For instance, the values of percent cover representing the fraction of grid cell areas actually covered by vegetative growth, are incorrect and result in incorrect estimates of vegetative consumptive use rates and quantitative impacts upon them caused by the proposed well pumping. Similar improprieties are characteristic of the Applicant's assignments of locations where vegetation types occur, the depth function and extinction depth used, etc.

  24. The representations of springs, seeps and ponds as discharge locations in the model are improper. For instance, the elevations of springs are incorrectly represented in model grid cells postulated to contain them. Combined with assumptions concerning the conductance of materials containing them, result in incorrect estimates of available flow rates and impacts to them caused by the proposed well pumping.

  25. The assignment of material property values and representations of significant mechanisms which affect a number of ground water recharge and discharge mechanisms are characterized by significant computational errors. For instance, errors in routing variables within several stream segments forces flows within them to be directed uphill.

  26. The procedure adopted to calibrate the model is flawed in significant respects, and the efforts to calibrate the model have been unsuccessful. For instance, the process of calibrating the model to targets under quasi-steady-state conditions was based upon choices of available water level data in wells covering decades of changes. Also for instance, the procedure of adjusting some physical parameters, representing real variables such as percent cover of vegetation in grid cells during calibration, is improper.

  27. The Applicant's model of the South Park ground water system has not been calibrated to the Applicant's own identified targets. For instance, the state of calibration does not meet any of the ground water flux targets. Even the state of model calibration to the chosen targets is demonstrably poor and insufficient to establish the model's predictive capability.

  28. The modifications made by the Applicant to the model's calibrated model parameters, such as prescribed aquifer properties of specific yield and hydraulic conductivity, in order to make subsequent predictions with it, are entirely improper. They invalidate both the calibration and the predictions made with the model.

  29. The fact that no attempt was even made to verify or validate the model after the calibration process was undertaken, prevents it from being demonstrated as valid and accurate and thus reliable for purposes of making quantitative predictions upon significant features of which the Application rests.

  30. Based upon the requirements stated in previous paragraphs, not only was the Applicant's model not properly peer reviewed but has not been demonstrated by any reviews that may have been conducted, but which have not been recorded, to be reliable as a predictive tool in this case.

  31. The most complete review conducted of the Applicant's model prior to the present effort was that undertaken by the Objectors' Technical Review Committee (TRC) in January, 1998 and conveyed to the Applicant shortly thereafter. Since the Applicant did not produce a copy of the model itself at that time but only an initial report providing general descriptions of how it was developed and applied, the TRC could not have completed a proper peer review. Nevertheless, using the Applicant's own documentation as a basis, the TRC concluded that the model was entirely unreliable in its most fundamental respects and recommended that it be excluded from further consideration at that time. The TRC also offered comments and suggestions for the model's improvements in those respects. The Applicant has chosen to ignore them during the approximately two years after such advance warning was received. Consequently, the Applicant's ground water model cannot now be referred to as peer reviewed and, implicitly, claimed to be found acceptable.

  32. The USGS' measurement data on ground water levels concerning the South Park ground water system was recently obtained in a manner which reflects proper scientific methods of identifying geologic formations, interference from nearby pumping wells, well construction features, etc. These data have been presented by the USGS properly as depths to the ground water table. The USGS clearly identified these data as useful in improving understanding of the ground water system, but not appropriate for use in preparing spatial interpretations of the ground water table or corresponding ground water flow directions. The Applicant's use of such measurement data to derive precisely the ground water elevations to serve as model calibration targets, is improper.

  33. Scientific reviews indicate that the Applicant's model of the South Park ground water system, its predicted results and all the quantitative claims which rest upon it, must be rejected as entirely unreliable and unsuitable for the purpose of decision making in this case. In the absence of a predictive model that is properly developed, calibrated and verified with respect to actual water conditions within the South Park ground water system, available measurement information is insufficient to establish just how ground water behaves in the South Park ground water system, how it interacts with streams and affects stream flows, how seasonal changes and how wet and dry seasonal cycles influence such behavior. Therefore, available measurement information remains insufficient to develop, calibrate, verify and apply a reliable mathematical model. By itself, it is also insufficient to evaluate quantitative impacts that will be caused by implementation of the Applicant's water development proposal.

  34. The science of fluid dynamics applied to non-linear ground water flows indicates that any future well pumping undertaken in the area of concern within the South Park ground water system, by entities other than the Applicant, will quantitatively influence the underground storage reservoir proposed by the Applicant. The position, shape and size of this reservoir will be altered as will the consequent impacts to surface streams in the area.

  35. The only sensible way to establish a proper understanding of ground water and surface water behavior in South Park, before any proposed water development plans can be implemented, is to ensure that a significantly large number of stream gauges and observation wells, located in a spatially distributed manner that is appropriate, are installed properly on streams and constructed properly within each of the affected water-bearing units. These stream gauges and observation wells should then be monitored regularly over a significant span of time and analyzed properly to advance understanding. Only after such an understanding is developed, can a reliable model be developed, calibrated and verified before being applied to evaluate quantitative proposals concerning water development.

  36. Observation wells and stream gauges installed for purposes as described in the previous paragraph should be designed to obtain water quality information in addition to data on water quantities. When such data are available the effects of dissolved constituents, including radioactive materials, in pumped ground water used for augmentation can be properly evaluated in the context of applicable water-quality regulations.

  37. In the absence of a proper understanding of the South Park ground water system, it will not be possible to implement reliable decisions concerning important matters in this case such as augmentation plans intended to compensate for stream depletions. An invalid and unreliable model, such as the Applicant's, that is used to make predictions concerning this non-linear system cannot itself overcome this lack of understanding. Further, in the absence of a proper understanding, ad hoc suggestions for making a few short-term (i.e. of the order of 3 years) measurements of stream flow and/or monitoring of ground water elevations at a few chosen locations will not suffice as alternative methods of quantifying the impacts of proposed water developments on stream flows. Such suggestions will have no proper means of evaluating where and how important factors, such as other unrelated ground water pumping proposals, will exert additional but indistinguishable quantitative influences upon stream depletions.


Index | Introduction | Qualifications & Experience | Documents Considered | Opinions Formulated | Supplementary Opinions Formulated | Glossary
Created 10-Mar-2000 by webmaster@prinmath.com
Copyright © 2000, Principia Mathematica, Inc. - All Rights Reserved