Model Calibration

When modeling water quality, pressures, per se, are not the best indicator of calibration. The best calibration would be against water quality measurements. Absent that, the most important variable to calibrate against is flow. Of course, flow measurements in a distribution system are a real luxury, so we need to use pressure data as advantageously as possible.

It seems to me that setting a simple tolerance (i.e. plus or minus x psi) is not the best way to calibrate a water quality model. What you really want to represent realistically is the shape of the grade surface.

We also need to consider the precision of pressure measurements themselves. Remember the old saying: measure with micrometer, mark with chalk, cut with axe.

I'm looking forward to what others have to say on this subject.

Regards,
Ben Harding
blh@hydrosphere.com

Fellow Modelers,

I am soliciting opinions and ideas on the following two questions:

1. when is a network model considered "sufficiently calibrated"
2. should calibration standards or criteria be established for network models, by say, a committee of one of the professional societies?

I know that the answer to (1) depends in part on the intended use of the model (e.g., planning or operation). But I think practicing engineers have a general sense or rules of thumb about how close a model should come to reproducing observed conditions before being deemed acceptable for use. An example might be "a network model is sufficiently calibrated if 80% of all pressure measurements agree within 5 psi with observed readings".

Question (2) asks if these un-stated criteria should be made explicit, in the same way as design standards on pipe materials or the accuracy/precision of chemical analysis tests, for example, are established by professional committees. Explicit criteria would assist modelers in setting a goal to reach when performing calibration (including the need to obtain more field data if necessary) and help insure that proper calibration is made before a model is used. On the other hand, there may be so many special cases and exceptions as to make any general criteria meaningless.

I volunteered to discuss the various sides of these issues, within the broader context of environmental modeling in general, in an upcoming "Forum" article in the Journal of Environmental Engineering. Your ideas on this subject would be most appreciated.

Lew Rossman
US EPA - NRMRL
rossman.lewis@epamail.epa.gov

I fully agree with Mr Ben Harding's view on model calibration. This illustrates the need for developing methods for water quality measurements. Especially source tracing would be very useful.

In most cases however, a model which is calibrated for flows under normal conditions will be acceptable for water quality analyses. Unfortunately flow measurements in a distribution system are indeed a "luxury" and simpler methods need to be developed. In Swedish modelling practice, demand values are normally based on annual billing measurements and scaled according to instantaneous values at supply points. Diurnal variation is deduced from large groups and imposed on every single user, possibly divided into two or three different types. This is a sufficient level of ambition for a normal municipal application, but it is not sufficient for correct modelling of flows (let alone quality parameters) in the detailed distribution network.

Lacking other parameters, pressure measurements have been used widely for model calibration in Sweden (as - I suppose - in other countries). However this gives very little information on the model performance under normal conditions. Pressures may be significant under peak flow conditions, for which the networks are normally designed. But under normal conditions energy losses are simpy too small to be a good flow indicator.

To summarize:

- The purpose of the model is crucial to the calibration requirement
- Simple methods for source tracing and flow measurements would mean a breakthrough for the calibration of network models
- Pressure is probably the least significant parameter for model calibration

Yours,
Krister Törneke
Municipality of Botkyrka, Sweden

Answering the question, "Is a model calibrated?" is a lot like answering the question, "Did you stop beating your dog?" There is no yes and no, black and white answer. There are just varying shades of gray.

Adequacy of model calibration depends on the cumulative effects of errors, approximations, assumptions and projections and necessarily falls on a spectrum somewhere between a perfect model (that doesn't exist) and a perfectly incorrect model. All models lie somewhere in that spectrum and the adequacy of calibration depends on the circumstances and the intended use.

There are so many applications of models that I have my doubts about the possibility of developing standards. The real test is whether you can use the model to support you decision making.

I disagree with Ben and Krister's comments that water quality data are needed for model calibration. About 98% of model applications deal with fire flow and pipe sizing and water quality isn't explicitly considered. This points out the fact that different people have different ideas about what a model should do. (Remember the old story about the blind men and the elephant.)

Best wishes,
Tom Walski
Pennsylvania American Water Company

>Answering the question, "Is a model calibrated?" is a lot like answering the question, "Did you stop beating your dog?" There is no yes and no, black and white answer. There are just varying shades of gray.

This is a point of agreement.

>I disagree with Ben and Krister's comments that water quality data are needed for model calibration. About 98% of model applications deal with fire flow and pipe sizing and water quality isn't explicitly considered. This points out the fact that different people have different ideas about what a model should do. (Remember the old story about the blind men and the elephant.)

I agree that the data is not necessary for a model designed to substantiate fire flows, however, in analyzing our local water district's use of MODELING, it has become painfully clear that they are ignoring the capability of their MODEL to predict the concentrations of chemical constituents. Normally the effort to model a system is dependent upon who on staff has the time available to pursue the model in a diligent manner. Tests to verify model predictions are based solely on spot checks of the system, with very little attention paid to calibration of test instrumentation. Our local supplier has experienced a number of problems with inadaquate treatment, and in some cases, concentrations of chemicals that have upset the composition of the water to a point that property owners are experiencing failures of piping systems due to corrosion (confirmed by independent labaratory tests).

You pointed out very succinctly that the model should be used as a tool to help control the quality, volume and availability of water. The model requires reasonable attention to detail and frequent input from spot checks on the system to verify predicted results. I see two areas of concern here for a water provider, what do they have to do to meet statutory requirements, i.e. minimums, and what must they do to deliver a quality product. Most of these systems by their very nature provide minimum acceptable STATUTORY quality control. They are in most cases a monopoly. End users seldom get to choose who their provider will be. The result is that not enough attention is paid to modeling even though the quality and ability of modeling software has improved to a point where it could be considered at least as important a tool as the pipe wrench used to assemble the system.

Regards,
M.L. Garrison
Garrison Management Group