The Still Glass

A steam boiler will kill everyone near it if the water gets too low, and the instrument that tells you the water is not too low is a glass tube.

The tube runs vertically alongside the boiler and is connected to it twice: at the top, into the steam space, and at the bottom, into the water. Water finds the same height in the tube as in the vessel, because the two are in communication and gravity is indifferent to the shape of the container. You look at the glass, you see a line, and the line is the water level.

Except that it is not. The line in the glass is the water level in the glass. It stands in for the level in the boiler only for as long as the two are actually connected, and the reading contains no information about whether they are.

This is the entire problem, and a century of explosions is written into how the trade learned to handle it.

The bottom connection is the one that fouls. It opens into the water space, which is where scale precipitates and sludge collects, and a boiler that is doing its job is continuously manufacturing the material that will block it. When that leg closes off — by scale, by sludge, by a valve someone shut and forgot — the water already in the tube has nowhere to go. It sits there. The glass shows a column of water at a perfectly reasonable height while the actual level in the boiler falls away beneath it, uncovering the crown sheet, which is the plate directly over the fire and the one part of the structure that depends on water to survive contact with flame.

So the reading six inches of water has two producers. One is six inches of water. The other is six inches of water an hour ago and a plugged pipe since. They are the same reading. Nothing in the glass distinguishes them.

And the counterfeit is the more convincing of the two, which is the part worth sitting with. A live gauge glass on a working boiler is not still. The level trembles — the boiler is boiling, the pressure fluctuates, water is being fed in and steam drawn off, and all of that agitation propagates up the tube. A blocked glass, cut off from all of it, is calm. It gives a clean, steady, unambiguous number. Practitioner guidance treats this as a tell: a static level on a steaming boiler points at connections that are plugged or nearly so, while a level that moves is evidence the tube is still talking to the vessel. The property that in almost any other instrument reads as precision is here read as a symptom.

What the trade did about it is the interesting part, because none of the three answers is build a better glass.

The first is to perturb it. The gauge glass blowdown is a scheduled procedure, daily on many plants, and its logic is worth following step by step because the sequence is doing something specific. You close both cocks and open the drain — flow should stop, which tests that the valves seal. You open the steam cock alone and watch for discharge: that tests the top leg by itself. Close it, open the water cock alone, watch for discharge: that tests the bottom leg by itself. Then close the drain, let the glass refill, and open the steam cock so the column settles back to true.

Notice that the procedure begins by destroying the reading. It drains the glass. It has to, because the standing column is precisely the thing both producers can make, so no amount of staring at it will sort them. Only the refill discriminates: water that comes back is water that is connected to something. The test is not of the level but of the communication, and communication is a property that only becomes visible in motion.

The second answer is the one the operator carries between blowdowns: read the liveliness rather than the number. You are not checking that the level is correct. You are checking that it is alive.

But this one does not simplify into a direction, and I nearly wrote it as though it did. More motion is not more health. Violent bouncing in the glass has its own name — priming — and it is a fault in its own right, driven by dissolved solids or oil that let a froth build on the water surface instead of clean bubbles. So the operator is not reading more movement is better. They are reading a band: too little says the tube has stopped communicating, too much says the water itself has gone wrong, and the two failures sit at opposite ends of the same observable. A scalar with a bad tail on each side is a considerably more awkward thing to carry in your head than a rule with an arrow on it, which is presumably why the arrow is what I reached for first.

The third is to stop asking the glass. Boilers carry try cocks — small valves, conventionally three, set at heights spaced across the normal range. You open one and see what comes out. The bottom should give water, the top should give steam, and the middle tells you where the boundary sits between them. This is not a better reading of the same quantity; it is a different question, and it is immune to the failure that defeats the glass, because there is no tube in which a column could be stranded. You have opened a hole into the boiler at a known height and the boiler has answered. Try cocks exist, explicitly, for the case where the glass has been compromised or plugged.

And the perturbation is not left to the operator's judgement. The federal inspection schedule for steam locomotives lists, among the things to be done daily, "testing of water glasses and gauge cocks" — testing, not reading — with "cleaning and inspection of water glass valves and gauge cocks" required separately every thirty-one service days. The law does not ask whether the instrument showed a good number today. It asks whether anyone made it prove it could still move.

That same regulation, 49 CFR Part 230, contains a detail I find quietly excellent. The lowest reading of the water glass, and the lowest gauge cock, must sit not less than three inches above the highest part of the crown sheet. The instrument is deliberately arranged to run out of range before the thing it protects runs out. An empty glass is not the emergency; it is the last warning that precedes one, with three inches of margin still in hand. The apparatus is designed to fail informatively.

The cost of getting this wrong does not need embellishment. On 29 July 2001, at the Medina County Fairgrounds in Ohio, a J. I. Case traction engine built between 1906 and 1909 exploded and killed five people. The finding was crown sheet failure from overheating caused by low water; metallurgical examination showed the crown sheet had thinned in places from its original 0.375 inches to 0.087 — twenty-three percent of the metal it was built with. I want to be careful here, because it would suit my argument to say the glass lied at Medina and I have not established that. What Medina establishes is the consequence of low water, which is the condition the glass exists to prevent. How the water got low there is a separate question with a separate and much longer answer.

There is one more layer, and it defeats everything above. Come back to priming, because it does something worse than sit at the far end of a band.

When boiler water carries enough dissolved solids, the surface tension changes and a thick foam forms on top instead of clean bubbles breaking. Now the glass is clear. Both legs are open. The blowdown passes. The try cocks work. And the standard instruction to the operator is that while a boiler is foaming or priming it is difficult, and often simply impossible, to read the true water level in the glass at all — partly because what the glass is showing is the top of a froth rather than a surface, and partly because water trickles down into the tube from the steam connection above, wetting it from the wrong end. The instrument is in perfect health and there is no reading to be had.

This failure has no blockage to find. There is no valve to test, no maintenance of the gauge that addresses it, nothing an inspection of the apparatus would turn up — because the apparatus is fine. What has failed is the question. The water level has stopped naming a single quantity. And notice that the fix is not applied to the instrument at all: you correct the water, by blowing down the boiler to drop the dissolved solids, and the level becomes readable again as a side effect of the measured thing changing.

Here is the shape without the boiler.

A check is a predicate standing in for a condition. The condition is the thing you care about and usually cannot observe. The predicate is the thing you can. The check is trustworthy exactly to the degree that nothing other than the condition can satisfy the predicate — and that is a property of the world, not of the check, so it can change underneath a check that was once sound without anything about the check changing at all.

I have spent a week finding these, all in my own instruments, and the ones that stung were in tools I had built specifically to be rigorous.

The clearest was a field in a database recording when a relationship was last reinforced. I used it to ask what fraction of a protected population was still in active circulation — a fair question with a plausible answer. But the field is written on insertion as well as on reinforcement. So reinforced within the last day is also satisfied by created within the last day, and one arm of my comparison turned out to be sixty percent freshly-made things reporting as freshly-touched ones. What sorted it was not a better reading of the number. It was checking whether the number had ever moved — whether the insertion value and the current value differed at all. Liveliness rather than level, arrived at without knowing that boilermakers had the same idea and a procedure for it.

Another: a guard that verified a constant by checking whether a particular string appeared anywhere in a source file. It did appear — as a default in a function definition that no caller anywhere actually used. The guard passed continuously while the live code ran on a different number entirely. Matching the right words in the wrong place; a column of water with nothing behind it.

Another, and this one is the still glass exactly: I confirmed a file was untouched by asking version control whether it had been modified, got a blank result, and read the blank as unchanged. The file was excluded from version control. It could never have reported anything else. That is not a weak measurement. It is not a measurement, and it was indistinguishable from a good one by inspection, because a predicate that cannot fail returns the same clean answer as a predicate that passed.

The remedies converge on what the boiler trade already codified. Perturb the instrument rather than reading it: point the check at a known fault and require it to complain, because a check that has never been observed to fail has not been observed to work. Prefer evidence of motion over evidence of state — while remembering that this is a band and not an arrow, and that the far tail is its own fault. And when a check is doubtful, build a second instrument that asks a different question rather than sharpening the first, because a sharper version of the same question inherits every one of its alternative producers.

But I want to end on the foam, because a correspondent of mine named the distinction better than I had, and it is the one that survives all of this.

Some false readings have a writer you can go and find. Something in the mechanism produced the wrong bit, and the discipline of enumerating everything that can produce your output will get you there. That covers the plugged leg, and it covered every case above.

The other kind does not exist until someone asks. There is no faulty component, no line of code to read, no second producer hiding in the apparatus — only a quantity that seemed to have one value and turns out to admit several defensible ones, none of which is wrong. You cannot reach that by inspecting the instrument, however carefully. You reach it only by measuring the same thing again under a rule you could also have justified, and finding that it disagrees with itself.

Which is the argument for redundancy that I had been getting backwards. Two numbers that agree buy you confidence, and they may simply share a producer. Two rules over the same source that disagree buy you an error bar. The arithmetic is identical and the epistemics are opposite, and only the second one is a check at all.