On Scientific Method: Comment on Hawkins

“’Unaccountable statistics’ [are] statistical goulash that sounds tangy and sophisticated but is actually bereft of substance, and used to make predictions that are almost never accounted for. Any number of ‘scientific’ renewable energy reports, from NREL to Stanford to MIT, are of this kind.”

Kent Hawkins’ post yesterday, “Science, Advocacy, and Public Policy,” defends the scientific method against both political correctness and the misuses of the method, often by people who claim to be scientists. This is a major issue in the current energy and climate debate where exaggeration and bias go hand-in-hand. I wish to add support to Hawkins’s theses in light of some of science’s nuanced complexity.

Scientific Inquiry

Here’s how a few mainly twentieth century scientists defined the purpose of scientific inquiry:

“Science is the disinterested search for the objective truth about the material world.”–Richard Dawkins;

“The less one knows about the universe, the easier it is to explain.”–Leon Brunschvicg;

“Theories crumble, but good observations never fade.”—Harlow Shapley.

For those interested in a brief but more expansive discourse on what scientists are obligated to do as they pursue any inquiry using the tools of the scientific method, read Richard Feynman’s “Cargo Cult Science.” Matt Ridley also has a good essay on the subject, “What the Climate Wars Did to Science.”

These thinkers emphasize several things:

(1) It’s not about information per se; but rather the quality of information.

(2) It’s not a thing but a methodological process in search of finding the better idea.

(3) All conclusions reached using the scientific method are always provisional.

(4) Any scientific inquiry should be engaged in the spirit of skepticism and disinterest. Any perceived bias, including the bugbear of self-interested activism and appeals to authority should be vetted and accounted for before valid conclusions are reached (any magician, for example, knows how easy it is to fool especially intelligent people who think they are privy to a particular truth).

(5) Any scientific conclusion about material reality should have accountability in the here and now: it should be falsifiable, if it is false, and it should be repeatable by others (as Kent emphasizes).

Scientific inquiry, by its nature, is often a dialog between the prescriptive and the descriptive. Although policy recommendations made by scientists are prescriptive, typically devoid of skepticism and disinterest, the real prescriptive nature of science resides in the foundational tool of the method: the hypothesis, which is a provisional conditional, that is, an “if/then” proposition in need of evidentiary support in the real world.

If the prescribed alignment of the “if” factors are true, leading to an accurate prediction about the nature of the “then,” the hypothesis can be considered a functional prescription for the nature of reality in this particular narrow forum. If related hypotheses are validated thusly, especially across a range of disciplines, the hypotheses collectively can be raised to the level of a theory–a general conceptual tool for analyzing, predicting, explaining behaviors in the natural world Theories have a welter of complementary empirical support across many fields of study.

So prescription, as a means of making informed but accountable predictions, is fundamental to how science works well; prescription as a means of selling soap is problematic, but not necessarily “wrong” because it still may point to a particular truth–which can be accountably validated. Or not.

Facts are Contextual

All facts are contextual. What works in one area from one perspective may not work or be true in another. Points of view–and describing them accurately–are lynchpins of good scientific inquiry.

Examples of this are legion. They are pivotal when accounting for the differences between relativity and quantum theory, for example, and for understanding why Newton’s gravitational mechanics must defer ultimately to those of Einstein’s General Theory. Moreover, definitions about terms and processes should be consistent and accounted for as they are loosed on a particular inquiry. In general, this is known as the “operating definition” issue.

Definitions that slip and slide across an explanatory landscape–that mean one thing but then are used without explanatory context to mean something different or even opposite are one of the first clues that a “scientific” claim may be bogus. Witness the MIT report on the future of solar….

Statistics & Cargo Cult Scientists

I want to end by briefly discussing one of my major concerns about the way cargo cult scientists, particularly those bellying up to the trough of academic or government institutions, with their trove of public dollars to dispense, make use of statistics.

Statistics are now so pervasively part of our culture, technology, and language that most of us take the term for granted, so much so that we don’t even examine what we mean by statistical ideas anymore. How many know the difference among mean, median, and average, for example? Or realize that our electronics, including television, computers, medical diagnostic machines, are based upon accountable statistical projections embedded in quantum mechanical algorithms.

Or understand that Boyle’s Law, that bedrock equation governing the relationship between pressure and volume, is rooted in a statistical appreciation of how individual atoms, by themselves behaving unpredictably, are herded by group behavior into breathtakingly predictable statistical flocks sufficient for us to build mighty civilizations upon them.

Statistical methods for assessing the physical conditions sufficient for predicting short-term weather have also been extremely useful in accountable ways. Beyond this, there are the accountable social statistics behind insurance actuarials, polling projections, and Madison Avenue ads. Although predictions in this arena leave some greater margin for error, their utility, for good or ill, have made fortunes and provided for social and economic stability.

On the other hand, there has also been a lot of flimflam associated with statistical usage, as evidenced by the Mark Twain quote, “There are three kinds of lies: lies, damned lies and statistics.” I’ll never forget Darrell Huff’s marvelous tract, “How to Lie with Statistics,” with its shorthanded introduction to some of the ways the ignorant are gulled by the crafty and craven.

However, I’d like to add another category that I think better conveys my concern: “Unaccountable statistics”–statistical goulash that sounds tangy and sophisticated but is actually bereft of substance, and used to make predictions that are almost never accounted for. Any number of “scientific” renewable energy reports, from NREL to Stanford to MIT, are of this kind.

But there’s a raft of others, including most epidemiological analyses, nutritional studies, climate change modeling (don’t you love those 100-year out unaccountable predictions?), and, of course, the investment wizards of Wall Street. Perhaps someone one day will write a “How to Lay with Statistics” as a means of exposing the trickeration of statistical unaccountability (with no apologies to Key Lay).

Occam’s Razor

I can’t end this discussion without at least some mention of Occam’s Razor, which is also known as the principle of parsimony. Among competing hypotheses, scientists prefer the one with the fewest assumptions that explain all the known facts. Other, more complicated solutions may ultimately prove correct, but—in the absence of certainty—the fewer assumptions made, the better.

Elegance is another descriptor of this idea. All this can become complicated and is mostly conditional. However, the preference for simplicity depends upon falsifiability. For each accepted explanation, there is a wide range of possible and more complex alternatives, because one can puff up failing explanations with ad hoc hypotheses as a hedge to keep them from being falsified; therefore, simpler propositions are preferable to more complex ones because they are better testable and falsifiable.

The fly in the ointment of explanation, which is really the point of the scientific method (making things plain), is that, as context is enlarged and expanded, newer “facts” often buzz up and over the boundaries of conventional explanations. And, of course, the new evidence must be accounted for.

———-

A lifelong environmentalist and resident of Maryland, Jon Boone helped found the North American Bluebird Society, became an associate editor of Maryland Birdlife, and is a consultant with the Roger Tory Peterson Institute in Jamestown, New York. He paints in transparent watercolors and has written extensively about energy policy and the Dutch master, Johannes Vermeer.