This Week's Citation Classic

Publish PostHurlbert S H. Pseudoreplication and the design of ecological field experiments. Ecol Monogr. 54:187-211, 1984.

It's not often a paper claiming that half of all papers published in ecology are crap is cited more than 2,880 times. But that's exactly what Stewart Hurlbert did. He pointed out that a significant fraction of papers published in ecology suffered from poor experimental design and inappropriate statistical analysis.

The main problem is pseudoreplication, which is non-independent measurements of any phenomena.

As Hurlbert (1984) puts it:

“Pseudoreplication is defined as the use of inferential statistics to test for treatment effects with data from experiments where either treatments are not replicated (though samples may be) or replicates are not statistically independent.”

For example, lets say you want to see if there is a relationship between diet and weight in frogs. You feed 10 frogs either 1 gram or 2 grams of flies a day. Over the next month, you weigh the frogs once a week. Then you analyze your data via an analysis of variance (ANOVA) with weight as the dependent variable, food level as a treatment and find that the high food frogs gained more weight than the low food frogs. So it must be the treatment right? Wrong! This is pseudoreplication. Weighing the same frog four times violates the assumption of non-independence in an ANOVA. Therefore, this ANOVA may underestimate your probability value (P-value or the probability of obtaining a result at least as extreme as a given data point, assuming the data point was the result of chance alone) by an order of magnitude or more. Thus you may be claiming there is a relationship between the two variables when in fact none exists.

If you read the old literature, or even some of the new literature, you see pseudoreplication over and over again. However, the problem is easily correctable by conducting a repeated measures ANOVA or by "nesting' your measurements.

Hurlbert had this to say about writing the paper:

"Some minor additional revision and expurgationwas requested. Prompted by a reviewer's charge that I used "private language and humor by misstatement," the managing editor, Lee Miller, questioned some of my metaphors such as "demonic intrusion" and "biting the bullet as well as the apple," and the copy editor, Sarah Gagnon, thought I was pushing matters by acknowledging the helpful comments of colleagues and then saying that "Any errors that remain are their responsibility and theirs alone." In the end, they kindly relented and let me have my fun."

Ahhh the good ole days. I seriously doubt anyone could get away with such humor today.

On a personal note, I was first introduced to this paper in my first semester in grad school in a seminar taught by Ed Garton, Advanced Population Biology. Never mind that I had never taken Intermediate Population Biology or even Population Biology for Dummies, I signed up thinking, how hard could it be? My cheeky attitude soon turned to outright fear and trembling as I gawked at the foot high stack of required literature readings and I gaped at the comments by one of my classmates, the long-haired Ryan J. Monello, during our first class. What was I doing in this class of ubermenschen such as Monello and John Pearce? Obviously I better tuck my tail between my legs and back out of this mess. Turns out Monello was bluffing; he didn't know anything about population biology (some would argue he still doesn't :P ). I ended up learning a lot of great biology, doing fairly well in the class (A or A- I think) and Monello became one of my best friends.

Photo: Hurlbert rafting on Laguna Colorada, Bolivia via his website.