Message #1412

[Lisa Wolfe]

In recent weeks science bloggers, researchers and assorted skeptics have challenged Dr. Bem’s methods and his statistics, with many critiques digging deep into the arcane but important fine points of crunching numbers. (Others question his intentions. “He’s got a great sense of humor,” said Dr. Hyman, of Oregon. “I wouldn’t rule out that this is an elaborate joke.”)

Dr. Bem has generally responded in kind, sometimes accusing critics of misunderstanding his paper, others times of building a strong bias into their own re-evaluations of his data.

In one sense, it is a historically familiar pattern. For more than a century, researchers have conducted hundreds of tests to detect ESP, telekinesis and other such things, and when such studies have surfaced, skeptics have been quick to shoot holes in them.

But in another way, Dr. Bem is far from typical. He is widely respected for his clear, original thinking in social psychology, and some people familiar with the case say his reputation may have played a role in the paper’s acceptance.

Peer review is usually an anonymous process, with authors and reviewers unknown to one another. But all four reviewers of this paper were social psychologists, and all would have known whose work they were checking and would have been responsive to the way it was reasoned.

Perhaps more important, none were topflight statisticians. “The problem was that this paper was treated like any other,” said an editor at the journal, Laura King, a psychologist at the University of Missouri. “And it wasn’t.”

Many statisticians say that conventional social-science techniques for analyzing data make an assumption that is disingenuous and ultimately self-deceiving: that researchers know nothing about the probability of the so-called null hypothesis.

In this case, the null hypothesis would be that ESP does not exist. Refusing to give that hypothesis weight makes no sense, these experts say; if ESP exists, why aren’t people getting rich by reliably predicting the movement of the stock market or the outcome of football games?

Instead, these statisticians prefer a technique called Bayesian analysis, which seeks to determine whether the outcome of a particular experiment “changes the odds that a hypothesis is true,” in the words of Jeffrey N. Rouder, a psychologist at the University of Missouri who, with Richard D. Morey of the University of Groningen in the Netherlands, has also submitted a critique of Dr. Bem’s paper to the journal.

Physics and biology, among other disciplines, overwhelmingly suggest that Dr. Bem’s experiments have not changed those odds, Dr. Rouder said.

So far, at least three efforts to replicate the experiments have failed. But more are in the works, Dr. Bem said, adding, “I have received hundreds of requests for the materials” to conduct studies.