Now you see it, now you don’t

Science should be all about transparency, but apparently that’s not always the case. Why does it matter? Well basically because folks need to have the opportunity to review the details of studies that have been done and to criticize and rip them to pieces if the study is in any way dubious or simply wrong.

Normally what happens is that when a study has been completed, it gets submitted to a recognized peer review journal that specializes in the subject that was being studied. The editors then arrange for knowledgeable peers to review and approve or reject it. If its approved, others then have access to the details and can attempt to duplicate the study, or to simply cite it within other studies that then build upon this new knowledge.

So far … so what … this is how it works. Ah but, what happens when a peer reviewed paper is retracted. The gap is that we never get to find out why, the journals simply don’t make that information public. Ben Goldacre has highlighted this problem within his Guardian article today. Here is how he explains it …

In 2004, the Annals of Thoracic Surgery (ATS) published a study comparing two heart drugs. This week it was retracted. Ivan Oransky and Adam Marcus are two geeks who set up a website called Retraction Watch because it was clear that retractions are often handled badly; they contacted the editor of ATS, Dr L Henry Edmunds Jr MD, to find out why the paper was retracted.

“It’s none of your damn business,” replied Edmunds, before railing against “journalists and bloggists”. The retraction notice, he told them, is merely there “to inform our readers that the article is retracted. If you get divorced from your wife, the public doesn’t need to know the details.”

Yikes … so that then begs these questions … “Why was it withdrawn?” and also “What about other papers by the same author, can we still trust those?”

What could have happened is anything such as fraud, or simply the researchers making a discovery that their tests were flawed and simply requested it be withdrawn. Either way, its important to know. Another hidden ripple effect here is that retracted papers are also often still cited by other papers, because the authors are unaware of what has happened and still believe it to be correct.

So where are we now? … Well, Ben puts it like this ..

The role of journals in policing academic misconduct is still ill-defined, but clearly, explaining the disappearance of a paper you published is a bare minimum.

Like publication bias, where negative findings are less likely to be published, this is a systemic failure, across all fields, so it has far greater ramifications than any one single, eyecatching academic cockup or fraud; unfortunately it’s also a boring corner in the technical world of academia, so nobody has been shamed into fixing it.

You can read his full article here.

You might also like to check out the Retraction Watch blog here (these are the folks who attempt to find out why a paper has been withdrawn).