Wednesday, October 19, 2011

The need of true transparency

A recent article in the Journal of Pediatrics (1) stated that 1 in 10 parents do not completely vaccinate their kids. The MMR vaccine was cited as the vaccine withheld most frequently. This is most likely due to the infamous 1998 manuscript in The Lancet stating that there is a “link” between the MMR vaccine and autism. Though the original manuscript has since been retracted from The Lancet due to falsified data, its effects are still lingering. There are children out there that are not getting vaccinated due to this bad science.

Bad science does not just impact researchers’ understanding; it impacts people’s lives. This study illustrates what can happen when articles are published, but are then retracted after a larger audience has vetted the ideas therein.

I wish I could say that retractions are a relatively rare occurrence. Sadly, this is not the case.

A website called Retraction Watch publishes cases like this on a very regular basis. Retractions are posted from journals across the scientific gamut. You might be thinking that only minor journals, with a low impact factor, would be the major culprits. This is most definitely not the case. Major journals like Nature and Science are featured frequently.

The interesting thing is that the scientific community always vilifies the authors. However, there is no real outcry against journals publishing, then retracting, articles with spurious findings.

This is wrong.

There should clearly be some blame passed to the journal publishing the manuscript.

This goes back to one of our guiding principles at OSK: transparency.

The scientist’s book of holy writ is the peer-reviewed journal; once an article is published in a journal, it then, in essence, becomes “canonized”. However, there is no real discussion about the merits and flaws of this system. We need to take a deeper look into what actually constitutes peer-reviewed publishing. We need to ask ourselves if there is a better way to do this and why we did not seek it in the midst of obvious flaws in our publishing system.

Currently all publication requires is an “okay” from a handful of peers. Similarly, a handful of “not okay”’s can condemn an article to rejection. This small number of reviewers leaves open the possibility of numerous biases; we rigorously test for these biases in statistical analysis, but never inquire about them in peer review. We similarly never inquire about the possibility of dissent among reviewers, leaving our faith in the journal to give us only manuscripts that are unquestionably of the highest quality.

This whole process needs to be turned on its head. Most directly, we must include more reviewers in peer review. Dissenting reviews should be provided for the reader, as they offer an important tool with which to understand the context of a published manuscript. Probably more importantly, all data used to make conclusions should be subject to review, as it is fundamental to the validity of a manuscript. Limitations must be emphasized, as they are the most important part of any manuscript!

Once these solutions are implemented, the retraction rate will decrease. Ultimately, this will impact not only the researchers, rewarding truly high-quality research, but also the people their research is likely to affect.

I invite your comments below

1. Dempsey AF, Schaffer S, Singer D, et al. Alternative vaccination schedule preferences among parents of young children. Pediatrics 2011 Nov;128(5)

1 comment:

  1. Being a soon to be parent, I have a better understanding of the intricacies related to these studies affecting new parents. It’s really scary to begin with, then an added confusion on top of that stemming from patients trusting doctors(which I whole heartedly believe we should do). If one doctor shows one result, ie vaccines cause autism, and another shows the opposite, who do you trust?

    I believe this step is a crucial point at which peer review holds the most meaning. Peer reviewers need to let everyone else know who to trust and it needs to be done right the first time with as you say, transparency. People need the confirmation from peer reviewers and the ability to do their own quick review of the information comparing good sides and bad. There seems to be a connection at this point in my mind with informed consent. A doctor needs to explain risks and benefits, complications, outcomes, etc. with a patient. This process may only take several minutes but needs to be there to insure the best standard and quality of care for the patient. Why don’t we have “informed consent” for these studies in the form of open peer review where all the strengths, benefits, faults, risks and limitations are brought into the open just as doctors explain these items before a procedure or treatment? After all, the patients are the ones experiencing the results whether it is from the immediate procedure or the results of comprehensive, cohort studies performed at far off institutions.


Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 3.0 United States License.