Just How Corrupt is the Peer Review Process?

Welcome to another Still in the Storm Substack post.

Today’s post is a continuation of my expose into the current state of scientific research and the massive amount of corruption therein.

Hope you enjoy and if you find it valuable please share so more can get this critical information.

Let’s dig in.

You just heard that a new study has come out with promising results on a potential breakthrough new cancer treatment.

The majority of people, scientists included, will accept that study at face value simply because it has been rubber stamped by having successfully passed through the peer review process.

But, just what is the peer review process and how is its corruption threatening to take down science?

Join me as I shed light on yet another staggering problem within scientific research today.

Always try to stand on the shoulders of greats, not rewrite them.

The logical place to begin would be with a few questions. What is peer review, why do we need it and what is wrong with it?

As I was starting to write this article and do some research, I realized quickly that in no way can I address this topic better than the work done by Richard Smith, the former editor of the British Medical Journal (BMJ).

Richard has written extensively on this topic and in a very clear manner.  

So after some contemplation I decided rather than re-writing what he has already done so well I would simply try to stand on his shoulders and add my perspective to his excellent body of work.

Subsequently, if you want to get some background on the topic of peer review I would highly suggest reading some of his writings on the topic.

And a piece that I would strongly recommend starting with is, “Peer review: a flawed process at the heart of science and journals”.

To give you a little flavor, I will share the opening paragraph below:

“Peer review is at the heart of the processes of not just medical journals but of all of science. It is the method by which grants are allocated, papers published, academics promoted, and Nobel prizes are won. Yet it is hard to define. It has until recently been unstudied. And its defects are easier to identify than its attributes. Yet it shows no sign of going away. Famously, it is compared with democracy: a system full of problems but the least worst we have.” (Smith, 2006)

If you want to take a detour and read that article, I won’t be offended but, please come back and finish this one too because now I’m going to share my perspective and you don’t want to miss it.

The pharma companies own the journals. What could possibly go wrong?

Yes, you read that right.  One of the more surprising issues that I’ve observed over my career as a scientist has to deal with who owns or funds most of the journals themselves.

And you’ll never guess who.

Ok, you probably did guess it but, in case you didn’t I’ll tell you anyway.

It’s big pharma!

Talk about the mother of all conflicts of interest, right?

Unfortunately, it turns out that many journals are nothing more than mouthpieces for the big pharma and biotech companies.

This has created a substantial challenge with getting original research published, especially if it isn’t in line with what the industry wants or even highlights a problem with previously accepted research.

That is clearly a no, no.

You may have even noticed the almost stunning lack of published negative studies.

As Richard Smith says, “The editorial review process has been strongly biased against ‘negative studies’, i.e. studies that find an intervention does not work. It is also clear that authors do not even bother to write up such studies.”

In addition to this, several different types of prominent scientific articles that appear in journals are directly driven by invitations from the editor of the journal.

Do you not think they’d use that to influence your thinking on certain topics or elevate certain one’s over others?

It is my hope here to shed some light on why that is, but just how did we get here.

It was spawned by the system itself

Yes, you read that correctly.  The system itself created and directly perpetuates this corrupt behavior.

Publications are like currency in science, especially in the academic world.

They can literally mean the difference between career advancement and gaining grant funding or failure and ostracization.

And the more you publish the better.

This has become known as ‘publish or perish’.

Publish or Perish is at the root of the problem

It is exactly what it sounds like.

Either you publish, and a lot, or you perish.  What do I mean by perish?

Well, you don’t get to build your CV with a body of peer-reviewed original data.

This leads to you not getting promoted or even a new job.

It will also lead to the loss of a grant and the associated funding award(s).

Most academic labs live and die on grant funding so, failure to publish could literally put the future of the lab in jeopardy.

Are you starting to see why there is a strong incentive to get as much published as possible even at the expense of the validity or replicability of the findings?

If so, that’s good because this is at the heart of the problem.

It’s all about incentives and while there are plenty toward the continuation of this paradigm there are a scant few that push against it.

And that is why despite more and more scientists becoming aware of problems with peer-review as well as validity of findings and reproducibility of data there has been very little progress made to change it.

Implications? Yes there are many.

The implications of the corruption in peer-review are many.

First, a lot of data is being published that is not valid or at least has a much higher probability than not of being false.

This impacts much of current and future scientific research, including drug discovery and development work.

In many cases, scientists are accepting primary pre-clinical data at face value just because it has cleared the peer-review process.

The problem is that if this data is not valid and is being used to gain approval for or to design clinical trials then we are putting experimental drugs in sick people without the data to back them up.

This is corrupt, immoral and possibly even criminal.

As well, we have a peer-review system that can promote or suppress data and information at will.

The journals themselves are driving this and when you see where they get their funding from you will understand how and why.

A personal anecdote

With more than 20 years doing scientific research I’ve seen this process unfold more than a few times.

I have seen manuscripts submitted for publication with data that is either knowingly questionable or at least highly cherry picked.

Yes, scientists often finesse or massage data to get it into what we call “presentation or publication quality”.

This typically involves the removal of data points deemed to be outliers that could conflate an interpretation or conclusion that is being made.

Oftentimes when you see in a paper that a figure is representative of the data obtained from multiple experiments it is just the best one or the one that most fits the predetermined conclusions.

The problem is that the reviewers don’t know this and they typically are not even sent the raw data to confirm any of it.  They simply assume that any validation would have been done prior to submission of the paper for peer-review.

Another thing I’ve seen is a wide disparity in the quality of review comments that are received.

Some will sign off on a paper or even reject it with seemingly little rationale.  Others will write an extensive critique.

It would be great if the latter was more common but unfortunately that is not the case.

As well, I’ve observed many scientists that get more than a little upset when you get a review that dares to try and poke holes in their data.

How dare they do a thorough job?  Who do they think they are?

Scientists tend to be a wee bit protective of their work and thus challenges are taken quite personally.

Again, this plays back into the idea of publish or perish.

Early in my career I had the pleasure to work for a PI that was highly critical of the publication process.  He saw clearly what most scientists were doing and even coined a new term.

The ‘least publishable unit’ is the smallest allowable content or data that can be published.

The ideal route is to wait until you have a sufficient amount of data to tell a complete story in your paper.

The fury to publish has led a lot of scientists to take what would or should be one paper and chop it up into as many as they can possibly squeak out.

One last thing I’ve seen that will probably open some eyes…

Would you believe that it is not uncommon to effectively write a paper, or at least the story you want to tell, before you have generated all the associated data?

What could possibly go wrong?  I mean it’s fine to completely bias your experiments in favor of a perdesired or predetermined outcome, right?  Wrong! 

Is there a path forward?

Let’s wrap this up by considering the path forward.

First, is there presently one and if so what does that look like?

A number of good people like Richard Smith have written extensively on the topic of corruption in peer-review for a while yet not much has changed.

As I mentioned previously, it largely comes down to incentive and there just isn’t enough to move the needle.

If it doesn’t change we will only continue the cycle of corruption and to further muddy the waters of scientific research.

Personally, I’d propose opening up the process to something more public and transparent.

The bottom line is that we need to hold the reviewers and authors themselves accountable.

Receipts

1. Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99(4), 178. https://doi.org/10.1258/JRSM.99.4.178

---

I greatly value your feedback and suggestions so, please let me know what you think in the comments below:

Do you know someone that could benefit from this?

If you find the information in these articles valuable, we would be grateful for your help to get it to those who could most benefit. 

Just click the button below to share and restack today’s post!

Get notified when new posts go live!

Lastly, to be notified as soon as a new post is live go ahead and subscribe to Still in the Storm by clicking the button below.

Have You Checked Out Science Defined Yet?

Learning science is an invaluable skill in these times.

Sign up now for our newsletter and you’ll get a FREE scientific term and tips delivered directly to your inbox each week.

You learn a new term and you get to study it on your own time.  It’s a win/win.

Click on the button below to sign up now:

Here is where you can find and/or follow Science Defined online to keep up with the latest updates as we continue to roll out our platform: