Caution: Exercise Extreme Discernment
The Swedish mRNA vaccine study and the reproducibility crisis
Today’s article is yet another science one. Yes, I know that I keep posting about science stuff despite promising other content as well. We will get to some other topics but, this one was too important to pass up. You see science is knowledge, literally. The word ‘science’ is derived from the latin scientia meaning knowledge. More completely, its knowledge of our world within and around us which is understood through experimentation and observation. To gain a complete understanding of the world we exist in requires asking a lot of questions and using plenty of discernment. Only when we understand the amount of power that comes with acquiring this knowledge can we see why we must not give it up any longer. We must hold an open mind as things will change and new evidence will be presented. At the end of the day, the goal must always be getting to the truth.
So, why is all this important? Consider the reproducibility crisis.
What reproducibility crisis you say? Here reproducibility is exactly what it sounds like. This refers to the ability of an external lab to replicate primary results published in a peer-reviewed paper. At this point, more cannot be reproduced than can, by a lot. In fact, it’s highly probable that findings in most published studies are not only false, but really just measures of the prevailing bias. One of the first people to call into question the validity and reproducibility of published data was John Ioannidis, an epidemiologist currently at Stanford Medical School. In his landmark 2005 essay in PLoS Medicine entitled, “Why Most Published Research Findings are False”, he went right at this question (Ioannidis, 2003).
“There is increasing concern that most current published research findings are false.”
“Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias.”
“It can be proven that most claimed research findings are false.” (Ioannidis, 2003)
Two large biopharma companies actually put numbers to Ioannidis’ claims. Many times companies try to replicate published studies in their search for new targets to fill their pipelines. Bayer Healthcare noticed a significant amount of studies were not successful in their scientists’ hands so, they polled them. Approximately 80% were not successfully reproduced (Bayer, 2012). Amgen, a large biotech company, conducted a formal study of 53 landmark papers. Only 6 could be replicated (Amgen, 2013). That’s a whopping 11%, which means almost 90% couldn’t be replicated. These reports came out around 2012 so, why is it still an issue? Most scientists are aware of the reproducibility crisis but, they don’t see why failure to replicate data means it’s invalid or that it means the system is flawed. In fact, a vice president at Genentech actually said, “Non-replication does not necessarily mean ‘not true’”. Say what? It turns out that replication is one of the best ways we can validate another’s work, making it highly likely that non-replication does mean ‘not true’. In Ioannidis’ essay, one of the specific things he mentions that leads to an increased probability that reported claims are more likely to be false is a low sample size and/or a small change in activity. He also brings up a trick whereby scientists utilize a formal statistical significance to demonstrate validity of findings. The problem is that this completely ignores functional or biological significance of the data. More times than not a small change in activity is not meaningful regardless of statistical significance.
I’m sure by now you have heard about the recent study from a group at Lund University in Sweden that claims that the mRNA in the Pfizer/BioNTech COVID vaccine, BNT162b2, is integrated into liver cells and replicated. The title of the paper by Alden et al. is “Intracellular Reverse Transcription of Pfizer BioNTech COVID-19 mRNA Vaccine BNT162b2 In Vitro in Human Liver Cell Line” (Alden, 2022). Their claims based on the data presented in this paper are as follows:
“We detected high levels of BNT162b2 in Huh7 cells and changes in gene expression of long interspersed nuclear element-1 (LINE-1), which is an endogenous reverse transcriptase. Immunohistochemistry using antibody binding to LINE-1 open reading frame-1 RNA-binding protein (ORFp1) on Huh7 cells treated with BNT162b2 indicated increased nucleus distribution of LINE-1. PCR on genomic DNA of Huh7 cells exposed to BNT162b2 amplified the DNA sequence unique to BNT162b2. Our results indicate a fast up-take of BNT162b2 into human liver cell line Huh7, leading to changes in LINE-1 expression and distribution. We also show that BNT162b2 mRNA is reverse transcribed intracellularly into DNA in as fast as 6 h upon BNT162b2 exposure.” (Alden, 2022)
This comes on the back of a previous study from Zhang et al. out of Harvard which claimed that “SARS-CoV-2 RNA can be reverse-transcribed and integrated into the genome of human cells” (Zhang, 2020). The two studies make significant claims about the ability of RNA to be reverse transcribed and integrated into cellular genomic DNA. If true, it would completely refute what the pharmaceutical companies developing mRNA vaccines previously said when they claimed it wasn’t possible. In order for this to be relevant the supposed SARS-CoV-2 virus has to be real but, best I can tell there is no clear proof of that whatsoever. If the virus doesn’t exist then is there actually a “spike” protein out in nature somewhere or was it created de novo as the virus most likely was? Of course, you’d also have to be able to prove that it’s possible for any of this to actually occur in an intact living human being but, that’s semantics and possibly topics for future posts. These are important considerations that should be discussed but, here I am just focusing on the paper by Alden et al and the validity of their claims.
Let’s now dive into the paper and carefully examine the methodology employed and the data that they are deriving their conclusions from. To summarize, they are claiming that the mRNA in the BioNTech vaccine gets reverse transcribed into DNA and then is replicated by the cells seemingly via integration into the genomic DNA. The first problem is that they are using a liver cancer (hepatocellular carcinoma) cell line. For one, cancer cell lines rapidly proliferate. This means that they grow really fast. Much more so than cells of normal, healthy tissue. They also have no bearing on what happens in a healthy liver cell in an intact human being, and this assumes that the liver is made up of individual cells or hepatocytes. So right off they are utilizing a model system that tells you nothing about reality. Then they are measuring the level of mRNA transcripts by PCR and suggesting that an increase over time is the result of the cells reverse transcribing it into DNA and integrating it. As far as I can tell they aren’t even attempting to sequence the genomic DNA to show that anything new has been integrated. Next, they “prove” an increase in reverse transcription by staining cells at various time points with a supposedly specific antibody to a certain type of reverse transcriptase, LINE-1, that the cells are supposed to express. Reverse transcriptase (RT) is the enzyme responsive for converting RNA into complementary DNA (cDNA). It was originally thought that only retroviruses encoded RTs and this was used as a marker for infection. The problem is that it was later discovered that all cells encode for RTs so, it really didn’t tell you anything about retroviruses if you happened to find them in cells. Moving on, they see what they believe to be an increase in the amount of the LINE-1 reverse transcriptase but that doesn’t necessarily correlate with an increase in activity which is what you’d really be looking for. So, it’s an artificial system with some poor surrogate readouts.
And that was just from looking at the methods. Next, I looked at the actual data. It appears that their key claims depend on statistical analysis and they lack the proper controls. Best I can tell, the only control employed is untreated cells or “CTL”. How can you possibly know that anything else in the vaccine wouldn't cause the effects they are seeing? Without proper controls to rule everything else out their conclusions are questionable at best but, theres more... So, technically they are seeing a significant increase in the BNT162b2 mRNA levels after 6h and at 24 and 48h. The problem is they are showing three different concentrations of the BNT162b2 vaccine so, you should see somewhat of a concentration dependent effect and you don't. Also, it wouldn't be the same at all three timepoints. The third figure where they are trying to claim that the LINE-1 reverse transcriptase expression is significantly increased after 6h seems to be dependent on statistical significance. The data has also been normalized which, more times than not, tends to make an apparent increase in activity look much bigger than reality.
Figure 3. Relative (normalized) LINE-1 expression by PCR.
First of all, its just based on RT-PCR of the transcripts they claim encode LINE-1. They believe that there is statistical significance, however based on the spread of the replicate data points I highly doubt it. Also, even if its technically significant it’s a very small change. How does that have anything to do with function? I saw this A LOT where there was such an over reliance on stats and I'd consistently argue that such as small change is meaningless in terms of biological function.
Figure 4. LINE-2 expression by staining.
Then in the fourth figure they are staining the cells for the LINE-1 protein and again claiming statistical significance. This is entirely depended on the specificity of the antibody used to stain for LINE-1. As well, the process of staining cells requires significant manipulation of the samples and treatment with harsh chemicals. Its clear that the samples are changed in this process and it cannot be ruled out with the controls they used that artifacts weren’t generated.
Figure 5. Gel images of DNA amplicons of BNT162b2 by RT-PCR.
The fifth figure is literally just gels of PCR products detecting amplicons of genomic DNA with primers for a region of BNT162b2. How can you possibly say that is definitely coming from the mRNA being integrated and not just something in the DNA. There does appear to be an increase in the mRNA vaccine treated cells but, its not quantitative. Again, I’d suggest that the controls were not sufficient.
This is so typical of what I have seen throughout my career. Its all stats and small effects. These are also two key items that Ioannidis identified that increase the probability that a given finding is false. Oh and one last thing... These data are from 5 independent experiments (or so they claim) so, this is either an aggregate or the best of the 5. Most papers aren't showing representative data. They cherry pick the best ones. They call it "presentation quality". They will also arbitrarily remove outliers within a data set to "massage it" or clean it up to make it look better. I’ve seen this done firsthand, multiple times. Not only that, but I’ve seen where data was used from a single experiment. The system breeds scientists who care more about getting published than the integrity of their data.
The bottom line is that I do not believe that the methodology employed is sufficient to determine the validity of their hypothesis nor can the data be used to back up their claims. This rests largely on the fact that they don’t have all of the appropriate controls and the changes that they are seeing in key datasets are very small. Ultimately, the claims hinge on statistical significance which is weak considering the surrogate readouts they’ve used. It’s very hard to determine whether there is anything of meaningful biological consequence within the data.
To be clear, my point is not to be critical of the authors themselves but, to encourage careful examination of their data. The studies from Bayer and Amgen, in addition to my own observations over my 20 year career, suggest that Ioannidis’ claims are spot on. If we are to continue to use peer-reviewed published papers we must utilize extreme discernment. I humbly request that when any new study comes out making significant claims that you apply a critical eye to the data before drawing conclusions. The devil is truly in the details and we must be empowered to be the arbiters of truth. Have confidence to research things yourself so, you don’t have to just accept the claims at face value. Our very future hangs in the balance.
Jump into the discussion below in the comments section with your own thoughts and analysis of the paper and more!
Note - all figures are from Alden et al. (2022).
Science. science noun - Definition, pictures, pronunciation and usage notes | Oxford Advanced Learner's Dictionary at OxfordLearnersDictionaries.com. (n.d.). Retrieved March 24, 2022, from https://www.oxfordlearnersdictionaries.com/us/definition/english/science?q=science
Science: Search online etymology dictionary. Etymology. (n.d.). Retrieved March 24, 2022, from https://www.etymonline.com/search?q=science
Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), 2–8. https://doi.org/10.1371/JOURNAL.PMED.0020124
Prinz, F., Schlange, T., & Asadullah, K. (2011). Believe it or not: how much can we rely on published data on potential drug targets? Nature Reviews. Drug Discovery, 10(9), 712–713. https://doi.org/10.1038/NRD3439-C1
Begley, C. G., & Ellis, L. M. (2012). Drug development: Raise standards for preclinical cancer research. Nature, 483(7391), 531–533. https://doi.org/10.1038/483531A
Baker, M. (2016). Biotech giant publishes failures to confirm high-profile science. Nature, 530(7589), 141. https://doi.org/10.1038/NATURE.2016.19269
Aldén, M., Olofsson Falla, F., Yang, D., Barghouth, M., Luan, C., Rasmussen, M., & de Marinis, Y. (2022). Intracellular Reverse Transcription of Pfizer BioNTech COVID-19 mRNA Vaccine BNT162b2 In Vitro in Human Liver Cell Line. Current Issues in Molecular Biology, 44(3), 1115–1126. https://doi.org/10.3390/CIMB44030073/S1
Zhang, L., Richards, A., Khalil, A., Wogram, E., Ma, H., Young, R. A., & Jaenisch, R. (2020). SARS-CoV-2 RNA reverse-transcribed and integrated into the human genome. BioRxiv : The Preprint Server for Biology. https://doi.org/10.1101/2020.12.12.422516
Walking boldly in truth is a dangerous business these days. In case we get censored or banned, here’s where you can find Still in the Storm and Mike Donio:
Join the discussion:
If you enjoy what you see here please share us with your friends (and maybe your enemies because they need to wake up too!):
If you haven’t subscribed to Still in the Storm, now would be a good time: