The Ugly Story about Remdesivir
When a man believes that he is on the verge of drowning in a raging river, he will reach for anything that is floating nearby, including a ravenous crocodile. Instinct and fear easily overcome any semblance of rational thinking.
This is exactly what recently occurred when the randomized clinical trial of remdesivir was terminated prior to its completion, permanently eliminating the unique opportunity to forever determine whether the drug is efficacious in the treatment of COVID-19. The irrational behavior of learned scientists regarding this disease defies that we live in the 21st century, 400 years after the scientific revolution began to provide sensible explanations of our natural world. Witchcraft and snake oil, mixed with a large dose of politics, are giving the scientific community a run for their money.
There have been more than 11 million people infected with the SARS-cov-2 strain of coronavirus since its outbreak 6 months ago in Wuhan, China. At least 500,000 people worldwide have died from complications of the respiratory illness caused by this organism. The declared pandemic has produced irrational behavior on the part of many members of the medical community. Anecdotes related by so called experts to an eager media continue to largely replace scientific data. During the 14th century plague epidemic termed the Black Death, which killed 20 million people in Europe, doctors were completely unable to prevent or cure the plague. Yet they prescribed bloodletting, mustard, mint, horseradish, eating a chopped-up snake or rotting food, minerals, arsenic, mercury, rubbing the entire body with pigeon blood, self-flagellation and either sitting next to a fire or sitting in the freezing rain or both. Today, just like then, doctors are giving different drugs to treat a micro-organism without any proof that it will help.
Remdesivir was developed by Gilead Sciences in response to the 2014 West African Ebola virus epidemic. The drug failed all clinical trials for the treatment of the Ebola virus. Its method of action is to mimic a natural building block for RNA synthesis and, assuming the drug is incorporated into a viral RNA chain, the virus will presumably no longer be able to replicate and cause disease. Remdesivir, prior to 2 months ago, was an investigational compound that was not licensed or approved for use in the United States. Because of the hysteria regarding the absence of any available treatment for COVID-19, remdesivir was touted as a possible cure for the disease and was entered into multiple clinical trials.
A clinical study involves research using human volunteers that is intended to add to medical knowledge. There are two main types of clinical studies: clinical trials, sometimes termed interventional studies and observational studies. Observational studies, although sometimes contributing to scientific knowledge, are retrospective and often lack rigor and validity. The major advantage of a well-designed trial over an observational study is the ability to demonstrate a cause and effect relationship between an intervention and a pre-determined endpoint. In a clinical trial, participants receive specific interventions according to the research plan or protocol created by those investigating the drug or technique. When a new drug is being studied, it is usually not known whether it will be helpful, harmful, or no different than available alternatives (including no intervention). The investigators try to determine the safety and efficacy of the intervention by measuring certain outcomes in the trial participants.
Decades of investigation have led scientists to the conclusion that there is only one truly trustworthy source of information on whether a medical therapy works: Prospective randomized double-blind placebo-controlled studies performed at multiple clinical sites. When compared with other research designs, the level of evidence given by randomized double blinded placebo-controlled clinical trials can be nearly 100% and hence it is considered the “gold standard” for determining safety and effectiveness of drugs or other interventions.
An imperative of a good clinical trial is that a determination be made as to whether improvement in the treated group is due to drug effect rather than the act of being treated. In drug trials, a control group is given an inactive substance termed placebo while another group is given the drug being studied. This allows researchers to compare the drug’s effectiveness against the placebo’s effectiveness. A placebo must be used. The highest-quality clinical trials are all placebo-controlled and are also randomized, meaning that subjects are randomly assigned to placebo and intervention groups. Randomization avoids selection bias that could occur if either the physician or the patient chooses whether to treat with the drug in question and, additionally, if the trial has an adequate population, removes most confounding by all known and unknown factors. It minimizes the possibility that the observed association between the exposure and the outcome is really caused by a third factor. Since the outcome of any clinical trial can be affected by patient or investigator’s expectations, blinding is important. A double blinded study is one where both the patient and the investigator are unaware of whether the patient is receiving the intervention or being treated with the placebo.
Prospective randomized placebo-controlled double blinded clinical trials are time consuming and expensive to organize and carry to completion. They require significant physician and patient commitment. However, the information derived from these are well worth the investment. These trials eliminate the confounding effects of observer bias both on data interpretation and influence of the physician on the patient. They eliminate memory distortion and cognitive dissonance. These clinical trials by design take into consideration the natural history of the disease being studied.
Medical researchers have agreed until now that a treatment cannot really be said to be proven effective unless it has been examined in meticulously designed and sufficiently large double-blind studies. Randomized clinical trials provide the best quality evidence for the identification of new therapies for a vast array of conditions. There are currently more than 250,000 ongoing randomized clinical trials registered with the United States at the website, ClinicalTrials.gov involving interventions including drugs, other disease modifying agents, behavioral therapies, surgical techniques, and devices. Some are well designed and may provide important information when completed, while others bear no relationship to the logic of science.
In designing the protocols for “gold-standard” clinical trials, investigators must determine how many patients need to be enrolled in a trial to provide for results that will potentially reach statistical significance. This is termed statistical power. If a study enrolls too few people, the chance of discovering a true treatment effect diminishes, even if it is there. The number of enrollees necessary to identify a benefit depends on the strength of the treatment — a powerful treatment can be identified as effective in a relatively small study, but a modestly effective treatment may require hundreds of study participants to identify an effect. This effect is compounded when it is tricky to measure the benefits of a treatment.
Evaluation of statistical significance is a mathematical analysis used to ensure that the apparent improvement seen in the treated group represents a genuine difference from the untreated group, rather than just chance.
Before performing an experiment, researchers are expected to pick a hypothesis that their study will test. This is called the primary outcome measure. For example, in a study of a treatment for Alzheimer’s disease, the primary outcome measure may be the change in score on a certain memory test over a set period of time. The researchers hypothesize that scores on this test will improve, stabilize, or decline less slowly with their proposed treatment versus a control arm that is receiving placebo rather than the treatment. The study is then conducted to determine whether their hypothesis is correct.
Once a study has begun, however, there is a temptation to gather more information by assessing numerous other items in the trial subjects. These are called secondary outcome measures. In the Alzheimer’s example, these may involve a questionnaire assessing ability to perform a daily task, physician’s or/and caregiver’s opinion of overall progress and other reasonable ways of evaluating the success of therapy. There is a problem, however, with using a multitude of secondary outcomes. According to the immutable laws of statistics, if you measure enough things, some will indicate improvement, unrelated to the treatment, just merely by chance. Researchers who look at dozens of factors in hopes of finding evidence of improvement in a few of them are said to be engaged in data dredging. Only the results on the primary outcome measure are trustworthy. There is simply too much leeway to find favorable data by digging deeply into the mass of other data recorded.
The vast number of people contracting COVID-19 on a daily basis offered the scientific community a population with more than enough statistical power to conduct many clinical trials testing what should be the primary outcome hypothesis, i.e., whether a specific drug makes people better. In addition, the extremely large number of hospitalizations and deaths from the illness, the latter number being more than 10% of those hospitalized in the first months of the pandemic, should have allowed for an answer to a very simple question: Does a specific drug decrease the mortality from COVID-19 in an ill population when compared with placebo? It was a situation ripe for participation by fearful patients with little to lose from trial participation since both arms of any trial would also be given standard supportive care. Data driven scientists should have had all the available modern tools to write good protocols, quickly enroll myriads of patients and wait for the results, hoping that those results would quickly translate into guidance to the medical establishment in successfully treating future patients.
Remdesivir, because of its activity in the test tube of interfering with viral replication, has been one of the many drugs entered into prospective randomized, placebo-controlled trials against COVID-19. It is the only one of them, to date, that has been approved by regulatory agencies in the United States and elsewhere to treat COVID-19. The remdesivir story is one of wasted scientific opportunity, the squandering of effort and money, the conveying of half-truths, and, most importantly, the misuse of patients’ lives, in a manner that belies that this is the 21st century and not the infuenza pandemic of one hundred years ago.
The first “gold-standard “ clinical trial of Remdesivir, a drug that must be given intravenously, was performed in February and March 2020, in Hubei Province, China, then the epicenter of COVID-19. The study coordinated by the China-Japan Friendship Hospital and conducted at multiple sites in Hubei province, enrolled 237 adults with COVID-19 at 10 large local hospitals. All patients had decreased blood oxygen saturations as well as confirmed pneumonia as indicated by lung scans. They were randomly assigned to receive remdesivir or a placebo for 10 days. They were allowed to be prescribed other medications in accordance with usual care. Remdesivir did not significantly speed up improvement of symptoms or reduce the likelihood of death compared with the placebo. The researchers also saw no significant differences in length of hospital stays, time on ventilators or how long to took to clear the coronavirus. The study did suggest faster improvement among those treated earlier — within 10 days of symptom onset — but that difference was not statistically significant. Twice as many people on remdesivir, as opposed to, placebo, stopped treatment early due to adverse events (12% versus 5%). The results of the randomized clinical trial were leaked to the World Health Organization, which prematurely posted the results on their website and then removed the leaked data from the internet. The clinical trial was terminated before the completion of the 453 patients initially planned for recruitment into the study because the hospital system had an insufficient number of seriously ill patients as Hubei cases sharply decreased in March. The results of the trial were published in the April 29, 2020, edition of Lancet. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31022-9/fulltext
The Adaptive Covid-19 Treatment Trial (ACTT), sponsored by the United States National Institute of Allergy and Infectious Disease, a division of the U.S. National Institute of Health (NIH), initiated a major clinical trial on remdesivir in February, 2020, and began enrollment of patients in early March. There were 45 trial sites in the United States and 15 in Europe and Asia. The plan was to enroll 1063 patients hospitalized with lower respiratory involvement from COVID-19 and prospectively randomize them to receive either remdesivir or a placebo administered intravenously daily for up to 10 days. The first patient entered into the study was an infected American repatriated from the Diamond Princess cruise ship. The primary outcome of the study, as originally formulated, was to score patients on the fifteenth day after enrollment on a scale of 1 through 8, with death of the patient scored as a one and a patient having been discharged and having no physical restrictions at home scored as an eight. The average score of all patients on remdesivir was to have been calculated and then have been compared to the average score of patients on placebo to conclude whether receiving remdesivir influenced outcomes. A secondary outcome was to compare percentages in each group dying of the disease. It was a well-designed study, fitting all of the criteria necessary for being labelled “gold standard.”
After hearing about the negative results from China regarding non-effectiveness of remdesivir, researchers needed a victory, physicians needed a quick victory and most importantly the American people bombarded by images of body bags in the hallways of Astoria hospital needed a victory, no matter how small. Scientific method, however, should never be abused by artificially speeding up the process.
By March 22, 2020, with the disappointing Chinese data having been leaked but not yet published in detail, there were only 77 patients entered into ACTT, less than 5% of the planned enrollment. In a series of communications and meetings beginning that day, discussion began among NIH researchers to change the primary end point of the trial. On April 2, 2020, a final decision was made that this would indeed occur. The trial’s end point would now be the number of days after enrollment until each patient reached score 6 (still in hospital, but no longer requiring oxygen), score 7 (discharged from hospital but not able to resume normal activities), or score 8 (not hospitalized, no limitations on activities.). Achieving one of these three scores would now artificially be termed “recovery” from the disease. These changes were officially posted to the government registry of clinical trials on April 16, 2020.
Changing an end point during a clinical trial is generally frowned upon. Once a study has begun and new information comes in from other sources, there is always the temptation to gather more answers by altering the primary outcome measure or adding new secondary outcome measures. The problem in doing this is that by the laws of statistics, if you measure enough things and dig too deeply into a database, some numbers will indicate improvement, not just by chance. Researchers who change outcome definitions or add many secondary outcomes hoping to find evidence of improvement in one of them usually find that their data dredging does provide at least one favorable outcome.
In addition to altering the study’s main research goal midstream, NIH also decided to look at an “interim analysis” of the data when 400 patients had “recovered,” per the new definition. Performing an interim analysis is quite common in “gold-standard’” clinical trials, but it is always performed by a committee of outside experts unaffiliated with the investigators to try to avoid conflicts of interest. These independent experts are termed a data and safety monitoring board, or DSMB. A DSMB is permitted to analyze trial data while it is ongoing, even as the patients, physicians and drug companies are kept unaware of who is getting the medicine and who is getting placebo. These boards are obligated to ascertain that patients are not suffering being harmed by an experimental drug and are permitted to conclude a trial early if a drug is undoubtedly effective.
The DSMB for the remdesivir study never met for an interim analysis prior to completion of enrollment of all 1063 patients, which occurred on April 20, 2020. They met on April 27, 2020, to review data on the 480 patients who had “recovered,” the altered endpoint of the ACTT. Based upon their review of the interim data, they noted that remdesivir was better than placebo from the perspective of the new primary endpoint, time to “recovery.” That data specifically showed that the median time to “recovery” was 11 days for patients treated with remdesivir compared with 15 days for those who received placebo, a statistically significant result given the power of the study. On April 28, the following day, the DSMB issued a report and a recommendation to the National Institute of Allergy and Infectious Disease and NIH. That recommendation did NOT include early termination of the clinical trial. The DSMB did not recommend that patients in the placebo arm should now receive remdesivir. The sole recommendation was that, in another NIH-sponsored clinical trial, shortly to commence enrollment, and designed to compare Eli Lilly’s drug Olumiant against remdesivir in COVID-19, there was no longer a need for a placebo-only group. The DSMB recommended that the Olumiant study proceed with two arms, rather than three, as previously planned. The NIH felt that this recommendation, unrelated to the ACTT trial, called for some action.
Immediately upon receiving the report from the DSMB on April 28, 2020, NIH decided to start giving remdesivir to patients who had been assigned to receive a placebo in the ACTT, effectively ending placebo controlled data collection and forever terminating the ability to determine whether remdesivir does or does not save lives. Clifford Lane, NIAID’s clinical director, commented that remdesivir is “not a home run but is probably better than nothing.”
Remdesivir is not even a walk or a single and may yet prove to be a strike out. The preliminary report of the ACTT (https://www.nejm.org/doi/full/10.1056/NEJMoa2007764) was published in The New England Journal of Medicine on May 22, 2020, describing baseline characteristics of all 1063 enrolled patients and the results of the 731 patients who, by the termination date, Aril 28, 2020, had each either completed 29 days of the trial without recovery or death or had recovered or died. The demographic and clinical characteristics at baseline in Table 1 of the article are similar between the 541 patients in the remdesivir arm and the 522 patients in the placebo arm except for one striking exception which was not commented on within the article. At the time of enrollment, almost 25% more patients in the placebo arm were on invasive mechanical ventilation than in the remdesivir arm. Randomization is theoretically supposed to remove most confounding factors and does so if the sample size is infinitely large. However, in this case that factor happened not to have been removed, and the already marginal outcome of this clinical trial could be caused by this major difference in the state of health at baseline between the two arms of the trial. In other words, the placebo patients were sicker. Should one not expect that sicker patients would remain hospitalized longer?
By April 29, 2020, the possibility of getting important data from the ACTT trial had disappeared. The blinding was terminated and all patients in the trial on placebo were placed on remdesivir. Over the years many other clinical trials have been terminated early, some for good cause. That decision this time was wrong. Clearly scientific rigor was sacrificed to, at best, shorten a hospital stay of a few patients. That loss of rigor will now lead to hundreds of thousands of people having decisions made regarding their life based on improper science.
On May 1, 2020, (https://www.fda.gov/news-events/press-announcements/coronavirus-covid-19-update-fda-issues-emergency-use-authorization-potential-covid-19-treatment), the U.S. Food and Drug Administration issued an emergency use authorization for the investigational antiviral drug remdesivir for “the treatment of suspected or laboratory-confirmed COVID-19 in patients hospitalized with severe disease. While there is limited information known about the safety and effectiveness of using remdesivir to treat people in the hospital with COVID-19, the investigational drug was shown in a clinical trial to shorten the time to recovery in some patients.” Although emergency use authorization is not equivalent to FDA approval, remdesivir overnight became the “standard of care “for all COVID-19 hospitalized patients. The reason given by the NIH and the subsequent FDA authorization was to enable placebo recipients as well as people with COVID-19 to “benefit” from treatment with remdesivir. A week later, on May 7, without any supplementary information, Japan became the first country to grant full regulatory approval of remdesivir. (https://www.reuters.com/article/us-health-coronavirus-japan-remdesivir/japan-approves-gilead-sciences-remdesivir-as-covid-19-drug-idUSKBN22J1UZ)
Importantly, any future clinical trial to evaluate another investigational drug in the treatment of COVID-19 will no longer be designed to have a placebo arm, since it would be unethical to subject a patient to less than “standard of care.” Society has made the decision that from now forward, the comparator will be remdesivir, rather than placebo, although remdesivir may truly be no better (or may be worse) than placebo, The ironic, and perhaps fortunate, part of this conundrum is that the bar that the new drug has to surmount will not be that much higher than that would have been set by a placebo.
There remain a number of remdesivir clinical trials that are still active, none of which come close to satisfying the “gold-standard” requirement Gilead itself has released results of two of their open label (unblinded) SIMPLE clinical trials A few weeks ago, the company announced completion of a placebo-controlled 600 patient open label study (not blinded) in hospitalized patients with “moderate” COVID-19, comparing a 5-day course with a 10-day course or remdesivir. The five-day course was statistically better than placebo, while surprisingly patients given the ten-day course did not do statistically better than the placebo arm. To add to the confusion, Gilead also has reported that in patients with “severe” COVID-19, a 5-day and a 10-day treatment course were statistically similar in efficacy. There was no control group in the latter study. Both studies had almost trivial clinical endpoints and the results of these two trials seem to be in conflict. The World Health Organization’s SOLIDARITY clinical trial is open-label and is being performed primarily in hospitals in countries with less than adequate data collection. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov/solidarity-clinical-trial-for-covid-19-treatments The French INSERM trial is also open label, has five treatment arms and may add more treatment arms even while the study progresses. https://presse.inserm.fr/en/milestone-inserms-commitment-to-the-fight-against-the-covid-19-pandemic/39531/ It is unknown how these latter two studies will be influenced by the premature acceptance of remdesivir as standard of care.
Football appears to stick to the rules more than our COVID scientists, and therefore the sport’s organizers, although they lack a true season, right now have more credibility. Football does not randomly move the goalposts during the course of a contest and, when the results are still undetermined, do not prematurely terminate the game. The answer to the question of whether remdesivir saves lives has eluded us in our rush to find a cure. The true crisis here lies in the rush to judgement, prompting us to forget longstanding principles of evidence-based medicine, to abandon logic and clear-headedness, and to lower the bar for adopting unproven standards of care. Reduction in recovery time should never have been considered as a primary endpoint in this situation. This is bad science and bad medicine. Doctors are not adequately testing their ideas before implementing them. In the age of COVID-19 hysteria, the ancient and often-quoted medical dictum, primum non nocere, first do no harm, has been trashed. The situation is baffling: It is common to see physicians using not one but many unproven drugs simultaneously. If the patient recovers, we will never know if one, all, or some combination of these drugs helped, hurt, or did not matter at all. If the patient dies, all these same questions remain unanswered.
Indeed, there were numerous drugs entered into clinical trials throughout the world in early 2020. Unfortunately, most of the clinical trials did not come close to meeting any “gold standard” at their initiation. They were anecdotal, retrospective, non-placebo controlled, unblinded, lacked specific enrollment criteria and had no clear endpoints. The scientists thought they were drowning and reached for crocodiles.
A good example of this is the pseudo-scientific clinical trial termed the “French Study” involving hydroxychloroquine (HCQ), which overnight propelled that drug into the top of each evening’s headline news, immediately caused a shortage of this widely available drug, and, a few weeks later. had already motivated the purchase of hundreds of millions of doses sold and utilized on a false hope. As of today, there are still those that tout the drug and present new faulty evidence such this week’s retrospective Henry Ford Hospitals study https://www.ijidonline.com/article/S1201-9712(20)30534-8/fulltext, where the absence of randomization makes one question the role of selection bias used to determine who would be treated with HCQ. There still are more than 50 “clinical trials” open world-wide studying HCQ, despite the vast preponderance of real evidence leaving no doubt that massive amounts of work, money, time and good will has already been wasted on the HCQ wild goose chase, in addition to leaving some patients dead or forever damaged by the experience.
There is a major dichotomy between the major pushback against the use of HCQ by the U.S scientific community and that same community’s willingness to accept remdesivir after a prematurely halted remdesivir trial with an altered outcome measure followed by its FDA “approval.” There is an alternative cosmology of Covid-19 misinformation masquerading as science.
Why have scientists been so eager to change tried and true principles regarding the nature of scientific evidence and clinical trials? Why have physicians thrown multiple unproven drugs at a virus? Why is there so much chaos, confusion, obfuscation, conspiracy theory and extraordinarily little true data. This is not the way to deal with a pandemic in 2020.
I believe that the root cause is fear. COVID-19 is a new and merciless virus. Our lives have been entirely transformed by it and our public and personal conversations are filled with nothing else. Our medical practices are empty because of fear of contagion and our hospitals and emergency rooms are flooded with COVID-19 patients bringing us unending management questions which cannot be answered by hysteria.
When the entire world is facing an unprecedented threat, it is natural to believe that our response must also be unprecedented. Yet fear and hysteria should not have prompted changing the endpoint of the Adaptive Covid-19 Treatment Trial. Fear should not, even more importantly, have led to the termination of a trial that had the potential to provide answers, rather than raise more questions. Evidence-based medicine was the loser here, as was the public.
