Unleash the Data on COVID-19

By Maryaline Catillon and Richard Zeckhauser*

Given the lethality of the COVID-19 pandemic, the urgent need is for actionable information directing care towards treatments offering higher probabilities of improving outcomes and preventing death. In normal times, randomized control trials (RCTs) would be the gold standard for determining whether innovative medical treatments are safe and effective. But with 1,500 Americans dying every day, these are hardly normal times.

There is an urgent need for high quality studies based on real world experience, which has already accumulated for many thousands of patients. Dr. Anthony Fauci, the nation’s pandemic physician in chief, said that RCT results will not be available "for months". The disease will not wait.

RCTs, which randomly assign patients to a treatment or a control group, are only ethically acceptable when the safety and performance of a treatment is unknown. When ample data exists, as now, that criterion is not met.

Analyzing real world data on actual outcomes, when it exists in abundance, offers an alternative approach to learn almost immediately. Moreover, it avoids the ethical challenge of an RCT, given that available data could predict outcomes. Massive numbers of COVID-19 patients are currently being administered "unproven" drugs based on medical decisions made by doctors. Massive numbers are not receiving any such drugs. Thus, carefully designed case control studies could leverage differences between ongoing protocols at large hospital systems and detailed information from patients’ electronic medical records. That could determine whether widely employed hydroxychloroquine, with or without azithromycin, provides significant benefits, and at which stages to which patients, and could provide similar information on the risks it imposes. It could yield the same information about remdesivir, and about many other drug treatments currently in use.

For each patient, doctors strive to optimize treatment in the current, uncertain environment. These drug versus non-drug decisions constitute an ongoing large observational study, in which the allocation to treatment and control groups varies widely. The large numbers of patients treated eliminates concerns that random variation might lead to misleading results. Those large numbers also yield results by demographic, comorbidities, and stage of disease. Leveraging real world evidence is more acceptable ethically when extensive information is already available.

As decision theorists who have studied the methodological quality of vast numbers of RCTs, we are enthusiasts for well-conducted RCTs. But delaying public health recommendations till RCTs are completed is not appropriate in the present circumstance. Imminent threats are enormous and widespread data is easily at hand. The outcomes of the thousands of individuals who have already received drug therapies on an ad hoc basis should inform practice now.

We make this recommendation recognizing that President Trump, having taken his own counsel, is cheerleader in chief for hydroxychloroquine. He might be right. He might be wrong. Let’s assemble the data as quickly as possible. Saving lives, like the ocean’s edge, is where politics should stop.

High quality case control studies based on thousands of cases, the silver standard we recommend, are immensely faster than RCTs. Recent articles in the world’s leading medical journals show that they consistently yield the same major findings. Experience with the recommendations of antiretroviral therapy (ART) for HIV provides an instructive warning. Even though 20 years of observational studies demonstrated its enormous benefits, the World Health Organization waited until 2015 and the publication of the first set of RCT results (which reached the same conclusions) to make a "treat all" recommendation. Many lives were lost as the world waited for its recommendation.

COVID-19 presents its own example. Through late March, medical authorities recommended the general public not employ masks to protect against it. In early April, that all switched: masks became strongly recommended. No RCT supported this reversal; little evidence was mounted. Yet officials applauded, the public widely complied, and the world was better off.

We support the well-intentioned RCTs already underway in many medical centers. But tens of thousands will die before we have their results. In the meantime, anecdotal success stories, physicians’ highly personal experiences, and informal information networks are serving as the basis for widespread adoption of treatment protocols. The treatment for COVID-19 would be much more responsibly informed, hence appropriately deployed, if we were just willing to unleash the data.

*Maryaline Catillon, formerly a hospital director in France, recently completed her Ph.D. in Health Policy at Harvard. Richard Zeckhauser is a professor of economics and decision theory at Harvard Kennedy School.