Recently, a large, multi-center based group of Canadian researchers (Bartoszco of McMaster University, et al) conducted a living systematic review and network meta-analysis to investigate and compare the effects of possible repurposed drugs for prophylaxis on SARS-CoV-2, the virus behind COVID-19. Basing their study on data from a WHO COVID-19 database as well as review of extensive multilingual COVID-19 literature up until January 19, 2021 as well as six Chinese databases up until January 20, 2021, the group sought to better understand the randomized trials in which people at risk of COVID-19 were randomized to drug prophylaxis or non prophylaxis (e.g. standard of care or placebo). The group looked at both hydroxychloroquine and ivermectin but the emphasis of this entry is ivermectin. While pairs of reviewers independently screened potential eligible articles, this group recently included the very same clinical trials that a UK-based group reviewed for their systematic review on ivermectin known as the British Ivermectin Recommendation Development (BIRD) COVID-19—the BIRD recommendation. But that’s where the two different meta-analyses diverge. The Canadian group, that is McMaster’s Bartoszco et al, graded the evidence on ivermectin for prevention of COVID-19 infection as “very low certainty,” while the recent BIRD group, which employed the gold standard guideline development tool—a DECIDE evidence to decision framework in adherence with the World Health Organization Handbook on Guideline Development, produced a “low certainty evidence” that ivermectin may reduce deaths and COVID-19 infection rates, as well as values, equity, resource, acceptability and feasibility considerations, the panel of around 65 stakeholders judged in favor of recommending ivermectin for both prevention and treatment of COVID-19 (link below). The BIRD proceedings’ low certainty evidence indicates that further research is very likely to have an important impact in the confidence in the evaluation and is likely to change the estimate.
How could two different highly intelligent, focused and committed research groups generate such a significant difference in reviewing the same underlying clinical trials? Again, the BIRD group applied the GOLD standard. While Bartoszco et al review graded the evidence on ivermectin for prevention of covid-19 infection as very low certainty despite the fact that the effect estimates for COVID-19 infection favors ivermectin over control groups.
Very low certainty means that the reviewers are very uncertain about the estimate of the effect. This grading level increases the probability of a recommendation against ivermectin for prophylaxis. Bartoszco et al downgraded the certainty of the evidence by two levels for imprecision. However, it is hard to understand their justification for downgrading by one level, let alone two levels for imprecision, given that the analyses show a clear effect in favor of ivermectin, according to dozens of experts TrialSite consulted.
Put another way, downgrading twice for imprecision seems strange if not unjustifiable as viewed from at least one seemingly rational vantage. While a downgrade of one level with an understanding that more trials are ahead could make sense, but from the present evidence as is the case with a living review, one must seriously question the rationale and underlying logic that determines the estimates for ivermectin are imprecise.
So some could interpret that low certainty evidence of a very large reduction in odds of contracting covid-19 infection, rather than very low certainty and suppressing interpretation of the result. Frankly, an OR of 0.06, 95% CrI 0.02 to 0.13, looks like a highly significant and precise result, but of course there are caveats around it, just not imprecision! How one’s point of view and various biases’ can change one’s perspective.
What is the BIRD proceeding?
A prominent group of about 65 scientists, researchers, various experts and patient advocacies convened an international meeting to evaluate the evidence on ivermectin. Using the gold standard guideline development tool as mentioned previously—the DECIDE evidence to decision framework, which adheres to the World Health Organization Handbook on Guideline Development, the consensus of the group was that based on low certainly evidence that ivermectin may reduce deaths and COVID-19 infections wares, not to mention the other considerations discussed. Follow the link to review the abstract and proceedings.
The BIRD Recommendation on the Use of Ivermectin for COVID-19 is still in draft format as that group in the UK is accepting endorsements from health professionals. TrialSite offers a link to that group for objective review. The group seeks feedback via an online survey that will remain open until March 20, 2021. Thereafter, they will incorporate and analyze to produce a final BIRD document.
What about the Bartoszco et al Study?
Follow the link to review the study conducted by this large group of impressive researchers from prominent Canadian institutions. The study was financed by the Canadian Institutes of Health Research.