Researchers believe combining a machine learning model with human experts’ skills could make the approval process for lower-risk medical devices faster, more accurate, and cheaper.
A new process that consolidates machine learning algorithms with unique human-based expertise promises to improve the regulatory and approval process for new medical devices by reducing the recall rate by nearly 39% and saving up to $2.7 billion a year, according to a new study.
Each year, the United States Food and Drug Administration (FDA) has to review and approve a significantly large number of new medical devices vying for a place in the market.
With limited resources, a large volume of devices to review, and a stated policy of approving the devices within a 90-day window, for a large subset of devices the FDA relies on a method that allows manufacturers to demonstrate that their device is “substantially equivalent” to a previously approved one—a process known as the 510(k) pathway. Each year that process accounts for about 3,000 devices, which include anything from a mobility scooter to a new type of catheter to orthodontic aligners.
But while this approval method helps ease the burden on both manufacturers and regulators, it has its drawbacks. According to previous studies, 11% of devices cleared through this process had been subject to recalls; and 71% of devices recalled by the FDA had been approved this way.
A group of researchers built a large-scale dataset of over 31,000 medium-to-low risk medical devices and 12,000 national and international manufacturers from over 65 countries. They then used that dataset to develop and train a machine learning (ML) model capable of predicting the chances of a recall.
The researchers are Soroush Saghafian, associate professor of public policy at HKS and founder and director of the Public Impact Analytics Science Lab (PIAS-Lab) at the Mossavar-Rahmani Center for Business and Government; Mohammad Zhalechian, a former post-doc at the PIAS-Lab and now an assistant professor at Indiana University; and Omar Robles, a regulatory expert with Emerging Health Consulting and a former M-RCBG senior fellow.
Their proposed approach would rely on a multi-step process, using their algorithm to recommend direct approval, rejection, or further evaluation by an FDA human expert.
“A conservative evaluation of our proposed policy based on this data shows a 38.9% improvement in the recall rate and a 43.0% reduction in the FDA’s workload,” the researchers write. “Our analyses also indicate that implementing our policy could result in significant annual cost-savings ranging between $2.4 billion and $2.7 billion.”
“Our modeling framework enables the FDA to integrate its expertise with quantitative evidence,” they write. “While it does not prescribe a specific course of action for devices that warrant further evaluation, it allows for the incorporation of the FDA experts’ judgment when necessary. Focusing the FDA’s expertise on devices requiring the most attention can significantly enhance the evaluation process, improving patient safety and reducing unnecessary workload for the FDA.”
The paper suggests further studies into cost savings—the authors believe their conservative estimates could underestimate the advantages of benefiting from their proposed human-algorithm approach. They also suggest a randomized experiment that assigns devices in the application system randomly to their proposed human-algorithm process and the existing FDA procedure.
By Robert O’Neill / Courtesy HARVARD Kennedy School
