When it comes to digital health products, accountability matters. The astounding growth in digital health care technology in the past decade may have come at the cost of startups being sure that the products and services actually do what they purport to do. Researchers from Rock Health, the Cambridge University Department of Public Health and Primary Care, and Johns Hopkins University’s Whiting School of Engineering and School of Medicine published a cross-sectional observational analysis of 224 digital health companies. Less-than-heartening study results found that many of the analyzed companies had low levels of clinical robustness. Because a significant number of digital health companies made few claims for their products, the research team concluded that health care technology developers do not fully understand the importance of clinical robustness and claims.
The researchers analyzed U.S. digital health startup companies with at least one round of funding from 2011 to 2020 that sell products for health care prevention, diagnosis, or treatment. To tally a digital health company’s claims, the researchers looked for “unique quantitative statements about product outcomes” on the company website, including links to external pages. Claims factors included engagement (such as number of users), economic matters (such as money saved or cost savings), and clinical impacts (such as diagnostic efficacy or disease reversal or cure), The team measured clinical robustness by summing company regulatory filings and clinical trials.
The clinical robustness point system resulted in scores ranging from 0 to 10 or higher. Overall, the 224 digital health companies averaged only 2.5 total on the clinical robustness scores (1.8 regulatory filings and 0.8 clinical trials). However, 98 companies or 44% had no regulatory filings and no clinical trials for a score of 0.
The startup companies averaged 1.3 total claims (0.4 engagement, 0.4 economic, and 0.5 clinical). The study breaks out various factors in the analysis such as the types of products, customers, and providers. The overall impression from the study is that many digital health startups make only a few measurable claims and a surprisingly large number don’t make regulatory filings or run clinical tests to support the few claims they do make.
Quoting Health Tech Insider publisher and editor Alfred Poor, “Five years ago, you could stick a breadboarded device on a wristband and call it a product. Three years ago, it had to be well designed and in production. Now, you need hard data to show that (1) you are measuring what you say you’re measuring and (2) that this information has demonstrable benefits.” On the basis of the reported study, one can conclude that many digital health startups didn’t receive that memo.
Trackbacks/Pingbacks