A troublesome new study explores a potentially dangerous flaw in health tech user ratings, comments, and reviews. Timothy B. Plante from the Department of Medicine, Larner College of Medicine at the University of Vermont was the lead author of a study that found consumer ratings for a mobile blood pressure app correlated with positive readings, even when those readings were consistently inaccurate and differed from the consumer’s expectations.
In a study published in Nature in August, researchers at five clinics worked with patients who had elevated blood pressure. Group participants estimated their blood pressure and then measured their blood pressure with the Auralife Instant Blood Pressure app. They then completed an experience survey. Analysis of the survey data resulted in three findings. The perceived accuracy of the app by the test subjects was the same whether the app BP readings were less than, similar to, or greater than user expectations. The subjects reported enjoyment scores that correlated positively with lower than expected app readings and correlated negatively with higher than expected app readings. Also, subjects who received higher-than-expected results tended to report a lower desire to measure BP in the future; this was not the case with those who received lower-than-expected results.
Instant BP was a tremendously popular, top-selling app on iTunes for 13 months from June 2014 through July 2015. It received an average four out of five star rating in more than 1,200 user reviews. The company does not claim that the app’s results should be used for “medical advice or diagnosis.” Instead, the app merely provides “estimates of blood pressure.” Plante’s summary of the study points to its most significant concern, which goes beyond the Instant BP app; “For mHealth apps, user experience may be at odds with accuracy of medical functionality.”
Why did consumers drive Instant BP sales to such high levels when the app was inaccurate? Should we be on the lookout for sycophantic apps that give users good news despite reality? When consumers are aware that apps are providing rosier than expected reports, why do they keep using the apps, rate the apps highly, and recommend them to others? Is it the case that the desire for good news overrides caution and rational judgment? Few people like bad news, but if it’s human nature to punish or reward the messenger based on the desirability of the message, medical apps and devices warrant extra caution and validation. If people are going to be making health decisions based on the information, it must be accurate and effective.
