Just because a health-related mobile app is popular and has a high “star rating” from users doesn’t mean the app is accurate or has any value. In fact, such apps can be potentially dangerous, for they can give people a false sense of security about their health.
Those points are the key take-home messages from a paper published recently in the Journal of the American Medical Informatics Association.
The authors of the paper — a team of doctors and researchers from Johns Hopkins University — warn consumers and health professionals alike to be highly skeptical of health-related mobile apps that do not have solid scientific evidence to back them up.
“People tend to trust user reviews when shopping online and use them to decide which products to purchase, but that doesn’t cut it for medical apps,” says Dr. Timothy Plante, the paper’s lead author and a former fellow at the Johns Hopkins University School of Medicine, in a released statement.
“There are certain thresholds of accuracy that need to be maintained, and a five-star rating doesn’t replace clinical validation studies and FDA [Food and Drug Administration] review,” adds Plante, who is now an assistant professor of medicine at the University of Vermont.
A case study
For their paper, Plante and his colleagues analyzed user reviews and comments posted online for a once-popular app called Instant Blood Pressure, which purported to be able to monitor blood pressure using a smartphone camera.
The app’s developer, AuraLife, made these claims even though the app never underwent review by the FDA. That’s not unusual, however. As Plante and his co-authors point out, the FDA “has largely avoided applying its regulatory oversight” to the emerging marketplace of health-related mobile apps.
By 2015, Instant Blood Pressure was one of the most popular health-related apps for the iPhone, bought for $4.99 by more than 140,000 users. Then, in July of that year — for “unexplained” reasons, according to Plante and his co-authors — the app was withdrawn from the market.
If it hadn’t been withdrawn then, it most likely would have been a year later. In May 2016, a team of Johns Hopkins researchers (including Plante) reported that Instant Blood Pressure was highly inaccurate. In a study involving 85 adults, the app misclassified 78 percent of the participants with high blood pressure as not having the condition.
In December 2016, AuraLife reached a $600,000 settlement with the Federal Trade Commission for claiming the app was as accurate as a traditional blood pressure cuff.
Reviewing the reviews
In their current paper, Plante and his colleagues analyzed 261 online user ratings and reviews of the Instant Blood Pressure app. All of the user ratings were for the last version of the app, the one that was sold right before it was taken off the market.
Here’s what the analysis found:
- The average star rating for the app was four out of five stars, and almost 60 percent of its users gave it five stars.
- Among the people who commented on the app, 42 percent declared it to be accurate in reading their blood pressure. Only 10 percent talked about it being inaccurate.
AuraLife did post a disclaimer for the app. The company stated that it was meant for “recreational” rather than medical use.
Just how the app could be “recreational” is not clear. (As Plante and his colleagues write rather drolly in their paper, “We are unaware of any recreational uses of a non-invasive BP monitor.”) Nor did the disclaimer seem to matter to many consumers. Almost 10 percent of the reviewers acknowledged that they were using the app to measure their blood pressure because of a medical condition. Eleven people specifically mentioned they had been diagnosed with high blood pressure. Another person had kidney disease, and yet another was a recent recipient of a heart transplant.
Also troubling was the finding that 11 people said their health care provider had approved their use of the app to monitor their blood pressure.
In fact, six of the reviewers identified themselves as medical professionals — one doctor, four nurses and one unspecified clinician. The average rating of this group was 4.2 stars.
“The data showed that disclaimers aren’t a complete solution,” says Dr. Seth Martin, one of the study’s authors and an assistant professor of medicine at Johns Hopkins, in a released statement. “Consumers will continue to use these devices to manage their health care, which could be dangerous if they are substituting the app for medical care with a professional.”
Be skeptical and demand evidence
This paper analyzed online user reviews for only one medical-related app, but there’s no reason to think that such reviews for other apps are any less accurate.
Indeed, a 2016 survey of 137 apps for various chronic diseases found a poor correlation between user ratings and the apps’ benefits as assessed by a medical professional.
Physicians and other health care providers need to hold apps “to the highest standards and not just give out recommendations will-nilly for apps that you think are neat,” says Plante.
“And consumers should be skeptical of reviews from self-proclaimed health professionals in product reviews,” he adds.
Indeed, as the Johns Hopkins researchers point out in their paper, numerous other medical apps — some quite similar to Instant Blood Pressure — are still available to consumers, including some that have been downloaded more than 1 million times each.
“I’m glad for the high demand from consumers to be able to monitor their health from day to day,” says Martin. “Now we need to actually make sure mobile apps work as intended, in properly designed and conducted clinical studies, before they are relied on for health care.”
FMI: You’ll find an abstract of the paper on the website of the Journal of the American Medical Informatics Association, but the full study is, unfortunately, behind a paywall.