Many health and wellness apps haven’t done research to back up claims

With the peddling self-improvement, your smartphone could one day hold the keys to helping you manage your lungs, mind, or waistline. But we’re not there yet.

Typically, healthcare treatments undergo rigorous testing before doctors start recommending them to patients. For someone who turns to their doctor to do something like quit smoking or lose weight, doctors rely on scientific studies that have established the best methods for changing behaviors. Drug companies especially must meet a high bar to prove that their drug is safe, and works, before the FDA allows them to market and sell it to the public. 

Recently, the makers of health care apps have been stepping in as people search for help to change their habits and improve their health. But there’s a problem: health apps do not face the same standards that drugs, doctors, and other experts do — instead treating “patients” as “users” like any other app would.

Experts have found that startlingly little research goes into building and assessing apps purporting to help treat serious conditions. Still, that’s not necessarily reason to panic. Some researchers think that the current scientific study methods for assessing how well drugs or behavior change treatments work shouldn’t necessarily be the gold standard of evaluating health apps. 

Instead, some experts believe that developers should combine scientific research standards — which use tested methods to measure the effectiveness of a treatment — with the best practices of User Interaction development – involving users in the development and testing from the earliest stages. That combination could enable them to create both scientifically sound and actually effective health apps that have the potential to truly help people.

“It’s really important to conduct smaller, more qualitative research, where you really interact with your subjects, and try to understand them,” said Roger Vilardaga, a professor of psychiatry and behavioral sciences at Duke University. 

Vilardaga and colleagues recently released a review of “smoking cessation apps,” which are meant to help people give up cigarettes by checking in with them, tracking their progress, and connecting them to community support.

The study looked at how many of these apps conducted in-depth interviews with the people the apps would be designed to help before actually making the digital treatments. Then, it assessed whether including this sort of research impacted how well the apps worked. Of the 30 studied apps, only four included both efficacy studies with consistent interviews with users, or what’s known in the app-making business as early phase research.

The point of the study was to quantify and assess this specific sort of research. But, why look at so few apps in the first place? Because out of more than 700, only 30 had any publicly available studies associated with them at all. Other studies have also pointed out the lack of scientific evidence in health apps; a 2017 study found that 66 percent of smoking apps adhered to 10 percent or fewer of “evidence based guidelines” for behavior change. Some apps even came from tobacco companies themselves.

“The public needs to be really careful with smoking cessation apps out there, because anyone can make an app,” Vilardaga said. 

“Anyone can make an app.”

The problems extend beyond smoking apps. It’s not uncommon for weight loss, exercise, meditation, and mental health apps to lack medical proof backing up their claims. That’s because the FDA has taken the position that it doesn’t need to assess the vast majority of health care apps or their claims before they’re released to the public. 

“Any behavior change under the sun I’ve seen apps make claims about,” said Stephen Schueller, a UC Irvine psychological science professor who studies mental health and behavior change. “The pathway through which these things get to consumers is much different from the way non-technological things get to consumers.”

Many mental health apps will do things like send motivating messages, prompt users to share how they’re feeling, or remind users to “breathe.” Schueller says that of the approximately 20,000 mental health apps like these, studies have shown that only 3 or 4 percent have “any scientific evidence behind them.”

Same thing with health and diet apps. A 2015 review of weight loss apps found that only two studies had been conducted to assess efficacy, and those found that popular apps didn’t include the most evidence-based strategies for weight loss.

“When apps are not based on those guidelines or interventions that have been showed to be effective, that’s when we see problems,” Vilardaga said. “It’s like recommending a drug that has not been tested before.”

Image: screenshot: app store

Still, behavior change experts like Schueller and Vilardaga are not walking away from the potential of health apps to improve our lives. Both expressed that patients’ need for regular healthcare in an overburdened medical system makes the potential of apps to fill that gap too great to ignore. Plus, the number of downloads show that patients want these apps. 

“Healthcare organizations just can’t cover everything,” Vilardaga said. “People want help and need guidance, and it’s an incredible opportunity.”

One thing that Schueller and Vilardaga do agree on is the critical need for both health experts, as well as users and patients, to be involved in the app creation process.

That would include talking with people who are going to use the app in the early development and testing stages, and trials with potentially smaller groups — and more qualitative feedback alongside efficacy data — than the current model. That kind of feedback would enable designing based around the needs of actual users, and real-time product tweaking, which is something a more traditional treatment, like a drug, just can’t do.

Some best practices are emerging, too. Companies are learning to keep interactions in their apps short, because, requiring a lot of time in the app can drive away users, making the task of self-improvement seem burdensome. They’ve also seen the importance of using a “hybrid” model that connects users to real people, because having human accountability helps keep people on track. 

Additionally, many app developers do consult with specialists in the field; Schueller says that users can assess whether an app might actually help them if they list these sorts of consultations, or any affiliations with a medical institution, in their descriptions. Another good thing to look for, according to Schueller, is how often the app updates: the more an app updates with substantive changes that reflect user feedback, the more likely it is that they’re actively working to make improvements, and provide a product that works.

Still, there’s a big question that needs to be answered: How much should doctors and regulators treat behavior change apps like any other medical treatment?

“It’s like recommending a drug that has not been tested before.”

Although the FDA has taken a non-intervention approach thus far, that standard may not hold forever.

The more hard-lined approach is to make apps conform to rigorous FDA testing qualifications before they’re released to the public. That would ensure they’re on solid scientific footing, but it could be onerous for developers, and limit the field to big companies that have the capacity to conduct large-scale studies. 

On the other end, we have the current landscape of little testing, regulation, or enforcement. Currently developers and health professionals are able to experiment and reach people with ease, but with little assurance that their apps might actually help anyone. 

Experts are still debating whether to treat health apps like a piece of technology, or a medical treatment.

“I had conflicting thoughts early in my career about clinical trials, but now I’m a firm believer that we need to test things, we owe that to the public,” Vilardaga said. 

There are also questions about how much behavior change apps should conform to standards of evidence-based change. Testing methods, and studying outcomes, outside of the gold standard of a randomized, controlled clinical trial allows people the flexibility to innovate and test what they’re doing in real time. 

“It is really possible that technology allows new models and strategies of behavior change that we’re not aware of or haven’t thought of,” Schueller said.

How the FDA certifies medical treatments might not make sense for behavior change apps. It remains to be seen. But it’s clear that app developers, as well as scientists, have a lot to learn from each other about the value of evidence and user testing. 

Instead, right now, as Vilardaga said, “it’s a wild world out there.”