Directorate-General for Research & Innovation logo Horizon: the EU Research & Innovation magazine | European Commission logo
Receive our editor’s picks

The mobile apps and wearable tech tapping into users' emotions to tackle depression and anxiety

A smartphone chatbot that reinforces positive thinking and emotion-tracking tech are designed to support users living with mental health conditions. Image credit - Pxhere, licensed under CC0

Personalised smartphone applications and wearable technologies that are attuned to the user’s state of mind are offering customised ways of helping people cope with mental illness.

The rising incidence of mental health problems places great strain on health systems and societies around the world. In the EU, mental health disorders are already estimated to cost the economy €798 billion a year – a figure that is expected to double by 2030. 

Given the prevalence of mental health issues, some researchers seeking alternative ways to treat the more common conditions are turning to technology to help.

While there has been a boom in self-help and digital wellness apps on smartphones and tablets promising support for mental health issues, much of the data they generate needs to be first interpreted by health professionals before it can be used in an effective recovery programme.

‘Most existing apps stop with the data – it’s not part of a programme of interventions you can take,’ said Professor Corina Sas, a researcher in human-computer interaction at Lancaster University in the UK. ‘At present, data capture is very disconnected with the high-level-type thinking we make in terms of emotional processing.’

To tackle this, Prof. Sas and her colleagues are attempting to create more intelligent, interactive technologies that harness information about the emotional states of users over time to automatically offer tailored advice on dealing with mental health issues.

Through the AffecTech project, they are developing a personalised, low-cost toolkit using a range of technologies for individuals with ‘affective’ health conditions like depression, anxiety and bipolar disorder. They aim to support people with mild to moderate conditions - some of whom may not visit a doctor, but could benefit from this technology.

‘The idea is to support people in capturing emotional responses, but then to use new types of interactive technologies that can help people make sense of this,’ said Prof. Sas, who is the principal investigator for AffecTech.

The project began at the start of 2017 and involves a consortium of universities, health and tech organisations. It is currently exploring different devices and technologies to help track patients’ emotions and then suggest ways for people to manage their issues. These include using apps in conjunction with sensors on mobile phones, smart watches or other wearable devices, along with artificial intelligence (AI) and virtual reality technology. 

Emotional management

‘My vision is that we build a toolbox of interfaces involving different types of technology,’ said Prof. Sas. After interpreting a user's emotional state using technologies such as AI, the system would then offer relevant therapeutic activities to help the individual manage their condition.

One approach, she says, could be to use wearable cameras to allow users to record events taking place when they are in a particular emotional state, which they could later watch to help them understand the factors that may trigger positive and negative emotions. The team has also tested prototypes of wrist-worn, colour-changing biosensors that help users track their own emotions.

Another idea stems from colouring in intricate mandala patterns, which some research has shown to reduce anxiety and is growing in popularity through the sale of adult colouring books. ‘Imagine a cube on which you move your finger and start colouring in patterns,’ said Prof. Sas.

She describes this type of emotional management as a ‘paradigm shift’ that could not only have long-term mental health benefits for patients, but also lead to major cost savings for health services.

The project, which includes the Leeds NHS Trust as a member, is also seeking to test the technologies with users recruited through national health services, charities and other communities.

But while technology can be used to help patients manage their emotional state, it can often lack a key element that personal contact with health professionals can provide – empathy. To address this, some researchers are using advances in AI to offer patients support on chat-based apps.

Chat app

The Shim project, run by a team of psychologists, researchers, writers, engineers and designers, is combining techniques from cognitive behavioural therapy – a talking therapy commonly used to treat people with mental health issues – with human-like support on a mobile-based chat app.

The system uses AI to spot language patterns and keywords to create personalised text-based conversations that help to shift the user’s mental perspective when they report a negative thought.

'In the future, we will see a world where you pick up your phone when you are feeling down to get emotional support in the moment.'

Dr Kien Hoa Ly, Linköping University, Sweden

Dr Kien Hoa Ly, a researcher in behavioural sciences and learning at Linköping University in Sweden and chief executive of the project’s company Hello Shim, explained that the app uses positive psychology to help a user reflect more optimistically on their own life.

‘In the everyday interaction, we are using positive psychology to make the user reflect upon positive things in life, experiences and what the user is grateful for,’ said Dr Ly. ‘We are building a mental map of every user and their inputs. As a user talks to Shim it grows, which makes Shim remember more things and ask more questions.’

The app might ask the user, for example, how they felt when they were doing an activity they enjoy such as hiking in the mountains, getting them to reflect on positive emotions.

This use of empathy differentiates the app from other mental health apps, says Dr Ly. ‘Our mission is to build the world’s best data model and predictive tools to help people improve their wellbeing,’ he said. ‘We do this by understanding what specific reply from Shim is most likely to be helpful given a negative thought from a specific user.’

The company is also experimenting with social features, such as sharing positive thoughts with friends and family.

Since launching on Apple mobile devices in Sweden last November, Shim has been downloaded 10,000 times. The company hopes to launch the app in the US later this year.

A pilot study published last year showed that participants who talked to Shim reported a higher level of emotional wellbeing and lower stress levels than a control group after using the app for two weeks. Shim is now planning a larger study, which may involve collaborating with several US universities.

‘In the future, we will see a world where you pick up your phone when you are feeling down to get emotional support in the moment,’ Dr Ly said.

The research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

More info