Computer-generated digital characters could help treat autism by teaching scientists more about the act of mimicry.
A European research project is helping to pave the way for the further use of digital characters, known as avatars, in studying how humans interact.
‘Lots of people have been looking at things like social perception for years, how you see faces, but we want to get beyond seeing stuff and have people actually interacting to see how interaction works,’ said Dr Antonia Hamilton, the principal investigator of the INTERACT project.
‘I’d heard about the potential of avatar technologies and wanted to see what it can do,’ said Dr Hamilton, who has received a European Research Council grant to carry out the study. ‘It’s a very new technology and people are only just getting to grips with the enormous benefits it will bring to social neuroscience.’
Unconscious mimicry is believed to play an important role in social interaction, and the extent to which people mimic each other is thought to show how close they feel to each other. People can pick up another person’s emotions by mimicking them through a mechanism called facial feedback, where inner emotions are affected by facial gestures.
People with autism mimic in a different way from other people, and the causes for this remain unknown. Dr Hamilton's project aims to provide new insights into the origin of poor mimicry among people with autism. While autism is a complex condition, many researchers believe that this deficit in mimicry may be central to the development of autism.
Why use avatars?
It’s tricky for researchers to use real people for studying mimicry as there are too many variables in social interactions. That’s why Dr Hamilton has decided to work with avatars. By using an avatar, she can change just one variable – such as gender, or clothing – to see if it has any effect on the way people mimic.
It will allow her to examine whether people mimic more closely someone of the same gender, for example.
‘You could take a technology like this and then, if you know the detail of how typical mimicry works, you could teach people with autism to mimic in the same way.’
Dr Antonia Hamilton, principal investigator of the INTERACT project
For the experiment, subjects will sit at a desk in front of an avatar on a large screen, while the researchers observe how they react. The screen is equipped with movement sensors and sometimes the avatar will also be instructed to copy the gestures of the subject.
‘As the participant moves, the avatar can respond to them, or copy them, or do things that are genuinely interactive with what the participant is doing,’ said Dr Hamilton.
The only thing she tells the participants is that she wants them to listen to the avatars or watch their hand movements. The avatar then, for example, taps out a rhythm with its hand, and the researchers watch to see to what extent the subject copies the avatar.
She also watches whether the participant copies the avatar’s head movement, an unconscious form of mimicry that can show whether two people empathise with each other or not. ‘This is all unconscious, people don’t notice how much their head is moving, but if they sway at the same rate then they are more likely to like each other and believe the other person.’
Regions of the brain
As a second stage, Dr Hamilton will scan the participant’s brains as they copy the avatar, gathering data on which parts of the brain are active as people make the decision whether to mimic or not. She plans to use this data to help develop therapies for autistic people so they can learn to mimic more effectively.
‘You could take a technology like this and then, if you know the detail of how typical mimicry works, you could teach people with autism to mimic in the same way,’ she said. However, the project will not have results until the end of 2017, so it will take a while before the data could be used in this way.
She hopes her research will also be useful to game manufacturers as it will show what type of avatar behaviour people respond to best, allowing them to improve the way players interact with digital characters.
Avatars are already being used to train the US military, and Dr Hamilton believes the capability of today’s computers means that they will become increasingly common. They could be used, for example, to train business people in public speaking.
‘I could see avatars becoming much more ubiquitous in research, and in ten years’ time, beyond research,’ she said. ‘There will be more of these all over the place because I think the computing power and the software behind them is getting so much better.’
Professor Martijn Nawijn, an immunologist at the University of Groningen in the Netherlands, tells Horizon about his quest to map every cell in a healthy human lung. He says this work should help to understand more about the causes of lung disease - which is comparatively understudied - and should lead to new therapies in the next 15 to 20 years.
There was one science story that dominated 2020 and coronavirus is likely to remain a dominant theme in 2021. But from vaccine rollout to lessons for future pandemics and – that other big challenge that we’re facing – climate change, how will the year in science play out? We asked a selection of our interviewees about lessons from 2020 and what needs to happen in their fields in the coming year.
Stone and concrete structures with the ability to heal themselves in a similar way to living organisms when damaged could help to make buildings safer and last longer.
Artificial intelligence (AI) used by governments and the corporate sector to detect and extinguish online extreme speech often misses important cultural nuance, but bringing in independent factcheckers as intermediaries could help step up the fight against online vitriol, according to Sahana Udupa, professor of media anthropology at Ludwig Maximilian University of Munich, Germany.
Bacteria can give structures an ‘in-built immune system’ to help them last longer.
Independent factcheckers can bring context to AI tools, says media anthropologist.
Dr Kate Rychert studies ocean plate structures.