Most people know even from a distance whether a couple is flirting or fighting, but brain researchers are studying why some, including those with autism, struggle to read these social cues.
A touch on the arm, a tilt of the head, a comfortable silence can all betray whether a couple in a café are spouses, siblings, colleagues or strangers.
By reading a mixture of subtle signals, such as body language, tone of voice and facial expression, we all make snap judgements about the level of intimacy and balance of power between other people.
‘Social interactions are something that humans are very good at figuring out,’ said Dr Kami Koldewyn, a psychologist at Bangor University in the UK. ‘We pick up on cues very quickly from other people’s faces, bodies and actions.’
But the brain networks responsible for this incredible intuition are still largely a mystery. Exactly when they develop, or even where in the brain they lie is still unknown. Nor is it clear whether all these skills are learned or if some are ready to go from birth.
Understanding this area of brain development may eventually lead to interventions to help people who struggle to read social cues, such as those on the autistic spectrum.
Dr Koldewyn is attempting to answer some of these questions in an EU-funded project called Becoming Social. Using functional MRI brain scans, she is hoping to identify which networks are active when volunteers observe other people interacting.
‘We suspect that reading interactions is not all in the face, or the body or the specific individual actions. We think some brain regions are specifically sensitive to joint, social interactions.’
Dr Kami Koldewyn, Bangor University, UK
A group of 100 children aged from nine to 14-years-old will be shown video footage of several pairs of people as they greet and talk to each other. The researchers will then ask the youngsters questions about the relationship between the two individuals. They may be asked whether the people are friends or, if two people approach a narrow doorway, which of them is most likely to give way and let the other person pass. Some of the tests will feature video footage that focuses only on the faces of the people, or the movement of their body, while others will show the whole person as they interact.
‘Specific parts of the brain are very sensitive to facial cues while others are more sensitive to body cues,’ said Dr Koldewyn. ‘But we suspect that reading interactions is not all in the face, or the body, or the specific individual actions. We think some brain regions are specifically sensitive to joint, social interactions.’
Once they have mapped the regions of the brain that are sensitive to social interactions, the researchers will begin watching how these networks develop over time. The same group of children will be tested again after two years to see whether their increasing sensitivity to the world around them as they mature is reflected in the brain.
‘As we move from childhood to adolescence, there is a significant shift from a more self-centred phase to becoming concerned about what others think of us and how the people around us affect us socially,’ explained Dr Koldewyn. ‘Children do pick up on social cues, but there is a strong reorientation in adolescence towards attending to and thinking about the social world, so we expect this transition to create some significant changes in brain regions sensitive to social interactions.’
Understanding the link between brain development and social ability could open the door to predicting real-world social skills. In practice, it may eventually be possible to tune and even improve children’s social literacy through training exercises.
The researchers also intend to recruit around 30 people with autism spectrum disorders and compare their test results to those from the typical population.
‘This will allow us to look at whether the predictions we would make for the typical group hold for the autism group,’ said Dr Koldewyn. She hopes it could lead to new ways to help people with autism to learn to read valuable social cues.
But studying the way the brain processes non-verbal cues during one-on-one social interactions can be a challenge because researchers need to control one side of a conversation to ensure that people are tested in a uniform way.
Now technology is helping to overcome this hurdle. Dr Antonia Hamilton, who leads the social neuroscience group at University College London (UCL) in the UK, is using virtual reality (VR) systems to create realistic social interactions through the INTERACT project, funded by the EU's European Research Council, and the INTERHYTHM project, which is lead by Dr Alexandra Georgescu at UCL.
‘We have built virtual people using a commercial VR system combined with our own code that allows us to control our virtual characters,’ she said. ‘It offers quite a realistic experience of social conversations and allows us to study things like turn-taking and mimicry.’
Using functional near-infrared spectroscopy (fNIRS) – a new brain scanning technique using a cap packed with optical sensors that shine infrared light into the brain – the researchers have explored issues such as trust-building during social interactions.
‘One somewhat controversial issue is how we respond to people mirroring our actions,’ said Dr Hamilton. ‘Dating back to the 1970s, it was believed that mimicking another person’s actions would build rapport and trust.’
Based on the assumption that this is a subtle but effective way of connecting with others, salespeople often copied their customers’ movements – and sometimes echoed their words – in a bid to boost sales of cars, homes and electronics. But the work by Dr Hamilton and her colleagues suggest their efforts may have been wasted.
‘Looking at head movements, we haven’t found any changes in trust when somebody copies you,’ she said. ‘The previous work in this area was not all as rigorous as new technology allows.’
By combining VR and brain scans, Dr Hamilton is bringing a new level of scientific rigour to social neuroscience, allowing theories that were previously hard to test to be subjected to controlled experiments. More assumptions may fall as new data is collected.
For example, people with autism are often said not to imitate the actions of others. Dr Hamilton’s group has found that those on the autistic spectrum do copy others, but they do not use social cues to decide when to imitate, so they often copy in different contexts to typical individuals.
Dr Hamilton also believes the habit of mirroring others is probably something we learn rather than a built-in skill.
‘Mirroring is context-specific,’ she said. ‘If someone reaches out to shake your hand you respond with a complementary action (usually your right hand to theirs) rather than mirroring their action.’
While imitation may or may not help salespeople to sell cars, copying is a valuable part of learning. For people with autism, or others who are less adept at mirroring, identifying problems in this area could help teachers and parents to support children in an educational setting.
One commercial application of this area of research will come in combining VR and artificial intelligence to make artificial agents who interact more naturally with humans.
‘By developing better VR systems and understanding what’s going on in a conversation, we will be able to build better social characters which might sell you a train ticket, for example,’ said Dr Hamilton. ‘We have Siri on our phones but she doesn’t have a face – building in VR might help create a more natural interface.’
If you liked this article, please consider sharing it on social media.
Digitalisation has a role to play in the conservation and promotion of modern-day cultural heritage but should enhance real-life experiences, rather than replace them, experts say.
The challenge of how to rebuild society following conflict is a difficult question that arises all too frequently, but recent studies have demonstrated that putting people at the centre of the process and enabling cooperation on politically neutral issues can help build peace.
A lot of lip service is being paid to making scientific papers free to access but when it comes to action there is a lot of hypocrisy, according to Robert-Jan Smits, the EU's outgoing director-general for research, science and innovation. He has recently been appointed the EU's special envoy on open access, tasked with helping make all publicly funded research in Europe freely available by 2020.
There is a need for renewed political attention, says EU’s new special envoy.
Digital cannot replace personal experiences.
Cultural heritage destruction can be a war crime as sites form part of people's emotional landscape, says Dr van Ess.