For the brain, time becomes a factor when dealing just with space, and vice versa – that’s the finding of researchers who are using modern technology to answer an age-old problem.
Philosophers have long debated how the mind combines sensations to form the world we perceive. Now researchers are using modern technology to study processes in the brain and provide insight into how we process sensory information about space and time to form complex experiences.
Professor David Paul Melcher, of the Department of Psychology and Cognitive Sciences at the University of Trento in Italy, is using brain scanners and behavioural methods to try to evaluate the different ways that auditory, tactile and visual information interact with each other to form perceptions of space and time, and particularly how these fit together to form rich experiences.
‘The goal is to look at how perception puts together phenomena: how we start from the very passive view of perception, where people respond almost like a reflex to brief stimulation, to the active construction of an entire experience over time and space,’ said Prof. Melcher, whose COPEST project is funded by a five-year European Research Council (ERC) grant.
Professor Melcher. Image: Bernardinatti PhotoHe has broken the experience of perceiving something down into specific reference frames, meaning the position of one thing in relation to another. This can either relate to time, for example five minutes from now, or to a location in space, for example when something is in front or behind you.
An example would be a mosquito on your hand. It would be registered by the brain with respect to its location on your retina for vision, with respect to your head for hearing, and with respect to your hand for touch. The time at which the information about the mosquito from the different senses reaches the brain also differs dramatically.
‘The main thing is trying to develop the idea of a link between the way that information is combined across different spatial reference frames and different senses, and the duration over which different time information is combined,’ Prof. Melcher said.
A noise or a moving image
Studying how the brain reacts to different types of stimuli, such as a sound or an image, could help to resolve questions about how the brain stores and uses or combines the sense data in order to create a unified perceptual experience.
‘This has been debated a lot in the philosophical literature, that either you recreate the world in your head, or that the world is out there and in effect you put the information together as needed,’ Prof. Melcher said.
‘Either you recreate the world in your head, or ... the world is out there and in effect you put the information together as needed.’
Professor David Paul Melcher, of the University of Trento in Italy
Modern scanning technology such as functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG) can play a major role in shining light on problems that debate alone cannot resolve.
Volunteers in the brain scanner might be exposed to a stimulus – perhaps a noise or a moving image – and then asked to make judgments about the experience.
The researchers monitor and assess the changes in the brain resulting from that particular combination of sensory stimuli, and compare these to the mental state the brain was in before the experience began.
‘So we are trying to run experiments where we create environments in which you shift from trial to trial between these spatial and temporal reference frames, and we are using neuro-imaging to see how individual areas of the brain change and also the way that connectivity between areas changes,’ Prof. Melcher said.
The initial plan was to study how the brain deals with space and time separately, and then to look at them together. But the experiments keep showing interactions between these apparently different reference frames.
‘In some of our temporal experiments we’ve found an effect of space and in some of our spatial experiments an effect of time,’ Prof. Melcher said.
The project, which runs until 2017, is looking at the idea that space and time could appear to mix because in fact much of the brain is hierarchically organised, with different levels of representation.
This could help to explain the relationship between the brain’s rapid response to urgent incoming stimulation, where data is sampled locally and briefly, and higher-level information processing, such as comprehension of complex language.
‘The idea is that higher levels of representation – allowing you to understand things like events, narratives, meaningful actions, stable objects, navigation and so on – combine information over longer periods of time and over multiple spatial reference frames,’ Prof. Melcher said. ‘In order to do something such as understanding a sentence, you need to have a more abstract representation that combines information over longer periods of time, but where you lose, for example, the sound of a particular syllable,’ he added.
Such a hierarchy would help to explain how we process sense data and could help to improve technologies intended to help those with difficulties in sensory processing, such as people who have had strokes or with some forms of autism.
‘A main goal of doing this research is to push along this science to the point where we can actually have a positive impact on people’s lives – everyday people, and of course, particularly patient groups. That’s one of the things that’s most exciting,’ Prof. Melcher said.
A better understanding of the mid- and higher-level perceptual interface with information would also help to improve technologies for creating virtual experiences. These could in turn be used to take the research further.
'If we have the ability to present a compelling virtual experience, then that gives us control over events that we would not have using real objects,’ Prof. Melcher said. ‘So it definitely goes both ways – it’s not a coincidence that research funding agencies on the one hand and companies on the other are spending a lot of energy and money on developing this type of science and new technologies.'
Tiny pieces of plastic, now ubiquitous in the marine environment, have long been a cause of concern for their ability to absorb toxic substances and potentially penetrate the food chain. Now scientists are beginning to understand the level of threat posed to life, by gauging the extent of marine accumulation and tracking the movement of these contaminants.
Testing the safety of medicines and chemicals on organ-like structures developed from various types of stem cells could reduce the reliance on animal testing and streamline chemical and drug development, according to scientists in the Netherlands who are in the early stages of developing such technology.
The world’s largest radio telescope, known as the Square Kilometre Array (SKA) and situated over two continents, will be able to detect the first stars and galaxies emerging from the ‘murk’ at the beginning of the universe and much more besides, according to Professor Phil Diamond, Director General of SKA. He spoke to Horizon at the opening of the Shared Sky art exhibition in Brussels, Belgium on 16 April, where indigenous artists from SKA host nations South Africa and Australia use traditional painting and folk art to explore the themes of astronomy, spirituality and a borderless sky.
Electric ferries and digital communication between ships could help in the quest to decarbonise maritime transport, a sector which is often perceived as being the green option but could still do much to lower its environmental footprint.
Tiny plastic particles could impact human health.
Astronomers could use giant radio telescope from 2025.
The EU’s research chief on his new role.