The brain has an amazing capacity for recognizing faces. It can identify a face in a few thousandths of a second, form a first impression of its owner and retain the memory for decades.
Central to these abilities is a longstanding puzzle: how the image of a face is encoded by the brain. Two Caltech biologists, Le Chang and Doris Y. Tsao, reported in Thursday’s issue of Cell that they have deciphered the code of how faces are recognized.
Their experiments were based on electrical recordings from face cells, the name given to neurons that respond with a burst of electric signals when an image of a face is presented to the retina.
By noting how face cells in macaque monkeys responded to manipulated photos of some 2,000 human faces, the Caltech team figured out exactly what aspects of the faces triggered the cells and how the features of the face were being encoded. The monkey face recognition system seems to be very similar to that of humans.Continue reading the main story
ADVERTISEMENTContinue reading the main story
Just 200 face cells are required to identify a face, the biologists say. After discovering how its features are encoded, the biologists were able to reconstruct the faces a monkey was looking at just by monitoring the pattern in which its face cells were firing.
The finding needs to be confirmed in other laboratories. But, if correct, it could help understand how the brain encodes all seen objects, as well as suggesting new approaches to artificial vision.
“Cracking the code for faces would definitely be a big deal,” said Brad Duchaine, an expert on face recognition at Dartmouth.
It is a remarkable advance to have identified the dimensions used by the primate brain to decode faces, he added — and impressive that the researchers were able to reconstruct from neural signals the face a monkey is looking at.
Human and monkey brains have evolved dedicated systems for recognizing faces, presumably because, as social animals, survival depends on identifying members of one’s own social group and distinguishing them from strangers.
In both species, the face recognition system consists of face cells that are grouped into patches of at least 10,000 each. There are six of these patches on each side of the brain, situated on the cortex, or surface, just behind the ear.
When the image of a face hits the retina of the eye, it is converted into electric signals. These pass through five or six sets of neurons and are processed at each stage before they reach the face cells. As a result, these cells receive high-level information about the shape and features of a face.
One way in which the brain might identify faces is simply to dedicate a cell to each face. Indeed, there are cells in another part of the brain that do respond to images of specific people.
But this can’t be the way the brain identifies faces, because we can perceive a face we have never seen before. Instead, the Caltech team has found, the brain’s face cells respond to the dimensions and features of a face in an elegantly simple, though abstract, way.
In their experiments, the biologists first identified groups of face cells in a macaque monkey’s brain by magnetic resonance imaging, and then probed individual face cells with a fine electrode that records their signals.
The monkeys were shown photos of human faces that were systematically manipulated to show differences in the size and appearance of facial features.
Cells at a high level in the brain often respond to a medley of things, making it hard to figure out what the cell is meant to do. The Caltech team was able to create faces that showed exactly what each face cell was tuned to.
The tuning of each face cell is to a combination of facial dimensions, a holistic system that explains why when someone shaves off his mustache, his friends may not notice for a while. Some 50 such dimensions are required to identify a face, the Caltech team reports.
These dimensions create a mental “face space” in which an infinite number of faces can be recognized. There is probably an average face, or something like it, at the origin, and the brain measures the deviation from this base.
A newly encountered face might lie five units away from the average face in one dimension, seven units in another, and so forth. Each face cell reads the combined vector of about six of these dimensions. The signals from 200 face cells altogether serve to uniquely identify a face.
Dr. Tsao said she was particularly impressed to find she could design a whole series of faces that a given face cell would not respond to, because they lacked its preferred combination of dimensions. This ruled out a possible alternative method of face identification: that the face cells were comparing incoming images with a set of standard reference faces and looking for differences.
Nancy Kanwisher, a neuroscientist at M.I.T., said it was a major advance to describe what a face cell does and predict how it will respond to a new stimulus. But she suggested that more than 50 dimensions might be needed to capture the full richness of human perception and the idiosyncrasies of particular faces.
“Do we need a dimension for Jack Nicholson’s eyebrows?” she asked.
Dr. Tsao has been working on face cells for 15 years and views her new report, with Dr. Chang, as “the capstone of all these efforts.” She said she hoped her new finding will restore a sense of optimism to neuroscience.
Advances in machine learning have been made by training a computerized mimic of a neural network on a given task. Though the networks are successful, they are also a black box because it is hard to reconstruct how they achieve their result.
“This has given neuroscience a sense of pessimism that the brain is similarly a black box,” she said. “Our paper provides a counterexample. We’re recording from neurons at the highest stage of the visual system and can see that there’s no black box. My bet is that that will be true throughout the brain.”