Recent Findings

Demos

* link to shape-from-motion demo, click here.
* link to shape-from-motion demo, binding of fragments into shapes, click here.

Visual Working Memory

Visual working memory (VWM) is a crucial cognitive faculty that supports many different behaviors. In my lab, we sought to understand the relationship between VWM and object recognition, as studies have indicated that these two cognitive processes might share the same neural substrate.

We conducted a study in which participants were given a VWM task (low vs. high load) while simultaneously performing an object recognition task (see Figure below). The number of to-be-recognized objects was varied and objects were presented both unilaterally and bilaterally. An fMRI version of the task was used to examine the cortical mechanisms underlying the relationship between VWM and object recognition.

The behavioral results revealed that under a high VWM load, but not under a low VWM load, there is a marked decrease in object recognition performance. Interestingly, a hemifield asymmetry was observed such that recognition for objects on the left side was more impaired than recognition performance for objects presented on the right side. This finding is akin to the deficit observed in neglect patients, and could potentially represent a transient neglect in healthy adults.

Partial Least Squares analysis of the fMRI data revealed two distinct yet partly overlapping networks contributing to object recognition. The first network consisted of regions traditionally associated with VWM (specifically along the intraparietal sulcus), and the second network was comprised of temporal lobe regions within the ventral visual stream. As visual working memory load increased, we observed a marked increase in activity in the first network, which was correlated with a decrease in recognition accuracy (see figure below). These results support the notion that VWM and object recognition share the same neural substrate. As VWM load increases, there are fewer resources available in these areas for more complex cognitive processes, such as object recognition.

*published in the Journal of Cognitive Neuroscience, 23(10):2968-82.

An EEG Marker of Visual Working Memmory

While Visual Working Memory (VWM) can be measured in behavioural tasks, recent studies have demonstrated the existence of an electrophysiological marker of VWM. In change-detection experiments, participants are presented with a display of a variable number of coloured items and are required to remember as many items as possible. After a delay, a probe display appears and participants indicate if a change has occurred. EEG studies have demonstrated that the maintenance of remembered items during the delay period is associated with increased negative amplitude in the channels contralateral to the attended side, referred to as the contralateral delay activity (CDA).

The Visual Search Task, in which participants search for a target amongst a display of distracters, is an established paradigm in which the role of VWM has been debated. While traditionally thought of as an attentional task, recent studies have suggested that VWM might be utilized to inhibit the searching of previously viewed distracters. We conducted an EEG study involving both a visual search and change-detection task to determine whether the CDA is present in a visual search task and to further understand the role of VWM in visual search.

Our study revealed sustained contralateral activity in the visual search task that was quite similar to the CDA measured in the change-detection task. This sustained contralateral activity was identical to the CDA in terms of peak and mean amplitude (see figure below). Importantly, this sustained activity during visual search increased over the duration of the search and was strongly correlated with the participantsÕ VWM capacities. In addition, this activity did not increase unrestricted with the number of items but plateaued at the capacity of VWM, which is thought to be 3 to 4 items.

This sustained activity in the visual search seems remarkably similar to the CDA and represents the employment of VWM during the task. As such, we could demonstrate the involvement of VWM in a task with continuous presentation of stimuli (as opposed to a delay in change detection) and without direct memory instructions. VWM might play a role in inhibiting participants from searching previously viewed distracters, allowing them to focus their attention on unexplored stimuli.

*published in PloS One(2009),4(11):e8042.

Spatial Neglect and Prism Adaptation

Patients with spatial neglect behave as if half of their world has ceased to exist. Several studies have demonstrated temporary amelioration of neglect by sensory stimulation, but it has been argued that these procedures modulate only the overt behaviour without actually getting to the root of the underlying disorder. To test whether or not the same holds true for prism adaptation (PA), we tested one patient with neglect before and after PA on two visual tasks that allowed us to dissociate behaviour from awareness.

To test his awareness, the patient was required to judge the "happiness" of twelve vertically arranged pairs of chimaeric faces composed of "smiling" and "neutral" half-faces. Patients with left neglect typically select the face in which the right half is depicted as smiling as the "happy" one (healthy subjects exhibit the opposite bias). The required judgment is certainly a behavioural task but of critical interest was whether the patient noticed the chimaeric nature of the stimuli at all and whether the rightward perceptual bias would persist after PA. To test the patient's overt behaviour, he was required to explore images of normal and chimaeric faces, presented individually for 10 seconds each, while his eye movements were recorded.

Before PA, the patient showed a clear rightward perceptual bias, choosing the face depicted as smiling on the right side in 91.67% of all trials. This perceptual bias was accompanied by a clear rightward bias in his eye movements (see figure).

The patient's exploratory eye movements had changed dramatically after PA (see figure), he showed a reversal of his pre-adaptation eye movement pattern. In sharp contrast, his perceptual bias remained unchanged. When he was now asked to judge the happiness of the chimaeric faces, he chose the face that was smiling on the right side again in 91.67% of the trials. After all testing was completed we asked the patient for his subjective impression both of the prism adaptation procedure and the faces stimuli. Remarkably, he explicitly stated that he did not notice anything unusual about the faces stimuli or the induced visual shift caused by the prisms. Thus, before and after PA he was aware of the fact that the stimuli he had been exploring consisted of full faces but he was not aware of the fact that the emotional expression of the two halves of the faces did not match.

We conclude that while the exploratory eye movements of a patient with neglect were clearly shifted after PA, he still showed no awareness for the left side of the stimuli he was now actively exploring. Prism adaptation modulates functions of the parietal lobe, such as the control of eye movements, but fails to influence the underlying mechanisms of neglect.

*published in Neurology 2003, 60: 1826-1829

Susanne Ferber, Ph.D.
Associate Professor
Director of Graduate Studies
University of Toronto
Department of Psychology
100 St. George Street
Toronto, Ontario
M5S 3G3
Canada

ferber@psych.utoronto.ca

Phone: 416 978 1537
Fax: 416 978 4811

Links:

• My Lab
• Department of Psychology
• University of Toronto