Computational Modeling and Cognitive Neuroscience approaches to Memory and Language

1. Methods

Cognitive neuroscientists use a wide range of methods to study the brain and the ways in which it supports cognition. Broadly speaking, most of these can be classified into three groups: functional neuroimaging techniques, lesion studies, and computational modeling. Each of these is reviewed below. Note, however, that other methods not covered here are also frequently used, e.g., structural imaging methods and electrophysiological recording. For a comprehensive introductory overview, see Chapter 3 in:

Gazzaniga, M. S., Ivry, R. B., & Mangun, G. R. (2002). Cognitive Neuroscience: The Biology of the Mind: WW Norton & Company.

a. Functional neuroimaging methods.

Functional neuroimaging methods enable researchers to study brain function by constructing images of the brain as it performs various tasks. The most common methods include functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and positron emission tomography (PET). The relationships between different neuroimaging methods are best understood in terms of each method's spatial and temporal resolution. Spatial resolution refers to the scale at which a method can differentiate between two segments of brain tissue. For example, fMRI has relatively good spatial resolution, potentially allowing a researcher to detect different levels of activation in two brain regions that are only several millimeters apart. In contrast, EEG has relatively poor spatial resolution, and one might only be able to make relatively crude distinctions about where a given signal is coming from (e.g., somewhere in the frontal cortex).

Similarly, temporal resolution refers to the ability of a method to resolve changes in the brain's function over time. A method with good temporal resolution (e.g., EEG) might be able to detect millisecond-by-millisecond changes in levels of activation. Conversely, a method with poor temporal resolution (e.g., PET) might be limited to changes that occur over several seconds or even minutes.

i. Functional Magnetic Resonance Imaging (fMRI)

fMRI is currently the most widely-used technique in cognitive neuroscience. It provides a measure of the amount of blood flowing through a particular region of the brain at a given point in time. Because the amount of blood flowing to a region is roughly proportional to the amount of neuronal activity in that region, the fMRI signal is typically thought of as an index of the amount of cognitive activity occurring in different parts of the brain at any given point in time.

By comparing the amount of brain activation that occurs during one task relative to another (e.g., an experimental condition relative to a control condition), researchers can identify regions thought to be involved in the cognitive processes that distinguish between the two tasks. The oldest and most basic way of conducting such comparisons is to use a block design, in which participants perform a block of trials (totaling, say, 30-40 seconds) of one experimental condition, followed by an equivalent number of trials of another condition. By subtracting the activation during condition B from the activation during condition A, one can identify regions that differentiate between the two tasks.

While the block design is a powerful tool, it has several important limitations, and consequently is no longer used very often in fMRI studies. Instead, researchers have adopted a number of newer and more flexible techniques. Chief among these are rapid event-related fMRI designs, which allow researchers to study activation associated with individual types of trials, dramatically improving the effective temporal resolution of fMRI. A variant design known as the 'mixed' design combines aspects of blocked and event-related fMRI to provide estimates of both sustained and transient brain activation. Most recently, an increasing number of studies have begun to use parametric designs, which allow variables to be modeled in even more natural ways.

Although fMRI is a relatively expensive method, with scans typically costing several hundred dollars per hour, the relatively good spatial resolution (several millimeters) of the method combined with acceptable temporal resolution (1-2 seconds) make it an ideal choice for addressing many research questions of interest to psychologists and neuroscientists.

Huettel, S. A., Song, A. W., & McCarthy, G. (2004). Functional magnetic resonance imaging: Sinauer Associates Sunderland, Mass.

This textbook provides a comprehensive yet accessible overview to functional MRI.

Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B., Frith, C. D., & Frackowiak, R. S. J. (1995). Statistical parametric maps in functional imaging: a general linear approach. Hum Brain Mapp, 2(4), 189-210.

A classic paper demonstrating the application of the standard general linear model (GLM) to fMRI data.

Dale, A. M., & Buckner, R. L. (1997). Selective averaging of rapidly presented individual trials using fMRI. Human Brain Mapping, 5(5), 329-340.

Josephs, O., Turner, R., & Friston, K. (1997). Event-related fMRI. Human Brain Mapping, 5(4), 243-248.

Burock, M. A., Buckner, R. L., Woldorff, M. G., Rosen, B. R., & Dale, A. M. (1998). Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport, 9(16), 3735-3739.

These three papers describe the development of event-related fMRI. The Josephs paper describes what is now referred to as 'slow' event-related fMRI, whereas the Dale and Burock papers demonstrate that fMRI can also be used to study very rapid, interleaved events.

Visscher, K. M., Miezin, F. M., Kelly, J. E., Buckner, R. L., Donaldson, D. I., McAvoy, M. P., et al. (2003). Mixed blocked/event-related designs separate transient and sustained activity in fMRI. Neuroimage, 19(4), 1694-1708.

Visscher et al. validate a 'mixed' design that combines blocked and event-related components. The advantage of this design is that it allows estimation of both sustained and transient neural processes.

Logothetis, N. K., & Wandell, B. A. (2004). Interpreting the BOLD signal. Annu Rev Physiol, 66, 735-769.

Logothetis and Wandell review current knowledge concerning the origin and nature of the BOLD signal that most fMRI studies measure. They review evidence suggesting that the BOLD signal shows a relatively close correspondence with neuronal spiking, but also appears to reflect some properties of neuronal assemblies to a greater extent than others (e.g., inputs and outputs rather than local field potentials).

ii. Positron Emission Tomography (PET)

Although the use of PET to study human brain function preceded fMRI by several years, the two methods follow similar general principles. As with fMRI, PET allows images of brain activation to be acquired while participants perform tasks in a scanner, enabling researchers to quantify the amount of activation in different brain regions associated with different cognitive processes. Importantly, however, PET differs from fMRI in several respects. First, PET has poorer spatial and temporal resolution. Second, PET is invasive: unlike fMRI, use of PET requires injection of a radioactive tracer into participants' bloodstream. As the tracer decays, it emits energy that is detected by the scanner and used to construct the image of brain activation. Third, PET requires additional equipment and expertise in order to produce the tracer, which decays rapidly. Because of these methodological and practical limitations, PET is used relatively infrequently in modern cognitive neuroscience. The notable exception is that PET is still often used to study aspects of brain function that cannot be investigated with fMRI, e.g., levels of neurotransmitter binding. Such investigations are possible with PET because radiotracers can be designed that bind selectively to particular kinds of receptors (e.g., serotonin or dopamine).

Fox, P. T., Mintun, M. A., Reiman, E. M., & Raichle, M. E. (1988). Enhanced detection of focal brain responses using intersubject averaging and change-distribution analysis of subtracted PET images. J Cereb Blood Flow Metab, 8(5), 642-653.

Mintun, M. A., Fox, P. T., & Raichle, M. E. (1989). A highly accurate method of localizing regions of neuronal activation in the human brain with positron emission tomography. J Cereb Blood Flow Metab, 9(1), 96-103.

Petersen, S. E., Fox, P. T., Posner, M. I., Mintun, M., & Raichle, M. E. (1989). Positron emission tomographic studies of the processing of single words. Journal of Cognitive Neuroscience, 1(2), 153-170.

Corbetta, M., Miezin, F. M., Dobmeyer, S., Shulman, G. L., & Petersen, S. E. (1991). Selective and divided attention during visual discriminations of shape, color, and speed: functional anatomy by positron emission tomography. Journal of Neuroscience, 11(8), 2383-2402.

This series of articles describes the development of PET and some its first applications to psychological questions. The Fox and Mintun papers focus on methodological issues, particularly analytical approaches to PET data. The Petersen and Corbetta papers describe applications of PET to single-word reading and visual attention.

iii. Electroencephalography (EEG)

EEG is used to study brain function by measuring electrical activity at the scalp. The principle behind EEG is that, as neuronal assemblies fire, they produce changes in the local electrical field that propagate outwards; these signals can be detected at the scalp using a mesh of electrodes. Although the spatial resolution of EEG is very poor (because the signal at any electrode reflects the summation of propagating electrical potentials throughout the brain), the temporal resolution of EEG is excellent, enabling researchers to study millisecond-by-millisecond changes in brain activation.

When EEG is used to study the brain's response to specific cognitive events (as opposed to measuring naturally-occurring activation over long intervals), the technique is typically referred to as ERP (Event Related or Evoked Response Potential).

Niedermeyer, E. (2005). Electroencephalography: Basic Principles, Clinical Applications, and Related Fields: Lippincott Williams & Wilkins.

This edited volume collects a broad range of contributions on EEG methodology, applications, and relationships to other related methods (e.g., MEG). It covers a range of levels of description, from the biophysics of the EEG signal to the clinical manifestations associated with specific EEG abnormalities. Numerous illustrations of EEG traces are provided.

Luck, S. J. (2005). An Introduction to the Event-related Potential Technique: MIT Press.

Luck provides an accessible overview of current ERP techniques, with particular focus on those experimental approaches commonly used in cognitive neuroscience and related disciplines.

iv. Other functional neuroimaging methods

Although used less frequently than the above three methods, other functional neuroimaging methods are also occasionally used in cognitive neuroscience research. These include magnetoencephalography (MEG) and optical imaging methods, which have a similar spatiotemporal profile to EEG, as well as electrophysiological recording (e.g., recording from individual neurons with an electrode). Although electrophysiological recording is an extremely powerful technique, and is a dominant method in animal studies of brain function, ethical considerations preclude its use in human populations under all but a small range of circumstances.

b. Computational modeling

Although computational modeling can refer to any kind of computational simulation used to model human cognition, cognitive neuroscientists typically use the term to refer specifically to neurally-inspired computational models. Such models, commonly known as connectionist models or neural networks, typically represent knowledge in a distributed rather than localized fashion: a given representation (e.g., of an object, name, or face) is spread across multiple nodes, mimicking the fact that most neural representations are thought to be supported by large-scale assemblies of neurons. The distributed nature of connectionist systems allows them to display many properties commonly observed in real biological systems, e.g., graceful degradation (the ability to preserve some amount of function in the face of damage to part of the system).

Part of the appeal in using connectionist models in cognitive neuroscience rather than the serial computational models found in many areas of cognitive psychology is the desire to demonstrate that complex cognition can be built out of relatively simple individual units (analogous to neurons or neuronal assemblies) that operate on known neurobiological principles. That is, other things being equal, connectionist simulations often have a biological plausibility that may make it easier to reconcile their results with those of other neurobiological approaches, or even to produce predictions that are testable using methods such as fMRI.

O'Reilly, R. C., & Munakata, Y. (2000). Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain: Bradford Book.

O'Reilly and Munakata's textbook simultaneously provides a guide to their own modeling framework (PDP++) and an introduction to the foundational concepts of connectionist models more generally. The book is structured as a series of simulations, with each chapter corresponding to a different simulation and introducing new concepts, allowing students to progressively familiarize themselves with computational modeling techniques.

Rumelhart, D. E., & McClelland, J. L. (1986). Parallel distributed processing: explorations in the microstructure of cognition, vol. 2: psychological and biological models: MIT Press Cambridge, MA, USA.

Although neural network modeling had a long history prior to Rumelhart & McClelland's work, this two-volume series is widely credited with popularizing connectionist simulations and providing a relatively accessible introduction to many of the foundational principles of such simulations. Note that some features and algorithms of the simulations Rumehlart & McClelland discuss are no longer widely used in contemporary implementations.

Dayan, P., & Abbott, L. F. (2001). Theoretical neuroscience: computational and mathematical modeling of neural systems: MIT Press.

Dayan and Abbott's text provides a broad overview of mathematical and computational applications in neuroscience. The book is structured around three themes: (a) the general principles relating environmental inputs to neuronal responses; (b) computational modeling of biologically-detailed, realistic neuronal circuits; and (c) applications to the domain of adaptation and learning, within which Dayan and colleagues have conducted much of their work.

c. Lesion studies

One of the oldest and most powerful ways of studying brain function in humans is the lesion study, in which the cognitive task performance of individuals with specific kinds of brain damage is compared to that of healthy control participants. Unlike neuroimaging methods, which primarily enable correlational inferences, the logic of the lesion study is one of causal necessity. If damage to a particular part of the brain results in an inability to perform a specific task, one can reasonably conclude that that part of the brain (or perhaps, axon fibers passing through that area) is necessary for performing the task. A particularly powerful variant of the lesion study is the double dissociation, in which one demonstrates that region A results in the impairment of behavior X but not Y, whereas region B impairs behavior Y but not X. Cognitive neuroscientists often accord lesion data a special primacy over other results. For example, in cases where neuroimaging studies suggest a region may be involved in a particular function but damage to that region fails to produce task deficits, it is often unclear how to interpret the neuroimaging results, and deference may be given to lesion studies suggesting that the region is not essential for task performance.

While undeniably powerful, lesion studies suffer from two important limitations. First, they are 'natural' experiments; researchers cannot ethically induce lesions in humans, and studies therefore tend to have small and heterogeneous samples. Second, and related, the size, location, and shape of lesions is unpredictable and varies widely across individuals. Consequently, focal lesions that affect only one brain region are relatively rare, and it may be difficult to conclusively establish exactly what parts of the brain are responsible for a particular behavioral deficit.

In recent years, cognitive neuroscientists have begun to use a technique called transcranial magnetic stimulation (TMS) to create temporary and reversible lesions in human participants. By creating a strong and spatially focused magnetic field at the surface of the brain, TMS temporarily 'knocks out' a brain region, thereby allowing one to determine which cognitive functions that region is necessary for. When used appropriately, TMS is harmless, and is relatively cheap compared to many neuroimaging techniques. Its main restriction is that it can only be used to study cortical function, since one cannot safely target subcortical regions.

Shallice, T. (1988). From Neuropsychology to Mental Structure: Cambridge University Press.

Shallice's definitive treatment motivates the use of neuropsychological methods to study the mind, and focuses on theoretical issues surrounding the use of lesion studies. Shallice discusses the inferences afforded by single and double dissociations, applications to existing domains of psychology (particularly language), and the implications of lesion studies for more general conceptions of the human brain as a modular vs. holistic system.

Lezak, M. D. (2004). Neuropsychological assessment: Oxford University Press New York.

Spreen, O., Strauss, E., & Sherman, E. M. S. (2006). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary: Oxford Univ Pr.

These two texts review the principles that guide neuropsychological testing and summarize virtually all of the neuropsychological measures in common use.

Walsh, V., & Cowey, A. (1999). Transcranial magnetic stimulation and cognitive neuroscience. J. Neurosci, 19, 5792-5801.

Walsh and Cowey review the history and development of TMS. They discuss important considerations in using TMS, and the relationship between this method and other developments in neuropsychology and cognitive neuroscience.

d. Combining methods

At the cutting edge of methodological innovation in cognitive neuroscience are attempts to combine multiple methods in order to overcome the spatial or temporal limitations that might affect one method in particular. Most commonly, fMRI is paired with another method such as computational modeling, EEG, or TMS to provide a more complete picture of brain function and support stronger inferences. For example, fMRI might be used to localize a specific part of the brain that can then be targeted more accurately with TMS. A patient scheduled to undergo resection of a particular brain region due to intractable epilepsy might be scanned with fMRI before and after the lesion. And the simultaneous use of fMRI and ERP allows researchers to add temporal detail to the results of fMRI studies, or conversely, to better localize the source of an ERP signal.

Brown, J. W., & Braver, T. S. (2005). Learned predictions of error likelihood in the anterior cingulate cortex. Science, 307(5712), 1118-1121.

Brown and Braver developed a computational model of the anterior cingulate cortex (ACC), a brain region typically thought to support conflict monitoring or error detection. They contrasted this putative role with the novel proposal that the ACC computes the likelihood of making an error. Because the computational model alone could not arbitrate between these two possibilities, they used fMRI to demonstrate that in human participants, the pattern of ACC activation supports an error likelihood rather than conflict monitoring account.

Rushworth, M. F. S., Hadland, K. A., Paus, T., & Sipila, P. K. (2002). Role of the Human Medial Frontal Cortex in Task Switching: A Combined fMRI and TMS Study. Journal of Neurophysiology, 87(5), 2577-2592.

Kahn, I., Pascual-Leone, A., Theoret, H., Fregni, F., Clark, D., & Wagner, A. D. (2005). Transient Disruption of Ventrolateral Prefrontal Cortex During Verbal Encoding Affects Subsequent Memory Performance. Journal of Neurophysiology, 94(1), 688-698.

These two studies used fMRI-guided TMS to disrupt regions associated with task-switching (Rushworth et al.) or verbal encoding (Kahn et al.). In both cases, performance was significantly impaired when the targeted region (but not control regions) were disrupted, providing converging evidence that regions implicated in these cognitive functions in fMRI studies appear to be necessary for successful task performance.

2. Applications to memory and language

Computational/cognitive neuroscience research on memory and language now constitutes a large portion of the total research output in these domains. A comprehensive overview of the many different applications to the study of memory and language is not possible here. Instead, this selective review highlights several areas within each domain that are currently attracting considerable research attention.

a. Neurobiological and computational investigations of memory

i. General Reviews

Tulving, E., & Craik, F. I. M. (2000). The Oxford Handbook of Memory: Oxford University Press US.

This handbook spans the full range of memory research, but contains several chapters on memory, including chapters on neuroanatomical, fMRI, and ERP research.

Gabrieli, J. D. E. (1998). Cognitive neuroscience of human memory. Annual review of psychology, 49, 87-115.

Gabrieli provides a comprehensive overview of cognitive neuroscience research on memory. Topics covered include declarative memory as well as various forms of non-declarative memory, including skill learning, priming, and conditioning. Emphasis is placed on the dissociable roles of different brain systems in supporting different forms of memory.

Milner, B., Squire, L. R., & Kandel, E. R. (2004). Cognitive Neuroscience and the Study of Memory. Cell, 119(5), 719-732.

In this comprehensive, historically-oriented overview, three pioneers of memory research review neurobiological research on memory. The authors selectively focus on two areas of memory research. First, they consider broad conceptual questions regarding the nature of brain systems involved in memory: Where are memories stored? Are there dissociable brain systems for different kinds of memory, and if so, which ones? Second, they address the question of how memories are stored at a molecular level.

ii. Constructive Memory

A dominant paradigm in memory research holds that memory is fundamentally a constructive process. That is, one's recollection of an event is not an infallible snapshot of that event, but depends on a range of constructive processes that occur at encoding or retrieval—e.g., strategic querying of memory, postretrieval monitoring, etc. The constructive memory framework has been extremely generative, leading to research on issues such as the relationship between long-term memory and mechanisms of executive control, the nature of failures of memory (e.g., false memory), and the relationship between memory for past events and the ability to imagine future events.

Moscovitch, M. (1994). Memory and working with memory: Evaluation of a component process model and comparisons with other models. Memory systems, 269-310.

Schacter, D. L., Norman, K. A., & Koutstaal, W. (1998). The cognitive neuroscience of constructive memory. Annual review of psychology, 49, 289-318.

Moscovitch, Schacter, and others (see Schacter et al. for review) have articulated a general constructive memory framework which assumes that memory arises out of the concerted actions of multiple brain systems. In particular, frontal systems involved in executive control and strategic retrieval are thought to co-operate with medial temporal regions such as the hippocampus at both encoding and retrieval to construct episodic memories. The authors articulate general principles intended to account not only for normative memory performance but also memory failures documented in the behavioral literature, e.g., false recognition and spontaneous confabulation.

Moscovitch, M., & Melo, B. (1997). Strategic retrieval and the frontal lobes: evidence from confabulation and amnesia. Neuropsychologia, 35(7), 1017-1034.

Braver, T. S., Barch, D. M., Kelley, W. M., Buckner, R. L., Cohen, N. J., Miezin, F. M., et al. (2001). Direct comparison of prefrontal cortex regions engaged by working and long-term memory tasks. Neuroimage, 14(1 Pt 1), 48-59.

These studies, along with many others, provide evidence that frontal lobe regions typically associated with executive control and working processes are also critically involved in long-term memory, particularly those aspects of long-term memory that require strategic processing (e.g., controlled retrieval). Lesion studies such as Moscovitch & Melo (1997) demonstrate that patients with frontal lobe damage show impairments at a broad range of memory tasks. Neuroimaging studies such as Braver et al. (2001) consistently observe extensive activation of frontal lobe regions implicated in executive control in response to strategic demands for long-term memory encoding and retrieval. For reviews of frontal lobe function in memory, and interactions with MTL regions, see:

Fletcher, P. C., & Henson, R. N. A. (2001). Frontal lobes and human memory. Brain, 124(5), 849-881.

Simons, J. S., & Spiers, H. J. (2003). Prefrontal and medial temporal lobe interactions in long-term memory. Nature Reviews Neuroscience, 4(8), 637-648.

Addis, D. R., Wong, A. T., & Schacter, D. L. (2007). Remembering the past and imagining the future: common and distinct neural substrates during event construction and elaboration. Neuropsychologia, 45(7), 1363-1377.

Szpunar, K. K., Watson, J. M., & McDermott, K. B. (2007). Neural substrates of envisioning the future. Proceedings of the National Academy of Sciences, 104(2), 642.

Consistent with the idea that memory is constructive and depends on distributed neural systems, researchers have recently begun to investigate the extent to which episodic memory retrieval and episodic future depend on similar mechanisms. What is the relationship between memory for past events and the ability to mentally 'project' one's self into the future? The papers by Addis et al. and Szpunar et al. demonstrate that the network of regions involved in remembering the past and imagining the future are virtually identical, and include medial temporal and frontal areas as well as ventromedial and medial posterior cortical regions.

Schacter, D. L., & Addis, D. R. (2007). The cognitive neuroscience of constructive memory: remembering the past and imagining the future. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1481), 773-786.

Buckner, R. L., & Carroll, D. C. (2007). Self-projection and the brain. Trends Cogn. Sci, 11, 49-57.

Hassabis, D., & Maguire, E. A. (2007). Deconstructing episodic memory with construction. Trends in Cognitive Sciences, 11(7), 299-306.

These three papers review evidence supporting the notion that episodic memory and episodic future thought depend on similar neural mechanisms. Although the three perspectives differ somewhat, there is a general convergence on the idea that scene construction constrained by past experience may represent a core function supporting both past and future episodic thought.

iii. Complementary memory systems

Memory researchers have long appreciated that there appear to be multiple memory systems in the brain. However, there is no consensus about which distinctions to draw. Should one distinguish between episodic and semantic memory systems, as early studies of lesion patients with amnesia suggested? Does procedural knowledge represent a distinct form of memory? What about emotional memory? Is recognition memory qualitatively different from episodic recollection? Are there spatially dissociable implicit and explicit memory systems, or do such distinctions reflect different modes of processing occurring within overlapping systems? Answers to such questions have been extensively reviewed by biologically-oriented researchers, e.g.,:

Squire, L. R. (2004). Memory systems of the brain: A brief history and current perspective. Neurobiology of Learning and Memory, 82(3), 171-177.

Schacter, D. L., Wagner, A. D., & Buckner, R. L. (2000). Memory systems of 1999. The Oxford handbook of memory, 627-643.

One distinction that is particularly relevant given the present focus on computational and cognitive neuroscience approaches to memory is the notion of complementary frontal and hippocampal memory system:

McClelland, J. L., McNaughton, B. L., & O’Reilly, R. C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3), 419-437.

O'Reilly, R. C., & Norman, K. A. (2002). Hippocampal and neocortical contributions to memory: advances in the complementary learning systems framework. Trends Cogn Sci, 6(12), 505-510.

Norman, K. A., & O’Reilly, R. C. (2003). Modeling hippocampal and neocortical contributions to recognition memory: A complementary learning systems approach. Psychological Review, 110(4), 611-646.

The complementary systems approach, associated with the work of McClelland, O'Reilly, Norman and colleagues, differs from most other multiple-systems models of memory in that it not only draws heavily on biological data from humans and animals, but has also been implemented in detailed computational simulations. Thus, the complementary systems approach represents perhaps the best current example of a cross-disciplinary computational cognitive neuroscience approach in the domain of human memory.

The core feature of the model is a distinction between two brain systems: one, a slow-learning cortical system that represents information in a distributed manner and provides for long-term storage; the other, a hippocampal system that maintains discrete but temporary representations that allow binding and consolidation of disparate elements into coherent memory traces. O'Reilly and Norman (2002) presents a succinct recent description of the complementary systems model, which are implemented in an elegant neural network model in Norman and O'Reilly (2003).

iv. Subsequent memory effects

The subsequent memory effect refers to the finding that specific patterns of brain activation during the encoding phase of a memory task can predict whether an individual is subsequently able to remember an item or not. A common finding is that items that are subsequently remembered are associated with greater activation in frontal and medial temporal regions than items that are subsequently forgotten. Subsequent memory effects have been demonstrated using both fMRI and ERP, and can be disrupted using TMS. They hold across a wide range of stimuli, including words, pictures, narratives, etc. Thus, the subsequent memory approach provides a powerful methodological tool for studying memory, and provides a cardinal example of how novel methodological developments in cognitive neuroscience can guide and reshape theoretical investigations into memory and other domains.

Wagner, A. D., Schacter, D. L., Rotte, M., Koutstaal, W., Maril, A., Dale, A. M., et al. (1998). Building memories: Remembering and forgetting of verbal experiences as predicted by brain activity. Science, 281(5380), 1188-1191.

Brewer, J. B. (1998). Making Memories: Brain Activity that Predicts How Well Visual Experience Will Be Remembered. Science, 281(5380), 1185-1187.

This pair of studies first demonstrated the ability of event-related fMRI to predict whether subjects would subsequently remember individual items.

Canli, T., Zhao, Z., Brewer, J., Gabrieli, J. D. E., & Cahill, L. (2000). Event-Related Activation in the Human Amygdala Associates with Later Memory for Individual Emotional Experience (Vol. 20, pp. 99-99): Soc Neuroscience.

Canli and colleagues demonstrated that the subsequent memory effect extends to emotional stimuli. Greater amygdala activation in response to highly arousing emotional stimuli was associated with later memory for individual items.

Kahn, I., Pascual-Leone, A., Theoret, H., Fregni, F., Clark, D., & Wagner, A. D. (2005). Transient Disruption of Ventrolateral Prefrontal Cortex During Verbal Encoding Affects Subsequent Memory Performance. Journal of Neurophysiology, 94(1), 688-698.

Kahn et al. demonstrate that subsequent memory for verbally encoded items can be disrupted by applying TMS over areas identified with fMRI.

Adcock, R. A., Thangavel, A., Whitfield-Gabrieli, S., Knutson, B., & Gabrieli, J. D. (2006). Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron, 50(3), 507-517.

Adcock et al. used a subsequent memory paradigm to demonstrate that reward-processing mechanisms can facilitate memory by modulating medial temporal activation even prior to the onset of a stimulus. They provided subjects with monetary cues during encoding, and demonstrated that both subsequent memory performance and functional coupling between dopaminergic midbrain regions and medial temporal structures increased on trials associated with larger financial incentives.

v. Emotional Memory

A large behavioral literature has found that emotion exerts a facilitatory influence on memory. Highly salient events are generally remembered more accurately than neutral events. Neurobiological data suggest this effect depends critically on the amygdala, a region critical for processing emotionally salient information.

Hamann, S. (2001). Cognitive and neural mechanisms of emotional memory. Trends in Cognitive Sciences, 5(9), 394-400.

LaBar, K. S., & Cabeza, R. (2006). Cognitive neuroscience of emotional memory. Nature Reviews Neuroscience, 7(4), 54-64.

Both of these papers review the neural bases of emotional memory, highlighting the role of the amygdala in facilitating memory for salient events. They argue that the facilitatory effect of emotion on memory derives from interactions between the amygdala and medial temporal lobe memory structures such as the hippocampus, as well as attentional networks involving thalamic and frontal cortical regions.

Adolphs, R., Tranel, D., & Buchanan, T. W. (2005). Amygdala damage impairs emotional memory for gist but not details of complex stimuli. Nature Neuroscience, 8, 512-518.

Adolphs et al. provide neuropsychological evidence that amygdala damage selectively impairs memory for salient events. They found that patients with unilateral amygdala lesions, but not patients with lesions in other regions, show a selective impairment at remembering the gist of emotionally arousing events.

Dolcos, F., LaBar, K. S., & Cabeza, R. (2004). Interaction between the Amygdala and the Medial Temporal Lobe Memory System Predicts Better Memory for Emotional Events. Neuron, 42(5), 855-863.

Dolcos et al. demonstrate that emotional stimuli elicit a stronger functional coupling between the amygdala and medial temporal memory structures, consistent with the hypothesis that the amygdala modulates memory consolidation processes supported by the hippocampal system.

Kensinger, E. A., & Corkin, S. (2004). Two routes to emotional memory: Distinct neural processes for valence and arousal. Proceedings of the National Academy of Sciences, 101(9), 3310-3315.

Kensinger and Corkin provide evidence for the existence of two separate systems supporting emotional facilitation of memory. They suggest that emotionally arousing information influences encoding via an amygdala-hippocampus network, whereas valenced information requires controlled processing supported by a prefrontal-hippocampal network.

b. Neural and computational mechanisms of single-word reading

Neurobiological and computational approaches to the study of language increasingly span the full range of language phenomena, from single-word processing to narrative comprehension to syntactic structure. However, for both methodological and theoretical reasons, a disproportionate amount of research has focused on mechanisms of single-word reading. Methodologically, it has proven difficult to disentangle word-level effects from higher-order effects in neuroimaging studies. Theoretically, some researchers feel that understanding of single-word processing has not advanced sufficiently to warrant investigating higher-order processes just yet. The present review therefore focuses on computational and neurobiological investigations of single-word processing. In particular, we highlight visual word recognition as a domain in which computational and neurobiological approaches have led to substantial gains in understanding.

i. Computational perspectives

Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996). Understanding normal and impaired word reading: computational principles in quasi-regular domains. Psychol Rev, 103(1), 56-115.

Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. (2001). DRC: a dual route cascaded model of visual word recognition and reading aloud. Psychol Rev, 108(1), 204-256.

Contemporary research on visual word recognition is dominated by two theoretical frameworks, each implemented in a series of computational models. Connectionist, or "triangle", models suggest that visual word recognition arises from interactive, overlapping phonological, orthographic, and semantic representations rather than discrete pathways. Plaut et al. implement such a network and demonstrate that its behavior mimics that of both healthy humans and lesion patients when performance on standard tasks such as word naming is simulated.

Dual Route models posits two separate pathways that guide visual word recognition: a lexical, or direct pathway, and a sublexical, or assembled pathway. Coltheart et al. (2001) have implemented the DRC model as a serial computational model. They argue that a dual route model can explain data that connectionist models cannot, e.g., the DRC implementation accounts for significantly more variance in human lexical decision and naming performance than competing connectionist models such as Plaut et al. (1996).

For a review of connectionist and DRC models situated within the broader context of the behavioral visual word recognition literature see:

Balota, D. A., Yap, M. J., & Cortese, M. J. (2006). Visual word recognition: The journey from features to meaning (a travel update). In M. J. Traxler & M. A. Gernsbacher (Eds.), Handbook of Psycholinguistics (pp. 285-376). Amsterdam, The Netherlands: Academic Press.

ii. Neuroimaging reviews

Partly driven by methodological constraints, a great deal of neuroimaging research on language and reading has focused on the neural mechanisms of single-word processing. What neural pathways support the rapid recognition of visual word forms? Do different neural systems support access to different kinds of representations, e.g., phonological versus semantic attributes of words? How do the results of neuroimaging studies mesh with the large neuropsychological literature on reading disorders?

Although existing theoretical and computational models of visual-word recognition arguably remain underdetermined by the available neurobiological evidence, neuroimaging studies have made several important contributions to understanding of the mechanisms underlying reading.

Fiez, J. A., & Petersen, S. E. (1998). Neuroimaging studies of word reading. Proc Natl Acad Sci U S A, 95(3), 914-921.

Turkeltaub, P. E., Eden, G. F., Jones, K. M., & Zeffiro, T. A. (2002). Meta-analysis of the functional neuroanatomy of single-word reading: method and validation. Neuroimage, 16(3 Pt 1), 765-780.

These papers present qualitative (Fiez) and quantitative (Turkeltaub) reviews of neuroimaging studies of reading. They converge on a similar set of regions, including classical language regions such as the left inferior frontal gyrus and temporal cortex, as well as areas of inferotemporal cortex, cerebellum, medial frontal cortex, and the supplementary motor area (SMA). Fiez and Petersen assign these regions several different functions, e.g, speech production (e.g., SMA, cerebellum) and auditory processing (superior temporal cortex) versus core language functions such as orthographic-to-phonological transformation (e.g., left frontal cortex).

Vigneau, M., Beaucousin, V., Herve, P. Y., Duffau, H., Crivello, F., Houde, O., et al. (2006). Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence processing. Neuroimage, 30(4), 1414-32

Vigneau et al. provide a comprehensive meta-analytic review of left hemisphere language region involvement in phonological and semantic processing. An important conclusion they draw is that while there appears to be extensive spatial overlap between regions sensitive to different language functions at a coarse grain of analysis, a fine-grained analysis reveals numerous dissociations between phonological, semantic, and syntactic clusters. Vigneau et al. suggest that distinctions between different language functions are best understood in terms of distinct circuits or 'loops' that span multiple left hemisphere regions, e.g., a basic audio-motor loop that includes auditory regions in temporal cortex and motor planning areas in frontal cortex.

Jobard, G., Crivello, F., & Tzourio-Mazoyer, N. (2003). Evaluation of the dual route theory of reading: a metanalysis of 35 neuroimaging studies. Neuroimage, 20(2), 693-712.

In a meta-analysis of 35 studies, Jobard et al. assess neurobiological evidence for dual-route models of visual word recognition. Although much of the evidence is equivocal (e.g., they fail to identify any activation clusters that show greater modulation by the direct lexical route than the phonological route), they conclude that there is at least tentative evidence for distinct lexical-semantic and graphophonological routes, with the former dependent on a circuit that includes inferotemporal cortex, posterior middle temporal gyrus, and inferior frontal gyrus, and the latter dependent on superior temporal, supramarginal, and frontal opercular regions.

iii. The Visual Word Form Area

An ongoing controversy in the cognitive neuroscience of language concerns the role of a region of left inferior temporal cortex that has been dubbed the Visual Word Form Area (VWFA) due to its apparent sensitivity to orthographically regular word forms.

Cohen, L., Dehaene, S., Naccache, L., Lehericy, S., Dehaene-Lambertz, G., Henaff, M. A., et al. (2000). The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients. Brain, 123(Pt 2), 291-307.

Cohen et al. reviewed evidence selectively implicating left inferior temporal cortex in visual word form processing, and reported new data from five control subjects and two patients with hemialexia (an inability to read words presented to the left visual field). They used both fMRI and ERP to characterize the spatial and temporal organization of the Visual Word Form Area. Their results demonstrate that in normal subjects, the VWFA is activated when words are presented to either visual field, but that VWFA activation is only observed in patients when words are presented to the right visual field (corresponding to the left hemisphere of the brain). Thus, Cohen et al. provide the first detailed evidence for the existence of a left inferotemporal region that responds selectively to visual word forms in a location-invariant manner.

McCandliss, B. D., Cohen, L., & Dehaene, S. (2003). The visual word form area: expertise for reading in the fusiform gyrus. Trends Cogn Sci, 7(7), 293-299.

McCandliss et al. review evidence from several studies confirming a robust and highly consistent neural specialization for visual word forms in left inferotemporal cortex. They suggest that the VWFA contains neuron assemblies tuned to orthographically regular strings. Thus, the role they assign the VWFA is primarily a sublexical one.

Price, C. J., & Devlin, J. T. (2003). The myth of the visual word form area. NeuroImage, 19, 473-481.

Cohen, L., & Dehaene, S. (2004). Specialization within the ventral stream: the case for the visual word form area. Neuroimage, 22(1), 466-476.

Price and Devlin argue that proponents of the VWFA have overstated the case for a functional specialization for word forms in left IT cortex. They note that there are no lesion patients whose pure alexia can be attributed to focal VWFA damage, and that the VWFA is activation not only by words but also by other kinds of stimulation, e.g., colors, pictures, and auditory repetition. Price and Devlin suggest that it is inappropriate to identify just one region with visual word form processing given previous data suggesting reading depends on a distributed network of regions. In a response to Price and Devlin's critique, Cohen and Dehaene acknowledge that the VWFA preference for word forms is relative and not absolute; however, they reaffirm their view that there is sufficient evidence to warrant labeling left posterior inferotemporal cortex the Visual Word Form Area. The exchange between Price and Devlin and Cohen and Dehaene is instructive not only as a review of the empirical evidence, but also as a conceptual discussion of the criteria for localizing cognitive functions in the brain.

Gaillard, R., Naccache, L., Pinel, P., Clémenceau, S., Volle, E., Hasboun, D., et al. (2006). Direct Intracranial, fMRI, and Lesion Evidence for the Causal Role of Left Inferotemporal Cortex in Reading. Neuron, 50(2), 191-204.

Gaillard et al. provide strong empirical support for the causal role of the VWFA in reading. They tested a neurosurgical patient scheduled to undergo removal of a portion of the left occipitotemporal cortex before and after surgery. Prior to surgery, the patient performed normally on reading tasks and showed robust VWFA activation when scanned with fMRI. After surgery, the patient developed a selective deficit in reading ability but not in the ability to recognize other visual categories. Scanning with fMRI revealed an absence of activation in the VWFA. Gaillard et al. interpret these results as evidence that the VWFA is causally necessary for normal reading performance, and reject accounts that accord it a more general role in processing both words and other classes of stimuli.