dual task experiment

Working Memory Model (Baddeley and Hitch)

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

The Working Memory Model, proposed by Baddeley and Hitch in 1974, describes short-term memory as a system with multiple components.

It comprises the central executive, which controls attention and coordinates the phonological loop (handling auditory information), and the visuospatial sketchpad (processing visual and spatial information).

Later, the episodic buffer was added to integrate information across these systems and link to long-term memory. This model suggests that short-term memory is dynamic and multifaceted.

Take-home Messages

• Working memory is a limited capacity store for retaining information for a brief period while performing mental operations on that information.
• Working memory is a multi-component system that includes the central executive, visuospatial sketchpad, phonological loop, and episodic buffer.
• Working memory is important for reasoning, learning, and comprehension.
• Working memory theories assume that complex reasoning and learning tasks require a mental workspace to hold and manipulate information.

Atkinson’s and Shiffrin’s (1968) multi-store model was extremely successful in terms of the amount of research it generated. However, as a result of this research, it became apparent that there were a number of problems with their ideas concerning the characteristics of short-term memory.

Fig 1 . The Working Memory Model (Baddeley and Hitch, 1974)

Baddeley and Hitch (1974) argue that the picture of short-term memory (STM) provided by the Multi-Store Model is far too simple.

According to the Multi-Store Model , STM holds limited amounts of information for short periods of time with relatively little processing.  It is a unitary system. This means it is a single system (or store) without any subsystems. Whereas working memory is a multi-component system (auditory and visual).

Therefore, whereas short-term memory can only hold information, working memory can both retain and process information.

Working memory is short-term memory . However, instead of all information going into one single store, there are different systems for different types of information.

Central Executive

Visuospatial sketchpad (inner eye), phonological loop.

• Phonological Store (inner ear) processes speech perception and stores spoken words we hear for 1-2 seconds.
• Articulatory control process (inner voice) processes speech production, and rehearses and stores verbal information from the phonological store.

Fig 2 . The Working Memory Model Components (Baddeley and Hitch, 1974)

The labels given to the components (see Fig 2) of the working memory reflect their function and the type of information they process and manipulate.

The phonological loop is assumed to be responsible for the manipulation of speech-based information, whereas the visuospatial sketchpad is assumed to be responsible for manipulating visual images.

The model proposes that every component of working memory has a limited capacity, and also that the components are relatively independent of each other.

The Central Executive

The central executive is the most important component of the model, although little is known about how it functions.  It is responsible for monitoring and coordinating the operation of the slave systems (i.e., visuospatial sketchpad and phonological loop) and relates them to long-term  memory (LTM).

The central executive decides which information is attended to and which parts of the working memory to send that information to be dealt with. For example, two activities sometimes come into conflict, such as driving a car and talking.

Rather than hitting a cyclist who is wobbling all over the road, it is preferable to stop talking and concentrate on driving. The central executive directs attention and gives priority to particular activities.

p> The central executive is the most versatile and important component of the working memory system. However, despite its importance in the working-memory model, we know considerably less about this component than the two subsystems it controls.

Baddeley suggests that the central executive acts more like a system which controls attentional processes rather than as a memory store.  This is unlike the phonological loop and the visuospatial sketchpad, which are specialized storage systems. The central executive enables the working memory system to selectively attend to some stimuli and ignore others.

Baddeley (1986) uses the metaphor of a company boss to describe the way in which the central executive operates.  The company boss makes decisions about which issues deserve attention and which should be ignored.

They also select strategies for dealing with problems, but like any person in the company, the boss can only do a limited number of things at the same time. The boss of a company will collect information from a number of different sources.

If we continue applying this metaphor, then we can see the central executive in working memory integrating (i.e., combining) information from two assistants (the phonological loop and the visuospatial sketchpad) and also drawing on information held in a large database (long-term memory).

The Phonological Loop

The phonological loop is the part of working memory that deals with spoken and written material. It consists of two parts (see Figure 3).

The phonological store (linked to speech perception) acts as an inner ear and holds information in a speech-based form (i.e., spoken words) for 1-2 seconds. Spoken words enter the store directly. Written words must first be converted into an articulatory (spoken) code before they can enter the phonological store.

Fig 3 . The phonological loop

The articulatory control process (linked to speech production) acts like an inner voice rehearsing information from the phonological store. It circulates information round and round like a tape loop. This is how we remember a telephone number we have just heard. As long as we keep repeating it, we can retain the information in working memory.

The articulatory control process also converts written material into an articulatory code and transfers it to the phonological store.

The Visuospatial Sketchpad

The visuospatial sketchpad ( inner eye ) deals with visual and spatial information. Visual information refers to what things look like. It is likely that the visuospatial sketchpad plays an important role in helping us keep track of where we are in relation to other objects as we move through our environment (Baddeley, 1997).

As we move around, our position in relation to objects is constantly changing and it is important that we can update this information.  For example, being aware of where we are in relation to desks, chairs and tables when we are walking around a classroom means that we don”t bump into things too often!

The sketchpad also displays and manipulates visual and spatial information held in long-term memory. For example, the spatial layout of your house is held in LTM. Try answering this question: How many windows are there in the front of your house?

You probably find yourself picturing the front of your house and counting the windows. An image has been retrieved from LTM and pictured on the sketchpad.

Evidence suggests that working memory uses two different systems for dealing with visual and verbal information. A visual processing task and a verbal processing task can be performed at the same time.

It is more difficult to perform two visual tasks at the same time because they interfere with each other and performance is reduced. The same applies to performing two verbal tasks at the same time. This supports the view that the phonological loop and the sketchpad are separate systems within working memory.

The Episodic Buffer

The original model was updated by Baddeley (2000) after the model failed to explain the results of various experiments. An additional component was added called the episodic buffer.

The episodic buffer acts as a “backup” store which communicates with both long-term memory and the components of working memory.

Fig 3 . Updated Model to include the Episodic Buffer

Critical Evaluation

Researchers today generally agree that short-term memory is made up of a number of components or subsystems. The working memory model has replaced the idea of a unitary (one part) STM as suggested by the multistore model.

The working memory model explains a lot more than the multistore model. It makes sense of a range of tasks – verbal reasoning, comprehension, reading, problem-solving and visual and spatial processing. The model is supported by considerable experimental evidence.

The working memory applies to real-life tasks:

• reading (phonological loop)
• problem-solving (central executive)
• navigation (visual and spatial processing)

The KF Case Study supports the Working Memory Model. KF suffered brain damage from a motorcycle accident that damaged his short-term memory.

KF’s impairment was mainly for verbal information – his memory for visual information was largely unaffected. This shows that there are separate STM components for visual information (VSS) and verbal information (phonological loop).

The working memory model does not over-emphasize the importance of rehearsal for STM retention, in contrast to the multi-store model.

Empirical Evidence for Working Memory

What evidence is there that working memory exists, that it comprises several parts, that perform different tasks? Working memory is supported by dual-task studies (Baddeley and Hitch, 1976).

The working memory model makes the following two predictions:

1 . If two tasks make use of the same component (of working memory), they cannot be performed successfully together. 2 . If two tasks make use of different components, it should be possible to perform them as well as together as separately.

Key Study: Baddeley and Hitch (1976)

Aim : To investigate if participants can use different parts of working memory at the same time.

Method : Conducted an experiment in which participants were asked to perform two tasks at the same time (dual task technique) – a digit span task which required them to repeat a list of numbers, and a verbal reasoning task which required them to answer true or false to various questions (e.g., B is followed by A?).

Results : As the number of digits increased in the digit span tasks, participants took longer to answer the reasoning questions, but not much longer – only fractions of a second.  And, they didn”t make any more errors in the verbal reasoning tasks as the number of digits increased.

Conclusion : The verbal reasoning task made use of the central executive and the digit span task made use of the phonological loop.

Brain Imaging Studies

Several neuroimaging studies have attempted to identify distinct neural correlates for the phonological loop and visuospatial sketchpad posited by the multi-component model.

For example, some studies have found that tasks tapping phonological storage tend to activate more left-hemisphere perisylvian language areas, whereas visuospatial tasks activate more right posterior regions like the parietal cortex (Smith & Jonides, 1997).

However, the overall pattern of results remains complex and controversial. Meta-analyses often fail to show consistent localization of verbal and visuospatial working memory (Baddeley, 2012).

There is significant overlap in activation, which may reflect binding processes through the episodic buffer, as well as common executive demands.

Differences in paradigms and limitations of neuroimaging methodology further complicate mapping the components of working memory onto distinct brain regions or circuits (Henson, 2001).

While neuroscience offers insight into working memory, Baddeley (2012) argues that clear anatomical localization is unlikely given the distributed and interactive nature of working memory. Specifically, he suggests that each component likely comprises a complex neural circuit rather than a circumscribed brain area.

Additionally, working memory processes are closely interrelated with other systems for attention, perception and long-term memory . Thus, neuroimaging provides clues but has not yet offered definitive evidence to validate the separable storage components posited in the multi-component framework.

Further research using techniques with higher spatial and temporal resolution may help better delineate the neural basis of verbal and visuo-spatial working memory.

Lieberman (1980) criticizes the working memory model as the visuospatial sketchpad (VSS) implies that all spatial information was first visual (they are linked).

However, Lieberman points out that blind people have excellent spatial awareness, although they have never had any visual information. Lieberman argues that the VSS should be separated into two different components: one for visual information and one for spatial.

There is little direct evidence for how the central executive works and what it does. The capacity of the central executive has never been measured.

Working memory only involves STM, so it is not a comprehensive model of memory (as it does not include SM or LTM).

The working memory model does not explain changes in processing ability that occur as the result of practice or time.

State-based models of WM

Early models of working memory proposed specialized storage systems, such as the phonological loop and visuospatial sketchpad, in Baddeley and Hitch’s (1974) influential multi-component model.

However, newer “state-based” models suggest working memory arises from temporarily activating representations that already exist in your brain’s long-term memory or perceptual systems.

For example, you activate your memory of number concepts to remember a phone number. Or, to remember where your keys are, you activate your mental map of the room.

According to state-based models, you hold information in mind by directing your attention to these internal representations. This gives them a temporary “boost” of activity.

More recent state-based models argue against dedicated buffers, and propose that working memory relies on temporarily activating long-term memory representations through attention (Cowan, 1995; Oberauer, 2002) or recruiting perceptual and motor systems (Postle, 2006; D’Esposito, 2007).

Evidence from multivariate pattern analysis (MVPA) of fMRI data largely supports state-based models, rather than dedicated storage buffers.

For example, Lewis-Peacock and Postle (2008) showed MVPA classifiers could decode information being held in working memory from patterns of activity associated with long-term memory for that content.

Other studies have shown stimulus-specific patterns of activity in sensory cortices support the retention of perceptual information (Harrison & Tong, 2009; Serences et al., 2009).

Atkinson, R. C., & Shiffrin, R. M. (1968). Chapter: Human memory: A proposed system and its control processes. In Spence, K. W., & Spence, J. T. The psychology of learning and motivation (Volume 2). New York: Academic Press. pp. 89–195.

Baddeley, A. D. (1986). Working memory . Oxford: Oxford University Press.

Baddeley, A. (1996). Exploring the central executive.  The Quarterly Journal of Experimental Psychology Section A ,  49 (1), 5-28.

Baddeley, A. D. (2000). The episodic buffer: A new component of working memory? Trends in Cognitive Sciences , 4, (11): 417-423.

Baddeley, A. (2010). Working memory.  Current biology ,  20 (4), R136-R140.

Baddeley, A. (2012). Working memory: Theories, models, and controversies.  Annual review of psychology ,  63 , 1-29.

Baddeley, A. D., & Hitch, G. (1974). Working memory. In G.H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 8, pp. 47–89). New York: Academic Press.

Baddeley, A. D., & Lieberman, K. (1980). Spatial working memory. ln R. Nickerson. Attention and Performance, VIII . Hillsdale, N): Erlbaum.

Borella, E., Carretti, B., Sciore, R., Capotosto, E., Taconnat, L., Cornoldi, C., & De Beni, R. (2017). Training working memory in older adults: Is there an advantage of using strategies?.  Psychology and Aging ,  32 (2), 178.

Chai, W. J., Abd Hamid, A. I., & Abdullah, J. M. (2018). Working memory from the psychological and neurosciences perspectives: a review.  Frontiers in Psychology ,  9 , 401.

Cowan, N. (1995). Attention and memory: An integrated framework . Oxford Psychology Series, No. 26. New York: Oxford University Press.

Cowan, N. (2005). Working memory capacity.  Exp. Psychology,  54, 245–246.

Cowan, N. (2008). What are the differences between long-term, short-term, and working memory?.  Progress in brain research ,  169 , 323-338.

Curtis, C.E., & D’Esposito, M. (2003). Persistent activity in the prefrontal cortex during working memory. Trends in Cognitive Sciences, 7 (9), 415-423.

D’Esposito, M. (2007). From cognitive to neural models of working memory. Philosophical Transactions of the Royal Society B: Biological Sciences, 362 (1481), 761-772.

D’Esposito, M., & Postle, B. R. (2015). The cognitive neuroscience of working memory.  Annual review of psychology ,  66 , 115-142.

Fell, J., & Axmacher, N. (2011). The role of phase synchronization in memory processes. Nature Reviews Neuroscience, 12 (2), 105-118.

Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458 (7238), 632-635.

Henson, R. (2001). Neural working memory. In J. Andrade (Ed.), Working memory in perspective (pp. 151-174). Psychology Press.

Lewis-Peacock, J. A., & Postle, B. R. (2008). Temporary activation of long-term memory supports working memory. Journal of Neuroscience, 28 (35), 8765-8771.

Lieberman, M. D. (2000). Introversion and working memory: Central executive differences.  Personality and Individual Differences ,  28 (3), 479-486.

Osaka, M., Osaka, N., Kondo, H., Morishita, M., Fukuyama, H., Aso, T., & Shibasaki, H. (2003). The neural basis of individual differences in working memory capacity: an fMRI study.  NeuroImage ,  18 (3), 789-797.

Serences, J.T., Ester, E.F., Vogel, E.K., & Awh, E. (2009). Stimulus-specific delay activity in human primary visual cortex. Psychological Science, 20( 2), 207-214.

Smith, E.E., & Jonides, J. (1997). Working memory: A view from neuroimaging. Cognitive Psychology, 33 (1), 5-42.

PERSPECTIVE article

A current view on dual-task paradigms and their limitations to capture cognitive load.

• Department of Educational Psychology, Institute of Psychology, Goethe University Frankfurt, Frankfurt, Germany

Dual-task paradigms encompass a broad range of approaches to measure cognitive load in instructional settings. As a common characteristic, an additional task is implemented alongside a learning task to capture the individual’s unengaged cognitive capacities during the learning process. Measures to determine these capacities are, for instance, reaction times and interval errors on the additional task, while the performance on the learning task is to be maintained. Opposite to retrospectively applied subjective ratings, the continuous assessment within a dual-task paradigm allows to simultaneously monitor changes in the performance related to previously defined tasks. Following the Cognitive Load Theory, these changes in performance correspond to cognitive changes related to the establishment of permanently existing knowledge structures. Yet the current state of research indicates a clear lack of standardization of dual-task paradigms over study settings and task procedures. Typically, dual-task designs are adapted uniquely for each study, albeit with some similarities across different settings and task procedures. These similarities range from the type of modality to the frequency used for the additional task. This results in a lack of validity and comparability between studies due to arbitrarily chosen patterns of frequency without a sound scientific base, potentially confounding variables, or undecided adaptation potentials for future studies. In this paper, the lack of validity and comparability between dual-task settings will be presented, the current taxonomies compared and the future steps for a better standardization and implementation discussed.

Introduction

Empirical studies in educational research are often accompanied by the term cognitive load and its measurement. As a construct based on the Cognitive Load Theory ( Sweller et al., 1998 ), it is depicted to reflect the utilization of mental resources, in particular the working memory of an individual, via their level of exhaustion. It is assumed to vary between a higher or lower state, depending on the tasks performed, for instance, writing an essay versus reciting simple vocabulary. By identifying the parameters exhausting the mental resources, instructional settings can be adapted for a higher learning outcome. For this purpose, different methods to measure cognitive load have been developed over the years. Brünken et al. (2003) classify these methods based on their objectivity and causal relationship into four categories: subjective-direct, subjective-indirect, objective-direct, and objective-indirect methods.

Subjective measurements can be summarized as self-reports like questionnaires ( Leppink et al., 2013 ) to assess the perceived mental effort. It is not a method best used for continuous assessment as it is executed retrospectively ( Brünken et al., 2003 ) and seems to be influenced in the sensitivity and accuracy of its results by the timing and frequency of its use ( Chen et al., 2011 ; van Gog et al., 2012 ). Nonetheless, it is so far the only method to attempt to identify the cognitive load distinguished by its three dimensions intrinsic, extraneous, and germane load ( Brünken et al., 2010 ; Leppink et al., 2013 ; Klepsch et al., 2017 ). In contrast, objective measurements assess the performance of the individual simultaneously to the task and vary from physiological methods like electroencephalography ( Antonenko et al., 2010 ) or fMRI ( Whelan, 2007 ) to dual tasks ( Park and Brünken, 2018 ). Chen et al. (2011) found the objective measurements more lacking compared to subjective measurements, because of their lower sensitivity toward small changes in the cognitive load during a task. Brünken et al. (2003) , however, emphasized the difference in accuracy between indirect and direct measurements based on the causal relation of mental effort and experienced cognitive load. In that regard, indirect measurements tend to be unreliable in their interpretation as other factors might have influenced the reported responses ( Brünken et al., 2010 ). Objective-direct measurements like neuroimaging and dual tasks, however, relate directly to the experienced cognitive load ( Brünken et al., 2003 ). And while neuroimaging methods like fMRI seem promising, some limitations arise by the intrusiveness of the technical device. Dual tasks, often also referred to as secondary tasks, present an objective-direct measurement in which two tasks are to be performed simultaneously to observe performance drops in either task. There are two ways to conduct dual tasks, either to induce or to assess cognitive load ( Brünken et al., 2002 ; Klepsch et al., 2017 ). To induce cognitive load, the secondary task is designed to demand the mental resources needed for the primary task, for instance, by tapping or humming a melody ( Park and Brünken 2015 ; Sun and Shea, 2016 ). Therefore, the performance of the primary task is affected. In contrast, the cognitive load can also be assessed by simple decision-making tasks like mathematical tasks ( Lee et al., 2015 ; Tang et al., 2015 ), to observe the performance of the secondary task without influencing the primary task.

Due to these differences in objectivity and causal relation, dual tasks might be seen as an adequate alternative to assess cognitive load as a simultaneous, objective-direct measurement. However, the current state of research showcases a broad variety and heterogeneity of dual-task methods that lack standardization and continuity in their implementation. This in turn hinders the validity and comparability between studies as well as an accurate depiction of the cognitive load throughout the learning process. To further expand on this discrepancy between intent and implementation of dual tasks, this paper will discern the underlying cause of the lack of validity and comparability and present the current state on the taxonomy of dual tasks.

The Lack of Validity and Comparability in Dual-Task Settings

For a better understanding of the proclaimed issues, the validation as formulated by Kane (2013) should be consulted. He states in his argument-based approach that two steps have to be executed to ensure validity: specifying the proposed interpretation or use of the test and evaluating these claims based on appropriate evidence. The evidence is collected through four inferences that build up from a single observation in a test setting, for instance, a multiple-choice question, to the implementation of the target score as a reflection of the real-life performance. In the dual-task setting, it is comparable to question who and what the task is going to assess, which parameters encompass the proposed interpretation and use and if the determined parameters result in its successful accomplishment. However, aside a few exceptions, there is a lack of empirical investigation of secondary tasks, not only regarding their psychometric properties but also in relation to their respective dual-task settings ( Watter et al., 2001 ; Jaeggi et al., 2010 ). Contrary to the assumption of validity being universal for every setting of its respective test ( Kane, 2013 ), validity has to be examined for each new proposed interpretation and use. A similar sentiment can be found in the study of Jaeggi et al. (2010) , where one of the more common secondary tasks, the n-back task, was examined on its validity. The mixed results showed not only difficulty in confirming its validity but also a further need for implementation and examination in different settings.

Another issue arises in the form of lacking comparability between the different dual-task studies. Currently, most dual tasks are custom-made for their specific instructional setting, without any reference to an evaluated and standardized method. Most often, the decision behind the choice of a dual-task method is not further discussed, which in turn might hinder future researchers in continuing or implementing these studies. The different types of dual task not only lack a framework by which a fitting task can be chosen but they also ignore natural limitations in combining different tasks, for instance, a primary motoric task of walking and a secondary task of typing on a phone. This setting would result in a reduced performance of the primary task as the secondary task is naturally intrusive by limiting the field of vision ( Lamberg and Muratori, 2012 ). Nor do they focus as much on the aspect that experience in multitasking can increase the ability to dual task ( Strobach et al., 2015 ) or that dual tasks are great to measure progress in novices but not experts ( Haji et al., 2015 ). Similarly, to the topic of experts, there can be confounding variables, for instance, response automatization ( van Nuland and Rogers, 2016 ) and age, in particular dementia, influencing the participants ( Toosizadeh et al., 2016 ; Sawami et al., 2017 ).

The Current Taxonomy of Dual Tasks

Despite the broad heterogeneity of dual-task methods in instructional settings, one common denominator can be found. A dual-task setting consists of two tasks: the primary task that the researcher wants to observe and the secondary task that has no connection to it beyond its competitive nature. The participant has to perform both tasks concurrently. Apart from that, most attempts at creating a systematic approach toward the variety of dual-task methods have been few and far between and lacking a holistic view.

One of the earlier taxonomies by Brown (1978) postulated four design factors to determine differences between dual-task methods: the information processing demand, the prioritized task performance, the temporal structure and the locus of interference. The first design factor focused on the demand the chosen secondary task puts onto the information processing – either by stimuli with constant or variable demands, for example, changing between easy and complex tasks, or by continuously variable and continuously constant demands not bound to specific stimuli. Another role played the priority given to the secondary task, which could be either primary, secondary, or of equal importance to the primary task. It could be compared to the priorly mentioned ways of inducing or assessing cognitive load ( Brünken et al., 2002 ; Klepsch et al., 2017 ). van Nuland and Rogers (2016) further recommended the task priority to be explicitly stated in the participants’ instructions, as there otherwise might be a task performance trade-off. The third design factor by Brown (1978) focused on the temporal structure of the secondary task, which was either force-paced by the experimental setting, self-paced by the participant or force-paced by the experimental setting within a specific time interval. Lastly, the locus of interference between both tasks could either be at the sensory input or motor output, within the process of the tasks or a combination of all three. He argued though that both sensory input and motor output should not be used as a locus of interference as the dual-task method intends to focus on the mental resources and therefore needs to be used during the process of the mental activity.

Another attempt at categorizing and standardizing dual tasks from a physician’s viewpoint has been made by McIsaac et al. (2015) . Three main categories were stated: tasks by action, task complexity, and task novelty. The category of tasks by action distinguishes between dual tasks consisting of both cognitive, both motor, and cognitive-motor or motor-cognitive primary and secondary task combinations. Therefore, the selection of the proper dual-task method does not only focus on finding a fitting secondary task contentwise but also on its execution in combination with the primary task. The second category, task complexity, is in general a relevant factor but not easy to standardize. The complexity of a task might be felt differently for someone that has never done it versus an experienced user. In this case, task novelty also plays a role as the experience influences the complexity and therefore also the measurement results ( Strobach et al., 2015 ).

Lastly, the recent taxonomy by Wollesen et al. (2019) focused on the different task types. They distinguished between reaction time tasks, controlled processing tasks, visuospatial tasks, mental tracking tasks, working memory tasks, and discrimination tasks. The reaction time tasks were defined as tasks that rely on the reaction time between the sensory stimulus and the behavioral response, for example, pressing a button whenever a light goes on. The controlled processing task expands the reaction time task by the addition of a decision-making process, for example, pressing a button only when a specific symbol appears. The visuospatial task focuses on detecting or processing visual information, for example, finding a symbol in a rotated position. The mental tracking tasks require the memorization of information and are split into two subcategories: the arithmetic tests, for example, counting backward in 3 s (n-back tasks), and the verbal fluency, for example, naming words starting with the same letter. The working memory tasks are a simpler form of the mental tracking tasks as they only require holding information but not processing it, for example, memorizing a picture that has to be found again afterward. Lastly, the discrimination tasks focus on the selective attention toward a specific stimulus, for example, the Go/NoGo tasks in which participants have to either provide or withhold a response depending on the stimulus ( Verbruggen and Logan, 2008 ).

Expanding on the visuospatial tasks presented by Wollesen et al. (2019) , a few more modality-related classifications can be found. The method of tapping or humming melodies ( Park and Brünken 2015 ; Sun and Shea, 2016 ), mathematical tasks ( Lee et al., 2015 ; Tang et al., 2015 ), and visual tasks like reading text or symbols ( Scerbo et al., 2017 ; Wirzberger et al., 2018 ) showcase that the modality between primary and secondary task can differ between auditory/vocally, visually, and motoric tasks. Furthermore, as mentioned by Brown (1978) and Wollesen et al. (2019) , there can be differences in the frequency of the dual task, from event- or interval-based tasks that appear, for example, every 3, 5, or 7 s to continuous tasks that constantly request the participants’ attention. Yet, there is not really a study to be found that uses dual tasks continuously. Most rely on either interval- or event-based frequency.

Outlining a Holistic Taxonomy

The three taxonomies presented lack a holistic view of the dual-task setting and tend to either simplify or strongly limit the classification. For instance, McIsaac et al. (2015) categorizes tasks by action into cognitive or motor tasks even though the description of detecting a cognitive action outside of an fMRI setting seems contradictory. The participant needs to either act motoric or verbally to respond. In contrast, the taxonomy of Wollesen et al. (2019) expands on the task action by displaying a broader variety of secondary tasks but stays limited to only this one parameter. Furthermore, simply the difference between the two dual-task types of inducing and assessing cognitive load needs to be included in a taxonomy as it changes the intent and therefore the use of it. For this purpose, an attempt at a holistic taxonomy was made ( Figure 1 ).

Figure 1 . Holistic taxonomy of dual-task settings with exemplary selection paths.

Parameters relevant to the design of the dual-task setting were included in a stepwise order, ultimately resulting in the selection of the secondary task based on the chosen path. Most of the options are not unique at that, for instance, middle complex tasks can be event-based too. Following the yellow-colored path as an example, after selecting to induce the cognitive load, the stimulus modality and task action modality of the primary task have to be regarded. For instance, choosing a verbal primary task would in turn either hinder a verbal secondary task or restrict the option of higher frequency types in the subsequent parameters. These selections are followed by the complexity of both tasks, and lastly the possible frequency types, frequency rate, and content of the secondary task. Lastly, the task action should show the possible options regarding the prior selections, in this case to either tap or push a button after the sound event, as the secondary task was intended to be auditory in its stimulus but motoric in its action. However, it should be noted that the taxonomy needs to be standardized to be usable as a guide or framework in designing a dual-task setting. The variations of the parameters need to be tested and validated, which, aside from a few exceptions, has yet to be done.

So far, the classifications of the current dual-task paradigms show a mix of different factors without a theoretical framework. Most studies lack a detailed explanation of the reasoning behind the implementation or adaptation of a secondary task, aside the general assumption of using a fitting cognitive load measurement. The presented taxonomies show a broad range of parameters but do not find a common ground. While McIsaac et al. (2015) summarize the different tasks by their action of cognitive versus motoric tasks, the complexity and the novelty of the task, Wollesen et al. (2019) go a bit further and categorize dual tasks by their execution, but with no regards to other parameters. In addition, both taxonomies need to be further specified for a profound framework, especially regarding the different modalities and frequency of dual tasks ( Brown, 1978 ). According to the dual-coding theory ( Paivio, 1971 , 1991 ), both verbal information and nonverbal/visual information interact for a better recall, but their information is processed differently in their own channel. Therefore, there should be a higher regard toward the selection of the task modalities and their influence on the cognitive load measurement. Using the same modalities in primary and secondary tasks might contribute to a higher cognitive load measurement because the information is not already distinguished simply by its sensory input. Further influences might be found in the different temporal structure of dual tasks, in particular the frequency in which the secondary task should be used. So far, even empirical studies that describe their task as continuous, end up being high-interval tasks or tasks that cannot be done over a longer time frame because of physical exhaustion, for instance, constant humming or tapping ( Park and Brünken 2015 ; Sun and Shea, 2016 ). This bears the question on how to change the lack of continuous dual tasks as this particular ability makes it a noteworthy measurement for the cognitive load. Furthermore, it not only needs to be usable over a longer period but also have more variations to be applicable in different settings. For this, it is advisable to look back at the modalities and the restrictions they contain as the physical strain and execution interfere with a continuous dual task. For example, humming a melody might influence an emotional reaction ( Schellenberg et al., 2013 ), but also simply put a physical strain over a longer period. Visual dual tasks would be hard to be kept up in a continuous setting as it would be hard to split the focus of the eyes toward two different tasks, see split-attention effect ( Ayres and Cierniak, 2012 ). A solution might be the use of eye-tracking to adapt the secondary task into a less intrusive method, for example, by changing colors and symbols in the background of the instructional setting to observe the eye movement. In motoric tasks, primary tasks usually cannot be physical as it tends to disturb the secondary task and heightens the physical strain. An exception can be created with physical tasks that work disconnected from each other, for example, tapping on a pedal while sitting and repairing machinery.

Conclusively, future research in relation to dual-task paradigms should take a step back in creating or expanding the different methods of dual tasks and firstly focus on creating a profound and universal taxonomy. Furthermore, the currently existing methods should be evaluated and adapted to create a standardized and reliable use. This of course needs an extensive analysis of the instructional settings and the possibilities to implement dual tasks based on pre-defined variables so that in the future researchers can more easily choose the fitting dual-task paradigms. Dual tasks should furthermore work more toward creating truly continuous tasks to ensure the direct measurement of cognitive load that it proclaims to be ( Brünken et al., 2003 ).

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, and further inquiries can be directed to the corresponding author.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Antonenko, P., Paas, F., Grabner, R., and van Gog, T. (2010). Using electroencephalography to measure cognitive load. Educ. Psychol. Rev. 22, 425–438. doi: 10.1007/s10648-010-9130-y

CrossRef Full Text | Google Scholar

Ayres, P., and Cierniak, G. (2012). “Split-attention effect” in Springer Reference. Encyclopedia of the Sciences of Learning . ed. N. M. Seel (Boston, MA: Springer US), 3172–3175.

Google Scholar

Brown, I. D. (1978). Dual task methods of assessing work-load. Ergonomics 21, 221–224. doi: 10.1080/00140137808931716

Brünken, R., Plass, J. L., and Leutner, D. (2003). Direct measurement of cognitive load in multimedia learning. Educ. Psychol. 38, 53–61. doi: 10.1207/S15326985EP3801_7

Brünken, R., Seufert, T., and Paas, F. (2010). “Measuring cognitive load” in Cognitive Load Theory . eds. J. L. Plass, R. Moreno, and R. Brünken (Cambridge, New York: Cambridge University Press), 181–202.

Brünken, R., Steinbacher, S., Plass, J. L., and Leutner, D. (2002). Assessment of cognitive load in multimedia learning using dual-task methodology. Exp. Psychol. 49, 109–119. doi: 10.1027//1618-3169.49.2.109

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, S., Epps, J., and Chen, F. (2011). “A comparison of four methods for cognitive load measurement,” in Proceedings of the 23rd Australian Computer-Human Interaction Conference (OzCHI 2011) . eds. N. Colineau, C. Paris, and D. Stevenson; Australian National University, Canberra; ACM SIGCHI; November 28–December 2, 2011; ACM, 76–79.

Haji, F. A., Khan, R., Regehr, G., Drake, J., de Ribaupierre, S., and Dubrowski, A. (2015). Measuring cognitive load during simulation-based psychomotor skills training: sensitivity of secondary-task performance and subjective ratings. Adv. Health Sci. Educ. Theory Pract. 20, 1237–1253. doi: 10.1007/s10459-015-9599-8

Jaeggi, S. M., Buschkuehl, M., Perrig, W. J., and Meier, B. (2010). The concurrent validity of the N-back task as a working memory measure. Memory 18, 394–412. doi: 10.1080/09658211003702171

Kane, M. T. (2013). Validating the interpretations and uses of test scores. J. Educ. Meas. 50, 1–73. doi: 10.1111/jedm.12000

Klepsch, M., Schmitz, F., and Seufert, T. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Front. Psychol. 8:1997. doi: 10.3389/fpsyg.2017.01997

Lamberg, E. M., and Muratori, L. M. (2012). Cell phones change the way we walk. Gait Posture 35, 688–690. doi: 10.1016/j.gaitpost.2011.12.005

Lee, H.-I., Park, S., Lim, J., Chang, S. H., Ji, J.-H., Lee, S., et al. (2015). Influence of driver’s career and secondary cognitive task on visual search behavior in driving: a dual-task paradigm. Adv. Phys. Educ. 5, 245–254. doi: 10.4236/ape.2015.54029

Leppink, J., Paas, F., van der Vleuten, C. P. M., van Gog, T., and van Merriënboer, J. J. G. (2013). Development of an instrument for measuring different types of cognitive load. Behav. Res. Methods 45, 1058–1072. doi: 10.3758/s13428-013-0334-1

McIsaac, T. L., Lamberg, E. M., and Muratori, L. M. (2015). Building a framework for a dual task taxonomy. Biomed. Res. Int. 2015:591475. doi: 10.1155/2015/591475

Paivio, A. (1971). Imaginary and verbal processes . New York: Holt, Rinehart and Winston, Inc.

Paivio, A. (1991). Dual coding theory: retrospect and current status. Can. J. Psychol./Revue Canadienne De Psychologie 45, 255–287. doi: 10.1037/h0084295

Park, B., and Brünken, R. (2015). The rhythm method: A new method for measuring cognitive load-An experimental dual-task study. Appl. Cogn. Psychol. 29, 232–243. doi: 10.1002/acp.3100

Park, B., and Brünken, R. (2018). “Secondary task as a measure of cognitive load” in Cognitive Load Measurement and Application: A Theoretical Framework for Meaningful Research and Practice . ed. R. Zheng (New York, NY, US: Routledge/Taylor & Francis Group), 75–92.

Sawami, K., Katahata, Y., Suishu, C., Kamiyoshikawa, T., Fujita, E., Uraoka, M., et al. (2017). Examination on brain training method: effects of n-back task and dual-task. F1000Research 6:116. doi: 10.12688/f1000research.10584.1

Scerbo, M. W., Britt, R. C., and Stefanidis, D. (2017). Differences in mental workload between traditional and single-incision laparoscopic procedures measured with a secondary task. Am. J. Surg. 213, 244–248. doi: 10.1016/j.amjsurg.2016.09.056

Schellenberg, E. G., and Weiss, W. M. (2013). “Music and cognitive abilities” in The Psychology of Music . 3rd Edn. ed. D. Deutsch (London: Academic Press), 499–550.

Strobach, T., Becker, M., Schubert, T., and Kühn, S. (2015). Better dual-task processing in simultaneous interpreters. Front. Psychol. 6:1590. doi: 10.3389/fpsyg.2015.01590

Sun, R., and Shea, J. B. (2016). Probing attention prioritization during dual-task step initiation: a novel method. Exp. Brain Res. 234, 1047–1056. doi: 10.1007/s00221-015-4534-z

Sweller, J., van Merrienboer, J. J. G., and Paas, F. G. W. C. (1998). Cognitive architecture and instructional design. Educ. Psychol. Rev. 10, 251–296. doi: 10.1023/A:1022193728205

Tang, P.-F., Yang, H.-J., Peng, Y.-C., and Chen, H.-Y. (2015). Motor dual-task timed up & go test better identifies prefrailty individuals than single-task timed up & go test. Geriatr. Gerontol. Int. 15, 204–210. doi: 10.1111/ggi.12258

Toosizadeh, N., Najafi, B., Reiman, E. M., Mager, R. M., Veldhuizen, J. K., O’Connor, K., et al. (2016). Upper-extremity dual-task function: an innovative method to assess cognitive impairment in older adults. Front. Aging Neurosci. 8:167. doi: 10.3389/fnagi.2016.00167

van Gog, T., Kirschner, F., Kester, L., and Paas, F. (2012). Timing and frequency of mental effort measurement: evidence in favour of repeated measures. Appl. Cogn. Psychol. 26, 833–839. doi: 10.1002/acp.2883

van Nuland, S. E., and Rogers, K. A. (2016). E-learning, dual-task, and cognitive load: the anatomy of a failed experiment. Anat. Sci. Educ. 9, 186–196. doi: 10.1002/ase.1576

Verbruggen, F., and Logan, G. D. (2008). Response inhibition in the stop-signal paradigm. Trends Cogn. Sci. 12, 418–424. doi: 10.1016/j.tics.2008.07.005

Whelan, R. R. (2007). Neuroimaging of cognitive load in instructional multimedia. Educ. Res. Rev. 2, 1–12. doi: 10.1016/j.edurev.2006.11.001

Watter, S., Geffen, G. M., and Geffen, L. B. (2001). The n-back as a dual-task: P300 morphology under divided attention. Psychophysiology 38, 998–1003. doi: 10.1111/1469-8986.3860998

Wirzberger, M., Herms, R., Bijarsari, S. E., Eibl, M., and Rey, G. D. (2018). Schema-related cognitive load influences performance, speech, and physiology in a dual-task setting: a continuous multi-measure approach. Cognit. Res. 3:46. doi: 10.1186/s41235-018-0138-z

Wollesen, B., Wanstrath, M., van Schooten, K. S., and Delbaere, K. (2019). A taxonomy of cognitive tasks to evaluate cognitive-motor interference on spatiotemoporal gait parameters in older people: a systematic review and meta-analysis. Eur. Rev. Aging Phys. Act. 16:12. doi: 10.1186/s11556-019-0218-1

Keywords: cognitive load, dual task, secondary task, measurement, validity, comparability, cognitive load measurement, taxonomy

Citation: Esmaeili Bijarsari S (2021) A Current View on Dual-Task Paradigms and Their Limitations to Capture Cognitive Load. Front. Psychol . 12:648586. doi: 10.3389/fpsyg.2021.648586

Received: 31 December 2020; Accepted: 28 April 2021; Published: 20 May 2021.

Reviewed by:

Copyright © 2021 Esmaeili Bijarsari. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Shirin Esmaeili Bijarsari, [email protected]

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

• Original article
• Open access
• Published: 04 January 2021

The impact of predictability on dual-task performance and implications for resource-sharing accounts

• Laura Broeker   ORCID: orcid.org/0000-0001-7552-1060 1 ,
• Harald Ewolds 2 ,
• Rita F. de Oliveira 3 ,
• Stefan Künzell 2 &
• Markus Raab 1 , 3  

Cognitive Research: Principles and Implications volume  6 , Article number:  1 ( 2021 ) Cite this article

4044 Accesses

18 Citations

1 Altmetric

Metrics details

The aim of this study was to examine the impact of predictability on dual-task performance by systematically manipulating predictability in either one of two tasks, as well as between tasks. According to capacity-sharing accounts of multitasking, assuming a general pool of resources two tasks can draw upon, predictability should reduce the need for resources and allow more resources to be used by the other task. However, it is currently not well understood what drives resource-allocation policy in dual tasks and which resource allocation policies participants pursue. We used a continuous tracking task together with an audiomotor task and manipulated advance visual information about the tracking path in the first experiment and a sound sequence in the second experiments (2a/b). Results show that performance predominantly improved in the predictable task but not in the unpredictable task, suggesting that participants did not invest more resources into the unpredictable task. One possible explanation was that the re-investment of resources into another task requires some relationship between the tasks. Therefore, in the third experiment, we covaried the two tasks by having sounds 250 ms before turning points in the tracking curve. This enabled participants to improve performance in both tasks, suggesting that resources were shared better between tasks.

According to capacity-sharing accounts, people can flexibly allocate generic processing resources to different competing tasks and stages of processing, which allows concurrent dual tasking (cf. Koch et al. 2018 , regarding a flexibility perspective in multitasking; Meyer and Kieras 1997 ). Limitations in dual-task processing may however occur because the total amount of utilizable resources is limited and may be depleted once different stimulus and response modalities draw on this pool of central attentional resources [Kahneman 1973 ; Posner and Petersen 1990 ; Tombu and Jolicœur 2003 ; but see Wickens ( 2002 , 2008 ), for a modality-specific resources account].

With a limited pool of processing resources, critical aspects to successful dual tasking would thus be the reduction of required resources and an effective resource allocation policy. In this study, we have examined the impact of predictability on dual-task performance and its potential implications for a reduction in resource needs and resource allocation policy. Considering that prediction is a permanently ongoing process of the human perceptual, cognitive and motor system (Bubic et al. 2010 ; de Oliveira et al. 2014 ) and single-task (ST) studies have shown that predictable tasks are processed more efficiently and require fewer attentional resources [as indicated by decreased cortical activity, see Eagleman et al. ( 2009 ], we expected predictability to also have an impact on resource utilization in dual tasks. This view has also been supported by Wahn and König ( 2017 ). They claimed that the degree to which a stimulus in the environment can be predicted influences the allocation of attentional resources, and that one future direction for research is to “investigate the extent to which attentional resource limitations can be circumvented by varying the predictability of the presented stimuli” (p. 91). In continuous tasks, predictability may take the form of visual information about the route ahead enabling participants to plan action required in a few milliseconds (de Oliveira et al. 2014 ). Predictability thus leads to an optimal configuration of the sensory system prior to stimulus onset, which facilitates processing of environmental input (Fougnie et al. 2018 ; Król and Król 2017 ). For this reason, predictability may be of particular importance in dual-task (DT) situations; if predictable tasks require fewer resources, then there should be residual resources available for the other task. For settings where the primary task is predictable, this process has also been termed the trickle-down effect of predictability (Król and Król 2017 ). None of the capacity-sharing accounts do however provide hypotheses about the utilization of residual resources. This might be due to the fact that testing the utilization of resources and residual resources is difficult as the metaphorical construct “resource” is not directly measurable or quantifiable [for a critical evaluation of dual-task theories see Hommel ( 2020 )]. In the literature however, any reduction in costs (either comparing single against dual tasks, or comparing two different dual tasks) and improvements on the dependent variable have been accepted as a proxy for reduced resources (e.g., Fougnie et al. 2018 ; Gopher et al. 1982 ; Wahn and König 2015 ). In addition, it seems advisable to not only look at performance improvements in the primary tasks or dual-task costs (difference between single- and dual-task performance), but to also report performance, and potentially changes, in the secondary task. A closer look at secondary task performance might give an indication of how resources are allocated.

We hypothesize that the human system draws on one general pool of resources [Kahneman 1973 ; Tombu and Jolicœur 2003 ; for an opposing view see Wickens ( 2008 ), as well as the discussion below], and that a predictable primary task frees resources that can be used for a secondary task. This allocation policy should result in improved performance in both tasks. On the contrary, if predictability improves performance in only one (the predictable) task, this would be in line with the previously suggested economic processing mode where humans aim to reduce, not reinvest resources (see also Navon and Gopher 1979 ; Plessow et al. 2012 ). In the literature, there is evidence for both secondary tasks benefiting from a predictable primary task (Cutanda et al. 2015 ; Töllner et al. 2012 ) and for improved performance in the predictable task but not in the secondary task (Corr 2003 ; Ewolds et al. 2017 ). For instance, Cutanda et al. ( 2015 ) showed that when participants concurrently performed an irregular vs. rhythmic auditory response task with an N-back memory task, they responded faster after regular rhythms compared to irregular rhythms, and this was regardless of memory load. By contrast, Ewolds et al. ( 2017 ) used a tracking task which became predictable through learning the track over several days. They showed that performance in a tracking task improved, yet reaction times to the auditory secondary task did not differ between the reactions needed during the learnt versus random tracking segments. Taken together, there is both limited and conflicting empirical evidence regarding the benefits of predictability in the primary task on the secondary task. On the other hand, there is empirical evidence that resource allocation policy can be influenced, and consequently that resources can be unevenly distributed among tasks. For instance, instructing participants to put more emphasis on one vs. the other task (Lehle and Hubner 2009 ; Tsang 2006 ), different perceptions of potential outcome value and the saliency of tasks (Schmidt and Dolis 2009 ; Wickens et al. 2003 , 2015 ; Wickens and Colcombe 2007 ), or distractions during dual-task execution (Strayer and Drews 2007 ) can impact resource allocation policy. However, these studies do not report what implications such an allocation policy might have for the other task which is why further attention should be given to potential drivers of resource reduction and allocation in order to optimize dual-task behavior (Salvucci and Taatgen 2008 ; Tombu and Jolicœur 2003 ).

In this study, we have taken a systematic approach, manipulating predictability in the first task, in the second task, and in both tasks to examine the impact of predictability on dual-task performance and the implications for resource reduction and resource allocation policies.

We used a continuous visuomotor tracking paradigm together with a discrete auditory reaction time task, because it has been shown that this combination of tasks reliably leads to dual-task costs (Ewolds et al. 2017 ; Fougnie et al. 2018 ; Lang et al. 2013 ). More importantly, tracking tasks allow the measure of temporal-spatial variables (i.e., velocity) which give insight into performance changes as soon as another task intervenes. If velocity increases or decreases once participants respond to the auditory task, this indicates that resources are taken away from tracking and we can make inferences about the resource allocation policy. Predictability was manipulated by displaying parts of the tracking path (Experiment 1) and sequencing sounds in the auditory task (Experiment 2a/b). In Experiment 3, we covaried both tasks by playing target sounds 250 ms before the inflection points of the tracking curve and as such the auditory task could be used to predict changes in the tracking task. The covariation created a meaningful relation between tasks, serving as an incentive for participants to reinvest resources into this task or even integrate the tasks into one (Ewolds et al. 2020 ; Schmidtke and Heuer 1997 ).

Experiment 1

Considering that predictability is provided by information in the environment or prior knowledge of a person (Gentsch et al. 2016 ; Körding and Wolpert 2006 ; Wolpert et al. 2003 ), the first experiment manipulated predictability in the tracking environment by providing participants with advance visual information about the tracking path. The continuous task provides a suitable paradigm to examine the hypothesized processes of resource allocation because it allows for flexible scheduling, in contrast to using two discrete tasks. This gives insights into allocation policies at the moment an interfering secondary stimulus occurs. In addition, the information about the tracking path allows feedforward control which can correct positional errors, delays between target and controller, or jerkier trajectories (Engel and Soechting 2000 ; Hill and Raab 2005 ; Lange 2013 ; Scott 2012 ; Weir et al. 1989 ; Wolpert et al. 2011 ). With fewer resources needed in one predictable task there should be residual resources available that can be used for another task. A DT tracking study by Eberts ( 1987 ) already showed that participants receiving visual information on both sides of a moving target improve DT tracking performance, but as no reaction times (RTs) for verbal secondary-task responses were reported, a look into the performance on the secondary task is required in order to make inferences about potential resource allocation.

Participants

In total, 38 participants were recruited on a university campus, via a mailing list or through a participant data bank. Three participants were identified as outliers and were excluded from the analysis, yielding a final sample of 35 participants (22 males and 13 females; aged between 19 and 30 years, M  = 21.80 years, SD = 2.56). An a priori G*Power (version 3.1.9.2) analysis revealed a required sample size of 32 participants for a test power of 0.80 (effect size f  = 0.25 for 2 groups (ST vs. DT) and 5 conditions (predictability), α  = 0.005 corrected for alpha-error accumulation, 1 −  β  = 0.80, r  = 0.5).

Participants in this and the following experiments had self-reported normal or corrected-to-normal vision, normal hearing ability, and no musculoskeletal or neurological disorders. Participants gave written informed consent prior to the experiment and received a small remuneration for taking part. The experiments were approved by the local ethics committee and conformed to the principles of the Declaration of Helsinki 2013.

Participants were seated in a dimly lit room at a viewing distance of 60 cm from a 24-in computer screen (144 Hz, 1920 × 1080 pixel resolution). The tracking software ran on a Windows 10, 64-bit system with a GTX750 graphics card. A spring-loaded joystick was fixed to the table 30 cm from the screen (SpeedLink Dark Tornado, max. sampling rate 60 Hz), and the pedal was fixed to the floor under each participant’s self-reported dominant foot (f-pro USB foot switch, 9 × 5 cm; Fig.  1 ). Participants wore headphones (Sennheiser HD 65TV). The experimenter sat out of view, behind an opaque divider to monitor compliance with the task.

Illustration of the experimental setup

Tasks and display

Visuomotor tracking task.

Participants performed a two-dimensional pursuit-tracking task with a joystick (adapted from Wulf and Schmidt 1997 ) while concurrently reacting to tones by pedal press. Participants operated the joystick with their self-reported dominant hand and controlled a white cursor cross to track a red target square. Unbeknownst to the participants, the cursor cross’s range of motion was limited to the vertical y axis, because its motion on the x -axis was coupled to target speed. This was implemented to prevent participants from moving the cursor straight to the right edge of the screen to cut trials short. Every tracking path was composed of three different segments (adapted from Pew 1974 ), each obeying the formula

with a i and b i being randomly generated numbers ranging from − 5 to 5 and x being a real number in the range [0; 2π]. As different amplitudes have been shown to lead to differences in performance (Magill 1998 ), all randomly generated segments were balanced with regard to mean amplitude beforehand (Wickens 1980 ). This yielded a final set of 41 segments from which the three segments were selected for each trial. Each trial therefore displayed a different path to prevent participants from learning target trajectories (Ewolds et al. 2017 ; Van Roon et al. 2008 ). To avoid the anticipation of peaks (Zhou et al. 2009 ), the red target followed a constant path velocity of 10.5 cm/s, and as a result, trial length varied from 25.6 to 27.9 s depending on the curve’s trajectory (cf., 27 s used in Raab et al. 2013 , and 25 s and 35 s used in de Oliveira et al. 2017 ).

Audiomotor task

The second task was an auditory discrimination task with high-pitched and low-pitched tones occurring randomly along the tracking path (1,086 Hz and 217 Hz, 75-ms duration). Participants reacted to the occurrence of high-pitched tones as fast and as accurately as possible while continuously ignoring the low-pitched tones. Both tones were scaled to the same sound intensity with equal loudness contours (Fletcher and Munson 1933 ). To avoid learning effects, the number of target and distractor sounds per trial varied between 9 and 14 (every 1.9–3.0 s, following Raab et al. 2013 ), but all participants received the same total number of sounds across the whole experiment. The first tone appeared no earlier than 500 ms after the trial had started, and to guarantee sufficient response time, the last tone was presented at least 500 ms before the trial ended. Because average RTs for auditory discrimination in earlier DT studies were 500–950 ms (e.g., Bherer et al. 2005 ), we used a minimum gap between two sounds of 1001 ms, and responses were considered valid only when they were given within 800 ms after the target sound was played.

Manipulation of predictability

The visuomotor tracking task was made predictable by rendering a portion of the tracking path ahead of the target visible (see Fig.  2 ). The visible path was a white line extending 200 ms (to account for visuomotor delay; e.g., Van Rullen and Thorpe 2001 ), 400 ms, 600 ms, or 800 ms ahead of the target square (cf., de Oliveira et al. 2014 ). None of the objects displayed left a trail on the screen. The 0-ms condition represented the unpredictable condition. All five predictability conditions were completed in blocks randomized across participants to avoid training effects (McNeil et al. 2006 ). High-pitched and low-pitched sounds occurred randomly along the tracking path.

In Experiment 1 participants did not receive any information ( a ; 0 ms) or saw b 200 ms, c 400 ms, d 600 ms, and e 800 ms of the tracking path ahead of the red target square. Participants had to follow the red square and its path as accurately as possible by controlling the white cross

In the familiarization phase, participants completed two ST tracking trials to become familiar with the joystick, then two ST auditory trials to familiarize themselves with the high- and low-pitched sounds, and finally two DT trials to become familiar with the DT setting. They were told that during the experiment these conditions would appear in random blocks. Participants were instructed to follow the target square as closely as possible, to react to target tones as fast and as accurately as possible, and to put equal emphasis on both tasks. To stimulate motivation, a feedback window informing participants about their tracking performance and RTs popped up after every five trials (McDowd 1986 ).

In the experimental phase which took approximately 60 min, participants performed 110 trials in total: 50 ST tracking trials (10 × 5 predictability conditions), 10 ST auditory trials, and 50 DT trials (10 × 5 predictability conditions). After completing the experiment, participants answered a questionnaire about their possible use of a specific DT coping strategy. We also asked which predictability condition they felt was the most helpful to improve DT performance by showing five screen shots of the predictability conditions.

Data analysis

To measure tracking performance, we calculated the root-mean-square error (RMSE), as a measure of mean deviation from the target tracking path (Wulf and Schmidt 1997 ; 1 RMSE  ≅  0.56 cm on screen). Performance on the audiomotor task was evaluated by RTs and errors for target sounds. We also measured participants’ absolute velocities. As outlined above, the target moved at a constant path velocity meaning the tracking cross had the same x coordinates as the target. Participants could control upward and downward movement of the tracking cross on the y -axis only. Thus, the tracking cross’s velocity was composed of participants’ y values and the path’s x values and mirrored participants’ speed changes on the y axis. Velocities make it possible to investigate changes in tracking behavior at different intervals around the discrete auditory event. We computed four velocity intervals, Footnote 1 one prior to and three after target sound onset: 200 ms before sound onset until the moment of sound onset; 200 ms after , which was 75–200 ms after sound onset (given audiomotor delay of 75 ms; Vu and Proctor 2002 ); 400 ms after , which was 200–400 ms after sound onset; and 600 ms after , which was from 400 to 600 ms after sound onset.

Prior to the analyses we checked for outliers in the data. Participants were removed from the datasets when RMSE or RT scores exceeded two standard deviations. The first trial of every condition was treated as a familiarization trial and excluded from the analysis. Pairwise comparisons were made using Bonferroni correction ( α  = 0.001), and Greenhouse–Geisser correction was used when sphericity was violated.

We use subscripts to denote the specific conditions of the STs and DTs. For example, we use DT 200 to denote a DT with 200-ms predictability or ST 400 to denote an ST with 400-ms predictability. DT costs (DT cost ) were calculated with the formula [(RMSE ST  − RMSE DT )/RMSE ST ] × 100 (Bock 2008 ).

RMSE and RTs were submitted to 2 × 5 repeated-measures analyses of variance (ANOVAs) with the factors Task Type (ST vs. DT) and Predictability (0 ms vs. 200 ms vs. 400 ms vs. 600 ms vs. 800 ms). Velocities were analyzed with a 5 × 4 repeated-measures ANOVA with the factors predictability (0 ms vs. 200 ms vs. 400 ms vs. 600 ms vs. 800 ms) and interval (200 ms before vs. 200 ms after vs. 400 ms after vs. 600 ms after sound onset).

Questionnaire

Of the 35 participants, two thirds stated that they did not pursue any specific DT strategy; the other third prioritized tracking over tone response. When asked about their preferred predictability condition, 28.6% chose 800 ms, 40.0% chose 600 ms, 25.7% chose 400 ms, and 2.85% each chose 200 ms and 0 ms. Some participants verbally reported that they felt distracted by too much visual information (cf., de Oliveira et al. 2014 ). Participants reported that the 600 ms predictability was most helpful although their best performance was at 400 ms.

There was a significant main effect of task type, F (1, 34) = 11.63, p  = 0.002, η 2  = 0.255, because participants were better in single-task tracking, and there was a significant effect of predictability, F (4, 136) = 165.62, p  < 0.001, η 2  = 0.830, with RMSE being lowest in the 400-ms predictability condition. There was no significant interaction, F (4, 136) = 0.69, p  = 0.597, η 2  = 0.020. There were significant differences between 0 ms and all conditions containing visual information, as well as between 200 ms and the remaining visual conditions, in both single- and dual-task trials. There were no significant differences between 600 and 800 ms (Fig.  3 a). Looking further into those conditions that contained visual information (200–800 ms), we found that the relationship between predictability and RMSE was best described by a quadratic function, F (1, 34) = 26.80, p  < 0.001.

Performance on the 6 predictability conditions. a Tracking performance in Experiment 1 as indicated by root-mean-square error (RMSE). The light gray line represents mean RMSE for dual-task conditions, the dark gray line single-task conditions. Asterisks denote significant differences between single- and dual-task conditions. Dual-task costs, which are added to the graph as percentages, were significantly reduced in the 200-ms and 400-ms conditions. b Reaction times in dual-task conditions. Asterisks denote significant differences between predictability conditions. Conditions varied in predictability (i.e., length of the visible path) from 0 to 800 ms. In both panels, error bars show the standard error

The repeated-measures ANOVA revealed a significant main effect of predictability, F (4, 124) = 7.81, p  = 0.036, η 2  = 0.079, because there was a tendency toward faster tracking with less visual information (0 and 200 ms) and slower tracking with more visual information (600 ms and 800 ms). There was a significant main effect of interval, F (23, 93) = 16.71, p  < 0.001, η 2  = 0.350, because in all visual predictability conditions participants were fastest in the interval of 400 ms after sound onset (Fig.  4 ). There was also a significant Predictability × Interval interaction, F (12, 372) = 3.19, p  < 0.001, η 2  = 0.093, because velocity in the unpredictable condition DT 0 was furthest from target velocity and velocity in the DT 400 condition was closest to target velocity.

Results of velocity analyses in Experiment 1. The dashed horizontal line represents the constant target velocity (10.5 cm/s). Baseline tracking velocity (i.e., 200 ms before the sound onset) was compared against 200 ms, 400 ms, and 600 ms after the sound onset. Error bars show the standard error. Different symbols represent the different predictability conditions

Audiomotor task RTs

There was no significant effect of predictability on RTs, F (4, 136) = 2.11, p  = 0.083, η 2  = 0.058. ST performance for the auditory task was M  = 464 ms (SD = 52).

Errors in the audiomotor task

There were two types of response errors in the auditory task: late responses (Err late ) were given after the valid period which was between 800 ms after the target sound and the onset of the next sound, or missing responses (Err miss ) where there was no response between target onset and the following target onset (Fig.  5 ). There was no significant effect of predictability on late responses, F (4, 124) = 1.84, p  = 0.126, η 2  = 0.056, or on missing responses, F (4, 124) = 1.90, p  = 0.115, η 2  = 0.058. Paired t tests showed significant differences between single- and dual-task error rates (all t (31) > 5.56, all p  < 0.001, all d  > 0.652).

Errors in Experiment 1 were either late responses given later than 800 ms after sound onset (in dark grey) or missing responses that were not given at all (in light grey). Error bars show the standard errors

First, predictability significantly improved visuomotor performance, because dual-task performance improved with visual information. Therefore we conclude that predictability reduces the need for resources. The beneficial effect was most evident for 400 ms, as this was the condition with the lowest RMSE, a more accurate velocity and also lower RT and fewer errors. However, there were no beneficial effects of predictability on secondary task performance, neither on RT nor errors, Footnote 2 which is why we infer that resources were most likely not reinvested. We will discuss the details of these results below.

Regarding the general impact of predictability we conclude that, in line with the basic premise that visual information fosters feedforward control (Weir et al. 1989 ), predictability enabled more accurate movements in the predictable task. As there was no significant improvement of tracking accuracy beyond 400 ms advance information, this amount of information seems sufficient for performance and optimal for feedforward control as already demonstrated by de Oliveira et al. ( 2014 ). It is also in line with research on oculomotor prediction, showing that 500 ms of visual information prior to stimulus occlusion is enough to scale ocular responses (Bennett et al. 2010 ). It is also in line with research on aiming movements, which demonstrated that people who practiced aiming and were provided with 600-ms vision performed equally well when later provided with only 400 ms (Elliott et al. 1995 ). This makes (visual) predictability also different from task difficulty. One could have intuitively suggested that with increasing predictability, the task gets simply easier. However, it has been suggested that the relationship between task difficulty and dual-task performance can be described by a linear relationship (e.g., Isreal et al. 1980 ; McDowd and Craik 1988 ), but our results suggest that visual information may not be unlimitedly beneficial.

Velocity profiles demonstrated that participants across all conditions showed more speed changes approximately 400 ms after onset of the auditory stimulus. This can be interpreted as DT interference, possibly around response selection, considering that RTs were 540 ms on average. This contrasts with interference found in prior tracking studies, where it typically propagates to Task 2 and results in longer RTs. Interference in Task 1 can be the result of limbs’ coupling and thus a neuromuscular effect (neural cross-over effect; Wages et al. 2016 ), an attentional spillover effect (Beilock and Gray 2012 ), or the result of a strategic timing gain to compensate for the reaction to the sound.

Therefore, regarding our aim to make inferences about resource allocation policy, our results are in line with the notion that visual and auditory tracking tasks draw on the same general pool of resources (Fougnie et al. 2018 ), because the tracking task seems to have claimed most of the resources, but the peak in velocity demonstrates that a share of resources was temporarily allocated to the audio task to prepare pedal responses. This share of resources satisfied the minimum requirement of giving a response, yet it seems that not enough residual resources were invested to actually reduce reaction times and improve secondary task performance. According to modality-specific resource accounts (Wickens 2008 ), resources utilized for a visual task should not interfere with demands from an auditory task and thus could not explain the increased tracking velocity. The velocity change was most pronounced in the 0-ms condition, which was the condition without any predictive component and therefore fundamentally different from the other conditions. It seems that the constantly changing environment forced participants to overtake and drop back behind the target more often, which resulted in more overall velocity changes, reflecting the highest need for resources (as also mirrored by no differences between ST and DT performance in RMSE). In contrast, the effect was least pronounced in the 600- and 800-ms conditions, suggesting forward control in response to the upcoming path (Hill and Raab 2005 ) which enabled participants to stay closely behind the target, without the need for constant alignment around the target, and possibly less need for resources.

Another possible explanation for the results is that the increased share of resources to the visually predictable task might be the result of task prioritization. It is plausible that more resources were allocated to the task that was most achievable, which would be in line with increasing error rates for conditions where visual information was present.

Experiment 2a

Experiment 1 showed that participants’ performance improved in the predictable task but not in the secondary, unpredictable task. It seems that most of the resources, drawn from one general pool of resources, were allocated to the predictable task but that residual resources freed by predictability were not reinvested into the secondary task. In Experiment 2, we turned the manipulations around by making the secondary task predictable and leaving the continuous task unpredictable, and examined resource allocation policies for a predictable secondary task.

Prior knowledge , as the second source of predictability (Wolpert and Kawato 1998 ), can be induced via sequences in discrete tasks. Sequences and regularities increase the likelihood of stimulus occurrence and reduce uncertainty about stimulus onset, which enables participants to respond in a timely fashion (Capizzi et al. 2012 ; Nobre et al. 2007 ; Requin et al. 1991 ; Rolke and Hofmann 2007 ). In line with the argument presented above, this should result in enhanced accuracy, considerably reduced RTs, and fewer attentional resources required (de la Rosa et al. 2012 ). Töllner et al. ( 2012 ) explained that knowledge about a stimulus or task leads to a pre-activation of that sensory modality, freeing up general resources and consequently enhancing encoding and leading to faster response selection. If this holds, sequence learning could lead to faster visual processing and shorter motor response execution times in visuomotor tasks (De Jong 1995 ; Sigman and Dehaene 2006 ). In fact, two DT studies (Cutanda et al. 2015 ; de la Rosa et al. 2012 ) demonstrated that regular auditory sequences led to faster reaction times, equally effective in ST and DT conditions and irrespective of high or low load in the working memory task of the DT condition. However, RTs for ST and DT performance of the secondary working memory task were not explicitly contrasted and allocation policies could not be inferred.

For Experiment 2a, we recruited 24 participants. Two participants were excluded from the analyses because testing was terminated due to a technical malfunction, yielding a final sample of 22 participants (10 males and 12 females; aged between 18 and 30 years, M  = 22.82 years, SD = 3.20). As Experiment 1 showed stable performance on the tracking task after very few trials, and thus high correlations among trials [DT 0 : Cronbach’s α  = 0.927 (mean correlation among trials: r = 0.674), DT 200 : α  = 0.935 (r = 0.648), DT 400 : α  = 0.959 (r = 0.769), or DT 600 : α  = 0.943 (r = 0.669)], the a priori sample-size estimations for Experiment 2 were adapted: α  = 0.05, 1 −  β  = 0.80, r  = 0.7 (G*Power 3.1.9.2). This revealed a test power of 0.81 and a required sample size of 22 participants.

The setup was the same as in Experiment 1. We used a 16-bit joystick (Thrustmaster T16000M FCS, max sampling rate 120 Hz).

The task and display were the same as in Experiment 1, but only the unpredictable 0-ms condition was applied.

The secondary task was an auditory discrimination task. In the predictable/sequenced condition, tones were arranged in a sequence with every fourth sound being the high-pitched target sound (see Fig.  6 ) with varying inter-stimulus intervals ranging between 750 and 1,050 ms. In the unpredictable/random condition, high- and low-pitched tones occurred randomly with the same varying inter-stimulus intervals. The number of target sounds per trial varied between 9 and 12 in unpredictable conditions where sounds occurred randomly (every 1.9 to 3.0 s, following Raab et al. 2013 ).

An example of a sequenced dual-task trial in Experiment 2a. A tracking target followed the sinusoidal path, which was invisible to the participants. Circles along the tracking path represent the occurrence of distractor sounds; crosses along the tracking path represent target sounds. All sounds had varying inter-stimulus intervals. The only regularity in the predictable condition was the occurrence of a target sound every fourth sounds

After the familiarization phase, participants took about 30 min to perform 50 trials. Participants began with 10 ST tracking trials, after which they completed four more blocks that were randomized across participants: 10 ST auditory trials with random sounds, 10 ST auditory trials with sequenced sounds, 10 DT trials with random sounds, and 10 DT trials with sequenced sounds.

As in Experiment 1, we calculated the average RMSE and tracking velocities as a measure of tracking performance and RTs plus errors as a measure of performance in the audiomotor task. We used rand to denote trials with randomly occurring sounds and seq to denote trials with sequenced sounds (e.g., DT rand , ST seq ).

The RMSE were compared between ST and DT trials with two paired- t tests (ST vs. DT rand ; ST vs. DT seq ). Further, for DT trials, RMSE was submitted to a one-way repeated-measures ANOVA with factor Sound Order (random vs. sequenced). Velocities were analyzed with a 2 × 4 repeated-measures ANOVA with factors Sound Order (random vs. sequenced) and Interval (200 ms before sound onset vs. 200 ms after onset vs. 400 ms after onset vs. 600 ms after onset). RTs were submitted to a 2 × 2 repeated-measures ANOVA with the factors Sound Order (random vs. sequenced) and Task Type (ST vs. DT).

There was no effect of sound order on RMSE, F (1, 21) = 0.03, p  = 0.873, η 2  = 0.001. Pairwise comparisons between ST and DT conditions revealed significant differences both when sounds were random, t (21) = 3.51, p  = 0.002, d  = 0.749, DT cost  = − 5.76%, and when sounds were sequenced, t (21) = 2.84, p  = 0.010, d  = 0.605, DT cost  = − 5.44%.

The repeated-measures ANOVA revealed a main effect of interval, F (3, 63) = 8.34, p  < 0.001, η 2  = 0.284, because there was an increase in velocity in the 400-ms after interval. There was also a significant Sound Order × Interval interaction, F (3, 63) = 2.87 , p  = 0.043, η 2  = 0.120, because this increase after target sound onset was less pronounced in sequenced compared to random trials (see Fig.  7 , top). There was no main effect of sound order on velocity, F (1, 21) = 0.44, p  = 0.516, η 2  = 0.020. In general, participants had higher velocities compared to the target square across all intervals and conditions, which means that the control cursor was ahead of the target square.

Tracking velocity analyses in Experiments 2a ( a ) and 2b ( b ). Baseline tracking velocity (200 ms before the occurrence of a target sound) was compared against 200 ms, 400 ms, and 600 ms after the sound onset. The dashed horizontal line represents the constant target velocity (10.5 cm/s). Error bars show standard errors

The repeated-measures ANOVA revealed a significant main effect of sound order, F (1, 21) = 136.29, p  < 0.001, η 2  = 0.866, because participants were faster in sequenced compared to random trials. There was also a significant effect of task type, F (1, 21) = 28.01, p  < 0.001, η 2  = 0.571, because participants were faster in ST conditions compared to DT conditions. There was no significant Sound Order × Task Type interaction, F (1, 21) = 3.81, p  = 0.065, η 2  = 0.153 (see Fig.  8 ). Mean RTs are shown in Table 1 .

Reaction time (RT) and root-mean-square error (RMSE) analyses in Experiment 2a ( a ) and 2b ( b ). Light gray lines depict dual-task conditions, dark gray lines depict single-task conditions. DT costs are the differences between single- and dual-task conditions, presented as percentages; asterisks denote significant DT costs, ** p  < .001, * p  < .005. Error bars show standard errors

There were two types of response errors in the auditory task: false responses when participants pressed the pedal in reaction to distractor sounds, and missing responses (Err miss ) which did not occur between two consecutive target onsets (Fig.  9 ). Importantly, responses which were given before sound onset (“premature”) were also counted as missing responses.

Errors in Experiment 2a were either false responses to distractor sounds or missing responses. There were only significant differences between single- and dual-task conditions when sounds were random, with Err _miss , t (21) = 2.96, p  = .007, d  = .711 and Err _false , t (21) = 3.83 , p  < .001, d  = 967, respectively

There was no significant effect of sound order on false responses, F (1, 21) = 2.13, p  = 0.160, η 2  = 0.092. There was a main effect of task type, F (1, 21) = 7.13, p  = 0.014, η 2  = 0.253, because participants performed better in single-task trials. There was no significant Sound Order × Task Type interaction, F (1, 21) = 2.83, p  = 0.105, η 2  = 0.120.

There was a significant effect of sound order on missing responses, F (1, 21) = 6.00, p  = 0.023, η 2  = 0.222, because participants missed fewer responses in the random conditions. There was no effect of task type on missing responses, F (1, 21) = 3.24, p  = 0.086, η 2  = 0.134, and no significant Sound Order × Task Type interaction, F (1, 21) = 3.98, p  = 0.059, η 2  = 0.159.

Contrary to our expectations, participants failed to react to target sounds and erroneously reacted to distractor sounds more often in sequenced conditions than random conditions (see Fig.  9 ). As only responses after sound onset were taken into consideration, the high number of missing responses might be explained by premature responses given before sound onset. Therefore, this result is somewhat inconclusive.

When comparing the difference between single- and dual-task conditions, as expected, participants more frequently failed to react to target sounds and falsely reacted to distractor sounds in dual-task conditions compared to single-task conditions.

First, like in Experiment 1, predictability significantly improved dual-task performance, suggesting that predictability reduced the need for resources. Inferring whether performance improvements only emerge in the predictable task and thus a conclusion about whether or not residual resources were reinvested is contingent on the dependent variables as we outline below.

In general, when comparing ST and DT performance in the two conditions, we found typical performance impairment for DT conditions, which was less pronounced for sequenced trials. Sequenced trials lowered RTs to target sounds, lowered DT cost , and possibly also reduced the need for resources. This effect occurred even though sequences had varying inter-stimulus intervals making the exact timing of sound onset unpredictable (in contrast to the rhythms used by Capizzi et al. 2012 ; Cutanda et al. 2015 ; Halvorson et al. 2013 ). However, the benefit of sequences was not apparent in the tracking’s RMSE. So while RMSE would not support the hypothesis that residual resources from one general pool were reinvested, velocities show a different pattern. Like in Experiment 1, there were more changes in velocity 400 ms after sound onset—interpretable as interference—but this was not significant for sequenced trials (no significant difference between the intervals 200 ms after and 400 ms after). The velocity analysis would thus suggest that predictability in the auditory task freed enough resources to maintain motor control and accuracy in tracking while preparing pedal responses, defeating, or diminishing interference. Considering that tracking is a continuous task, velocities allow a more fine-grained analysis of performance and interference compared to the exclusively spatial measure RMSE.

Experiment 2b

Experiment 2a showed that predictability reduced the need for resources, which have been possibly redistributed to tracking in order to maintain motor control during response preparation. Experiment 2b was designed to challenge this finding by increasing cognitive and motor load in the auditory task. To do so, we transformed the go/no-go task into a choice RT task. Participants were no longer required to ignore the low-pitched tones but had to react to both tones with a double pedal, using both feet.

For Study 2b, we recruited 24 participants. Four participants dropped out during the experiment, leaving a final sample of 20 participants (15 males and 5 females; aged between 18 and 36 years, M  = 24.80 years, SD = 4.32).

The setup was the same as in Experiment 2a, except for the foot pedal, which was now a double-foot switch (Scythe USB 2FS-2), fixed centrally under the table.

Task and display

The tracking task and display were identical to those in Experiment 2a.

The audio task was a choice task, and participants reacted to both tones via the double pedal. They responded to low-pitched (distractor) sounds by pressing the left pedal with the left foot and to high-pitched (target) sounds by pressing the right pedal with the right foot. The sound-order conditions (random and sequenced sounds) remained the same as in Experiment 2a.

After the familiarization phase, participants took approximately 30 min to complete 50 trials: 10 ST tracking trials, 20 ST auditory trials (10 × random sounds, 10 × sequenced sounds), and 20 DT trials (10 × random sounds, 10 × sequenced sounds).

RMSE and velocities were calculated as a measure of visuomotor performance, and RTs were calculated as a measure of audiomotor performance. Differences in RMSE between ST and DT trials were analyzed with two paired- t tests (ST vs. DT rand ; ST vs. DT seq ). Further, for DT trials, RMSE was submitted to a one-way repeated-measures ANOVA with factor Sound Order (random vs. sequenced). Velocities were analyzed with a 2 × 4 repeated-measures ANOVA with factors Sound Order (random vs. sequenced) and Interval (200 ms before sound onset vs. 200 ms after onset vs. 400 ms after onset vs. 600 ms after onset). RTs were submitted to a 2 × 2 ANOVA with factors Sound Order (random vs. sequenced) and Task Type (ST vs. DT). Errors were subject to a 2 × 2 × 2 ANOVA with factors Sound Order (random vs. sequenced), Task Type (ST vs. DT), and Sound Type (target vs. distractor sound).

There was no effect of sound order on RMSE, F (1, 19) = 2.12, p  = 0.162, η 2  = 0.100. Pairwise comparisons between ST and DT trials revealed deteriorated tracking performance in DTs, both when sounds were random, t (19) = 5.91, p  < 0.001, d  = 1.322 (ST: M  = 4.43, SD = 0.35; DT rand : M  = 5.05, SD = 0.68), and when sounds were sequenced, t (19) = 4.61, p  < 0.001, d  = 0.1031 (ST: M  = 4.43, SD = 0.35; DT seq : M  = 4.95, SD = 0.68).

The repeated-measures ANOVA revealed a main effect of interval, F (3, 51) = 9.57, p  < 0.001, η 2  = 0.360 (see Fig.  7 , bottom), but there was no main effect of sound order on velocity, F (1, 17) = 0.92, p  = 0.350, η 2  = 0.051, and no significant interaction, F (3, 51) = 0.50 , p  = 0.686, η 2  = 0.028.

The repeated-measures ANOVA revealed a significant main effect of sound order, F (1, 19) = 86.33, p  < 0.001, η 2  = 0.820, because participants were faster in sequenced compared to random trials. There was a significant main effect of task type, F (1, 19) = 15.84, p  < 0.001, η 2  = 0.455, because participants were generally faster in ST conditions than DT conditions, as in Experiment 2a. However, there was no significant Sound Order × Task Type interaction, F (1, 19) = 0.16, p  = 0.690, η 2  = 0.009 (Fig.  7 ). Mean RTs in the double-pedal experiment are presented in Table 2 .

There were two types of response errors in the auditory task: false responses when participants used the wrong pedal, i.e., left instead of right pedal for target sounds and right instead of left pedal for distractor sounds; and missing responses (Err miss ) for target and distractor sounds. As in Experiment 2a, responses which were given before sound onset (“premature”) were also counted as missing responses. There was large percentage of false responses to target sounds, most likely due to a large amount of premature responses (as they were counted in the interval after distractor sounds). We therefore decided not to consider errors further, but details can be seen in appendix.

In Experiment 2b we showed that predictability had a positive impact on audiomotor performance, even though this effect was less pronounced than in Experiment 2a. Whereas in Experiment 2a the impact of predictability on tracking performance seemed to be dependent on the variable examined, results of Experiment 2b were more clear-cut. There was neither a positive impact on RMSE nor a less pronounced velocity increase for sequenced conditions. We conclude that auditory predictability was strong enough to buffer load induced by simple reactions (Experiment 2a), but that more complex choice reactions require additional resources that could not be reinvested in tracking (possibly because they include the excitation of different hemispheres and the initiation of motor action in different limbs). Note that Experiment 2b included fewer participants than planned and this may put into question the nonsignificant results obtained; this is a limitation of this study. However, given the effect sizes and significant results obtained in the experiment we believe the sample size was adequate for the statistical analysis done.

In sum, Experiments 1 and 2 showed that predictability reduced the need for resources; visual predictability reduced the need for resources in tracking and auditory predictability reduced the need for resources in audiomotor reactions. As there were no improvements in the unpredictable task, it seems unlikely that residuals were reinvested; however, velocity profiles speak for one general rather than modality-specific pools of resources.

It is possible that participants did not reinvest residuals because the two tasks were unrelated. Naturally, participants invested more resources in the tracking task because of its continuous nature. The auditory task was therefore always disruptive, irrespective of whether it was predictable, and required fewer resources. Hence, there may have been little incentive to invest in a disruptive task. If, however, the distractive task was transformed into a helping task, then this could be an incentive for reinvestment. This could be achieved by having one task predict changes in the other task. Therefore, in Experiment 3 we examined the role of task structure and between-task predictability in resource allocation policies.

Experiment 3

So far, Experiments 1 and 2 showed that predictability positively influences dual-task performance, predominantly through improvements in the predictable task. While this result per se could have questioned a general pool of resources, velocity analyses have shown that the auditory task takes away some of the resources from tracking and thus support the generic resource assumption. Yet our data did not support reinvestment of resources into a secondary task.

Wahn and König ( 2015 , 2017 ) argued that resource allocation can be task-dependent and that while object-based vs. spatial tasks (visual and auditory) partially share resources, two spatial tasks (visual and auditory) fully share resources. If this is true, then adding a spatial component to the auditory discrimination task in our study, should enable resource reinvestment. We therefore placed target sounds 250 ms before inflection points of the curve and hypothesized that this would decrease the need for resources and enable participants to reinvest resources. Similar approaches have been taken by task integration studies that covaried two tasks (e.g., de Oliveira et al. 2017 ). Schmidtke and Heuer ( 1997 ) showed for instance that sequences could be more easily implemented when they were temporally correlated with another discrete task. Likewise, de Oliveira et al. ( 2017 ) also positioned target tones 250 ms before inflection points of a tracking path, so that participants could relate the occurrence of a tone to a motor action and found that participants in the covariation group showed significantly better performance in DT than in ST. This effect was pronounced not only in repeating segments of the curve but also in random outer segments, suggesting that covariation can facilitate performance even in otherwise unpredictable environments.

We recruited 22 participants. After we removed one person as an outlier, the final sample consisted of 21 participants (11 males and 10 females; aged between 19 and 35 years, M  = 23.90 years, SD = 3.49). Sample size estimations were based on Experiment 2 (i.e., α  = 0.05, 1 −  β  = 0.80, r  = 0.7, test power of 0.81 and a required sample size of 22 participants).

The setup of Experiment 3 was the same as in Experiment 1.

The tracking task and display were identical to those in the other experiments, but the tracking path was calculated using a different formula. To guarantee enough distance between sounds and curves, the new paths were stretched out. They were composed of three segments, each obeying the formula:

with a i and b i being randomly generated numbers ranging from − 10 to 10 and x being a real number in the range [0; 2π].

Participants responded to high-pitched sounds by pressing on a pedal. High-pitched sounds always occurred 250 ms before a turning point in the tracking curve (integrated conditions); low-pitched sounds occurred randomly between these events and did not require a response by the participant.

After the familiarization phase (DT familiarization with random sounds), participants took about 35 min to perform 60 trials: 20 ST tracking trials, 20 ST auditory trials, and 20 DT trials (10 × random, 10 × integrated). After completing the experiment, participants answered a questionnaire that contained five questions designed to gradually reveal participants’ knowledge of the manipulation. The primary purpose of this questionnaire was to label participants with “knowledge” vs. “no knowledge,” so that knowledge could be entered as a between-subjects factor (see Data Analysis). We first asked whether they had noticed anything special during the experiment, then whether they felt supported or distracted in some of the DT conditions, and then whether they had detected any regularities. After this, participants were told that high-pitched tones served to indicate changes in tracking and were asked whether they had noticed this. If participants answered yes, the fifth question asked them how the tone indicated changes.

For Experiment 3 we use DT cov for DT trials where tracking and auditory task covaried (i.e., stimuli could be integrated) and DT rand for random sounds. We compared the RMSE between ST and DT trials with two paired t tests (ST vs. DT rand ; ST vs. DT cov ). Further, for DT trials, RMSE was submitted to a one-way repeated-measures ANOVA with the factor Sound Location (random vs. covariation). Velocities were analyzed with a two-way repeated-measures ANOVA with the factors Sound Location (random vs. covariation) and Interval (200 ms before onset vs. 200 ms after onset vs. 400 ms after onset vs. 600 ms after onset). RTs were submitted to a 2 × 2 ANOVA with the factors Sound Location (random vs. covariation) and Task Type (ST vs. DT). Knowledge about the task integration was entered into the analysis as a between-subjects factor.

Participants were classified as having knowledge about the manipulation when they were able to correctly describe the task integration manipulation in the fifth question. In total, 10 participants (47.62%) were able to verbalize the positioning of sounds in the questionnaire after finishing the experiment.

Visuomotor tracking task.

There was a significant main effect of sound location on RMSE, F (1, 20) = 5.46, p  = 0.030, η 2  = 0.214, because participants showed better tracking performance when sounds were indicative of turns in the tracking task (DT cov : M  = 3.93, SD = 0.54; DT rand : M  = 4.10, SD = 0.47; ST rand : M  = 3.95, SD = 0.49; Fig.  10 ). Participants who acquired knowledge about the manipulation did not show better tracking performance, Sound Location × Knowledge, F (1, 19) = 2.28, p  = 0.148, η 2  = 0.085.

Performance in Experiment 3 by covaried or random sound location. a Results of tracking performance for dual-task conditions in Experiment 3 as indicated by root-mean-square error (RMSE). b Reaction times (RTs) in milliseconds in dual-task conditions. In both panels, single-task (ST) performance is depicted by a single data point represented by a square

For tracking velocities, there were main effects of sound location, F (1, 20) = 46.14, p  < 0.001, η 2  = 0.698, and interval , F (3, 60) = 23.74, p  < 0.001, η 2  = 0.543, as well as a significant interaction, F (3, 60) = 21.83, p  < 0.001, η 2  = 0.522. For random conditions the velocity pattern was similar to that in Experiments 1 and 2, but for integrated conditions there was a very different pattern as participants slowed down after target sound onset (see Fig.  11 ).

Velocity analyses in Experiment 3. Baseline tracking velocity (200 ms before the occurrence of a target sound) was compared against 200 ms, 400 ms, and 600 ms after the sound onset. The dashed horizontal line represents the constant target velocity (10.5 cm/s). Covaried refers to dual-task trials in which sounds were coupled to the tracking path. Error bars show standard errors

There was a main effect of sound location on RTs, F (1, 20) = 6.59, p  = 0.018, η 2  = 0.248 (Fig.  10 ), showing that participants reacted significantly faster when sounds covaried with the tracking path (DT rand : M  = 483 ms, SD = 30; DT cov : M  = 470 ms, SD = 26; ST rand : M  = 449 ms, SD = 34). Knowledge about the location of sounds did not affect RTs, Sound Location × Knowledge, F (1, 19) = 0.32, p  = 0.581, η 2  = 0.012.

There were two types of response errors in the auditory task: late responses (Err late ) which were given after the valid period (from 800 ms after the target sound until onset of the next sound), and missing responses (Err miss ) where there was no response between two consecutive target onsets (Fig.  12 ). There was a significant main effect of predictability on late responses, F (1, 20) = 5.19, p  = 0.034, η 2  = 0.034, and on missing responses, F (1, 20) = 26.96, p  < 0.001, η 2  = 0.100, showing that errors were larger when the tasks covaried. Paired t tests showed significant differences between single- and dual-task error rates (all t (20) > 7.21, all p  < 0.001, all d  > 1.47), as well as between the random and covaried dual-task conditions (late responses: t (20) = 2.28, p  = 0.034, d  = 0.497; missing responses: t (20) = 5.19, p  < 0.001, d  = 1.132).

Errors in Experiment 3 were either late responses given later than 800 ms after sound onset (in dark grey) or missing responses that were not given at all (in light grey). Error bars show standard errors

In Experiment 3 we found a beneficial effect of predictability on performance in both tasks, which is in contrast with Experiments 1 and 2. Participants had faster reaction times and also fewer tracking errors. Further, they decreased velocity after sound onset, showing that they probably learned to drop back behind the target and prepare motor responses as soon as the sound had announced upcoming changes in the tracking curve. It is conceivable that the target sounds and their clear spatial location thus served as a warning signal, which in turn eased resource allocation. As the sound was no longer intrusive but helpful, resources could be more easily allocated and shared between tasks, which is why we conclude that covariation between tasks improves DT performance by fostering resource reinvestment. Future research could examine possible mechanisms underlying this effect. As suggested by Künzell et al. ( 2018 ) and Koch et al. ( 2018 ), covariation and a shared higher-level goal can prompt participants to treat two tasks as one integrated task (see also Schmidtke and Heuer 1997 ). One such (implicit) goal or action might have been “turn after pressing” rather than the separate goals of “pedal press” and “track the cursor,” but further measures are needed to test such conceptualization mechanisms. Whether the conceptualization is implicit or explicit does not seem to matter, given that the one half of our subjects which was able to verbalize the position of the sound, performed as well as those without explicit knowledge.

General discussion

The purpose of our experiments was to examine the impact of predictability on dual-task performance and gain insight into resource allocation policies. Experiments 1 and 2, which manipulated predictability in either the first or the second task, showed that performance improved in the predictable task, but that residual resources were not reinvested in the other task. This is in line with economical processing accounts (Navon and Gopher 1979 ). In contrast, Experiment 3 covaried two tasks by including an auditory element in the tracking task (and conversely, a spatial element in the auditory task). The results show clear performance improvements in both tasks and thus possibly better resource sharing and reinvestment across tasks. We therefore conclude that predictability helps to circumvent attentional resource limitations (cf., Wahn and König 2017 , p. 91) and that the extent to which resources can be shared among tasks depends on the tasks and their characteristics [see also the claim by Tombu and Jolicœur ( 2003 , p. 4), that “determining exactly which task characteristics affect capacity allocation is an empirical issue that will need to be resolved”].

Overall, our results contribute to the ongoing debate about whether limited resources are specific to modalities. Our findings lend support to the theory of general resources rather than modality-specific resources. It is possible that predictability freed up modality-specific resources that could not be reinvested into the other-modality task. However both the velocity profiles in all Experiments and the results of Experiment 3 (an auditory cue aiding visuomotor performance) demonstrate that the visual tracking task and the auditory RT seem to draw on common central attentional resources. Because velocity data demonstrate that participants are able to continue tracking while responding to sounds [i.e., called hesitations in Klapp et al. ( 1987 ) and Tsang and Chan ( 2015 )], this strengthens the basic premise that parallel processing and execution is possible. However, dual-task costs showed a small impact of the secondary task, so it is possible that the tracking task demands constant resources and a certain share is always taken by this task. If we consider the concurrent use of hand and foot as same-modality response, the results further strengthen the hypothesis that interference occurs when tasks draw on the same resources (Meyer and Kieras 1997 ; Wickens 2002 ). Consistent velocity increases around pedal responses suggest interference at response-activation or execution stages, because motor-related resources would have to be taken away from manual tracking. It has been suggested that such cross-talk can be overcome with practice by integrating two tasks (Bratzke et al. 2009 ; Heuer and Schmidtke 1996 ; Swinnen and Wenderoth 2004 ), which would also be substantiated by the findings of Experiment 3.

An alternative explanation for our findings concerns task prioritization. Wickens et al. ( 2015 ) suggested in their strategic task overload management model that some task characteristics such as salience can foster the prioritization of a task. It is possible that participants did not reinvest resources into the other task in Experiments 1 and 2 because predictability prompted a shift in priority toward the predictable task. In a dual-task learning experiment (Broeker et al. 2020a ), participants performed the tracking task with a constant middle segment (random outer segments) for two days. One group was informed about the repeating segment, the other group was supposed to acquire implicit motor knowledge. On day three, visual information (400 ms) was added to the tracking task. Results showed an additive effect of knowledge and visual information, meaning that both sources of predictability independently improved tracking performance, but importantly, reaction times did not improve. Capacity-sharing accounts support the notion that cognitive capacity can be voluntarily allocated and that allocation may be dependent on task priority (Tombu and Jolicœur 2003 ; Wickens 2002 ). This would mean participants strategically allocated resources to predictable tasks because they were most likely to be accomplished. This interpretation is valid for Experiment 3 because predictability referred to both tasks together and could not be disentangled.

Regarding limitations of our study, theorizing should be addressed first. The interpretations of our results are based on a hypothetical basic premise, namely that resources exist and that resource allocation policy can explain dual-task limitations. As Hommel ( 2020 ) recently emphasized, this assumption can neither be falsified nor be replaced by a mechanistic model so far. With this study, however, we did not aim at establishing a mechanism, yet we are aware of the theoretical discourse of the research field. Second, some technical limitations should be mentioned. For example, we interpreted the impact of sequenced tone structures as overall faster RTs, because participants were instructed to press the pedal after hearing the target sound and therefore only responses given after onset were taken into consideration. Even though there was no rhythm and we varied the inter-stimulus intervals, it is possible that participants learned the sequence so well that they gave “anticipatory responses.” Because the tracking software did not capture early responses, any pedal presses ahead of sound onset counted as very late responses to distractor sounds. Hence late responses to distractor sounds in sequenced trials might actually be very early responses to target sounds and thus neither errors nor anticipatory responses could be interpreted with certainty. Future uses of the paradigm should carefully consider three aspects in order to allow more reliable error analyses: varying trial lengths, using different amounts of distractor and target sounds in every trial, and varying inter-stimulus intervals in order to allow for instance d-prime or similar error analyses.

The unique contributions of this study are that it strengthens empirical evidence for the beneficial impact of predictability on performance in general and for the perceptual, cognitive, and motor system’s ability to use covariations in the environment. The implementation of a continuous task and thereby the temporal variable velocity were an important methodological extension to classic tracking/DT studies. Velocities allowed us to examine resource allocation at the moment of interference because they demonstrate changes in tracking behavior during secondary-task processing. This is not possible only with RMSE, which is the standard measure in DT research. Another methodological extension was contrasting ST and DT performance for both tasks instead of only reporting DT costs. This was important to understand resource allocation. Experiment 1 also contributes an innovative redesign of Wulf and Schmidt’s paradigm ( 1997 ). Past research has mainly manipulated the middle segment to examine motor learning and its impact on dual tasking, but the implementation of visual information allowed us to examine tracking behavior with online information in a fully unpredictable task environment.

The study also offers practical implications and may guide practitioners who design work spaces or training interventions. First, the workload humans face at work often involves continuous processing and parallel handling of multiple tasks. Our results suggest that, where possible in working spaces, either one task should be made predictable or the environment should allow for tasks’ covariation in space or time. For instance it is possible that an air traffic controller can more efficiently attend to radar control and flight progress strips together, because those two tasks are related in time. Ideally, warning signals help to prepare responses in the more complex task to coordinate tasks more effectively. Second, results from the continuous tracking task may generalize to more complex tasks like driving. Future applied studies should investigate task integration in driving to test the role of predictability. For example, manipulating the temporal positioning of braking signs to effectively maintain steering control might ultimately improve safety in driving. Discussions on using smart phones, voice control, navigation systems, and new technology in (semi-)autonomous driving make such investigations societally relevant. In a related study (Broeker et al. 2020b ), participants’ tracking accuracy was compared with performance in a driving simulator and showed that visual predictability has an impact on dual-task driving performance. This is a first step toward generalizing the present results to more applied settings. Third, the finding that task integration improves continuous dual tasks could be relevant for clinical settings and training programs. For instance, if practitioners used co-varying dual tasks such as counting while walking rather than independent dual tasks, performance might improve due to reduced demand for resources and additional risks like falling could be avoided. This would be a promising avenue for further applied research.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

In a pilot analysis, we compared 200 ms, 400 ms, and 600 ms before onset. We hypothesized that if participants had a constant baseline velocity, these intervals would not differ from each other, and this was indeed the case. Therefore, we used only 200 ms before as the baseline velocity before stimulus onset.

This effect is robust given that there was also no effect of predictability on RTs in a replication study with another 28 participants, F (2, 54) = 2.20, p  = .120, η 2  = .075. In this replication study, only dual-task trials were tested and no dual-task costs were examined, which is why the study is not presented in full here.

Beilock, S. L., & Gray, R. (2012). From attentional control to attentional spillover: A skill-level investigation of attention, movement, and performance outcomes. Human Movement Science, 31 (6), 1473–1499. https://doi.org/10.1016/j.humov.2012.02.014 .

Article   PubMed   Google Scholar  

Bennett, S. J., De Xivry, J. J. O., Lefèvre, P., & Barnes, G. R. (2010). Oculomotor prediction of accelerative target motion during occlusion: Long-term and short-term effects. Experimental Brain Research, 204 (4), 493–504. https://doi.org/10.1007/s00221-010-2313-4 .

Bherer, L., Kramer, A. F., Peterson, M. S., Colcombe, S., Erickson, K., & Becic, E. (2005). Training effects on dual-task performance: Are there age-related differences in plasticity of attentional control? Psychology and Aging, 20 (4), 695–709. https://doi.org/10.1037/0882-7974.20.4.695 .

Bock, O. (2008). Dual-task costs while walking increase in old age for some, but not for other tasks: An experimental study of healthy young and elderly persons. Journal of Neuroengineering and Rehabilitation, 5, 27. https://doi.org/10.1186/1743-0003-5-27 .

Article   PubMed   PubMed Central   Google Scholar  

Bratzke, D., Rolke, B., & Ulrich, R. (2009). The source of execution-related dual-task interference: Motor bottleneck or response monitoring? Journal of Experimental Psychology: Human Perception and Performance, 35 (5), 1413–1426. https://doi.org/10.1037/a0015874 .

Broeker, L., Ewolds, H. E., de Oliveira, R. F., Künzell, S., & Raab, M. (2020). Additive effects of prior knowledge and predictive visual information in improving continuous tracking performance. Journal of Cognition . https://doi.org/10.5334/joc.130 .

Broeker, L., Haeger, M., Bock, O., Kretschmann, B., Ewolds, H., Künzell, S., & Raab, M. (2020). How visual information influences dual-task driving and tracking. Experimental Brain Research, 238 (3), 675–687. https://doi.org/10.1007/s00221-020-05744-8 .

Bubic, A., Von Cramon, D. Y., & Schubotz, R. I. (2010). Prediction, cognition and the brain. Frontiers in Human Neuroscience . https://doi.org/10.3389/fnhum.2010.00025 .

Capizzi, M., Sanabria, D., & Correa, Á. (2012). Dissociating controlled from automatic processing in temporal preparation. Cognition, 123 (2), 293–302. https://doi.org/10.1016/j.cognition.2012.02.005 .

Corr, P. J. (2003). Personality and dual-task processing: Disruption of procedural learning by declarative processing. Personality and Individual Differences, 34 (7), 1245–1269. https://doi.org/10.1016/S0191-8869(02)00112-5 .

Article   Google Scholar  

Cutanda, D., Correa, Á., & Sanabria, D. (2015). Auditory temporal preparation induced by rhythmic cues during concurrent auditory working memory tasks. Journal of Experimental Psychology: Human Perception and Performance, 41 (3), 790–797. https://doi.org/10.1037/a0039167 .

De Jong, R. (1995). The role of preparation in overlapping-task performance. The Quarterly Journal of Experimental Psychology. A, Human Experimental Psychology, 48 (1), 2–25. https://doi.org/10.1080/14640749508401372 .

de la Rosa, M. D., Sanabria, D., Capizzi, M., & Correa, A. (2012). Temporal preparation driven by rhythms is resistant to working memory interference. Frontiers in Psychology, 3 (AUG), 1–9. https://doi.org/10.3389/fpsyg.2012.00308 .

de Oliveira, R. F., Billington, J., & Wann, J. P. (2014). Optimal use of visual information in adolescents and young adults with developmental coordination disorder. Experimental Brain Research, 232 (9), 2989–2995. https://doi.org/10.1007/s00221-014-3983-0 .

de Oliveira, R. F., Raab, M., Hegele, M., & Schorer, J. (2017). Task integration facilitates multitasking. Frontiers in Psychology, 8, 398. https://doi.org/10.3389/fpsyg.2017.00398 .

Eagleman, D., Pariyadath, V., & Churchill, S. J. (2009). Predictability engenders more efficient neural responses. Nature Precedings . https://doi.org/10.1038/npre.2009.2847.1 .

Eberts, R. E. (1987). Internal models, tracking strategies, and dual-task performance. Human Factors, 29 (4), 407–419.

Elliott, D., Chua, R., Pollock, B. J., & Lyons, J. (1995). Optimizing the use of vision in manual aiming: The role of practice. The Quarterly Journal of Experimental Psychology Section A, 48 (1), 72–83. https://doi.org/10.1080/14640749508401376 .

Engel, K. C., & Soechting, J. F. (2000). Manual tracking in two dimensions. Journal of Neurophysiology, 83 (6), 3483–3496. https://doi.org/10.1152/jn.2000.83.6.3483 .

Ewolds, H., Broeker, L., de Oliveira, R. F., Raab, M., & Künzell, S. (2020). No impact of instructions and feedback on task integration in motor learning. Memory & Cognition . https://doi.org/10.3758/s13421-020-01094-6 .

Ewolds, H. E., Bröker, L., de Oliveira, R. F., Raab, M., & Künzell, S. (2017). Implicit and explicit knowledge both improve dual task performance in a continuous pursuit tracking task. Frontiers in Psychology . https://doi.org/10.3389/fpsyg.2017.02241 .

Fletcher, H., & Munson, W. A. (1933). Loudness, its definition, measurement and calculation. The Journal of the Acoustical Society of America, 5 (2), 82–108. https://doi.org/10.1121/1.1915637 .

Fougnie, D., Cockhren, J., & Marois, R. (2018). A common source of attention for auditory and visual tracking. Attention, Perception, & Psychophysics . https://doi.org/10.3758/s13414-018-1524-9 .

Gentsch, A., Weber, A., Synofzik, M., Vosgerau, G., & Schütz-Bosbach, S. (2016). Towards a common framework of grounded action cognition: Relating motor control, perception and cognition. Cognition, 146, 81–89. https://doi.org/10.1016/j.cognition.2015.09.010 .

Gopher, D., Brickner, M., & Navon, D. (1982). Different difficulty manipulations interact differently with task emphasis: Evidence for multiple resources. Journal of Experimental Psychology: Human Perception and Performance, 8 (1), 146–157. https://doi.org/10.1037/0096-1523.8.1.146 .

Halvorson, K. M., Wagschal, T. T., & Hazeltine, E. (2013). Conceptualization of task boundaries preserves implicit sequence learning under dual-task conditions. Psychonomic Bulletin & Review, 20 (5), 1005–1010. https://doi.org/10.3758/s13423-013-0409-0 .

Heuer, H., & Schmidtke, V. (1996). Secondary-task effects on sequence learning. Psychological Research , 59 , 119–133. https://doi.org/10.1007/BF01792433 .

Hill, H., & Raab, M. (2005). Analyzing a complex visuomotor tracking task with brain-electrical event related potentials. Human Movement Science, 24 (1), 1–30. https://doi.org/10.1016/j.humov.2004.11.002 .

Hommel, B. (2020). Dual-Task Performance: Theoretical analysis and an event-coding account. Journal of Cognition, 3 (1), 1–13. https://doi.org/10.5334/joc.114 .

Isreal, J. B., Chesney, G. L., Wickens, C. D., & Donchin, E. (1980). P300 and tracking difficulty: Evidence for multiple resources in dual-task performance. Psychophysiology, 17 (3), 259–273. https://doi.org/10.1111/j.1469-8986.1980.tb00146.x .

Kahneman, D. (1973). Attention and effort . Upper Saddle River: Prentice-Hall.

Google Scholar  

Klapp, S. T., Kelly, P. A., & Netick, A. (1987). Hesitations in continuous tracking induced by a concurrent discrete task. Human Factors, 29 (3), 327–337. https://doi.org/10.1177/001872088702900306 .

Koch, I., Poljac, E., Müller, H., & Kiesel, A. (2018). Cognitive structure, flexibility, and plasticity in human multitasking—an integrative review of dual-task and task-switching research. Psychological Bulletin, 144 (6), 557–583. https://doi.org/10.1037/bul0000144 .

Körding, K. P., & Wolpert, D. M. (2006). Bayesian decision theory in sensorimotor control. Trends in Cognitive Sciences, 10 (7), 319–326. https://doi.org/10.1016/j.tics.2006.05.003 .

Król, E. M., & Król, M. (2017). The trickle-down effect of predictability: Secondary task performance benefits from predictability in the primary task. PLoS ONE, 12 (7), 1–20. https://doi.org/10.1371/journal.pone.0180573 .

Künzell, S., Broeker, L., Dignath, D., Ewolds, H., Raab, M., & Thomaschke, R. (2018). What is a task? An ideomotor perspective. Psychological Research Psychologische Forschung, 82 (1), 4–11. https://doi.org/10.1007/s00426-017-0942-y .

Lang, A., Gapenne, O., Aubert, D., & Ferrel-Chapus, C. (2013). Implicit sequence learning in a continuous pursuit-tracking task. Psychological Research Psychologische Forschung, 77 (5), 517–527. https://doi.org/10.1007/s00426-012-0460-x .

Lange, K. (2013). The ups and downs of temporal orienting: a review of auditory temporal orienting studies and a model associating the heterogeneous findings on the auditory N1 with opposite effects of attention and prediction. Frontiers in Human Neuroscience, 7 (June), 263. https://doi.org/10.3389/fnhum.2013.00263 .

Lehle, C., & Hubner, R. (2009). Strategic capacity sharing between two tasks: Evidence from tasks with the same and with different task sets. Psychological Research Psychologische Forschung, 73 (5), 707–726. https://doi.org/10.1007/s00426-008-0162-6 .

Magill, R. A. (1998). Knowledge is more than we can talk about: Implicit learning in motor skill acquisition. Research Quarterly for Exercise and Sport, 69 (2), 104–110. https://doi.org/10.1080/02701367.1998.10607676 .

McDowd, J. M. (1986). The effects of age and extended practice on divided attention performance. Journal of Gerontology, 41 (6), 764–769. https://doi.org/10.1093/geronj/41.6.764 .

McDowd, J. M., & Craik, F. I. M. (1988). Effects of aging and task difficulty on divided attention performance. Journal of Experimental Psychology: Human Perception and Performance, 14 (2), 267–280. https://doi.org/10.1037/0096-1523.14.2.267 .

McNeil, M. R., Matthews, C. T., Hula, W. D., Doyle, P. J., & Fossett, T. R. D. (2006). Effects of visual-manual tracking under dual-task conditions on auditory language comprehension and story retelling in persons with aphasia. Aphasiology, 20 (2–4), 167–174. https://doi.org/10.1080/02687030500472660 .

Meyer, D. E., & Kieras, D. E. (1997). A computational theory of executive cognitive processes and multiple-task performance: Part I. Basic mechanisms. Psychological Review, 104 (1), 3–65. https://doi.org/10.1037/0033-295X.104.1.3 .

Navon, D., & Gopher, D. (1979). On the economy of the human-processing system. Psychological Review, 86 (3), 214–255.

Nobre, A., Correa, A., & Coull, J. (2007). The hazards of time. Current Opinion in Neurobiology, 17 (4), 465–470. https://doi.org/10.1016/j.conb.2007.07.006 .

Pew, R. W. (1974). Levels of analysis in motor control. Brain Research, 71 (2–3), 393–400. https://doi.org/10.1016/0006-8993(74)90983-4 .

Plessow, F., Schade, S., Kirschbaum, C., & Fischer, R. (2012). Better not to deal with two tasks at the same time when stressed? Acute psychosocial stress reduces task shielding in dual-task performance. Cognitive, Affective, & Behavioral Neuroscience, 12 (3), 557–570. https://doi.org/10.3758/s13415-012-0098-6 .

Posner, M. I., & Petersen, S. E. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13 (1), 25–42. https://doi.org/10.1146/annurev.ne.13.030190.000325 .

Raab, M., de Oliveira, R. F., Schorer, J., & Hegele, M. (2013). Adaptation of motor control strategies to environmental cues in a pursuit-tracking task. Experimental Brain Research, 228 (2), 155–160. https://doi.org/10.1007/s00221-013-3546-9 .

Requin, J., Brener, J., & Ring, C. (1991). Preparation for action. In J. R. Jennings & M. G. H. Coles (Eds.), Handbook of cognitive psychophysiology: Central and autonomic nervous system approaches (pp. 357–448). New York: Wiley.

Rolke, B., & Hofmann, P. (2007). Temporal uncertainty degrades perceptual processing. Psychonomic Bulletin and Review, 14 (3), 522–526. https://doi.org/10.3758/BF03194101 .

Salvucci, D. D., & Taatgen, N. A. (2008). Threaded cognition: An integrated theory of concurrent multitasking. Psychological Review, 115 (1), 101–130. https://doi.org/10.1037/0033-295X.115.1.101 .

Schmidt, A. M., & Dolis, C. M. (2009). Something’s got to give: The effects of dual-goal difficulty, goal progress, and expectancies on resource allocation. Journal of Applied Psychology, 94 (3), 678–691. https://doi.org/10.1037/a0014945 .

Schmidtke, V., & Heuer, H. (1997). Task integration as a factor in secondary-task effects on sequence learning. Psychological Research Psychologische Forschung, 60 (1–2), 53–71. https://doi.org/10.1007/BF00419680 .

Scott, S. H. (2012). The computational and neural basis of voluntary motor control and planning. Trends in Cognitive Sciences, 16 (11), 541–549. https://doi.org/10.1016/j.tics.2012.09.008 .

Sigman, M., & Dehaene, S. (2006). Dynamics of the central bottleneck: Dual-task and task uncertainty. PLoS Biology, 4 (7), 1227–1238. https://doi.org/10.1371/journal.pbio.0040220 .

Strayer, D. L., & Drews, F. A. (2007). Cell-phone-induced driver distraction. Current Directions in Psychological Science, 16 (3), 128–131. https://doi.org/10.1111/j.1467-8721.2007.00489.x .

Swinnen, S. P., & Wenderoth, N. (2004). Two hands, one brain: Cognitive neuroscience of bimanual skill. Trends in Cognitive Sciences, 8 (1), 18–25. https://doi.org/10.1016/j.tics.2003.10.017 .

Töllner, T., Strobach, T., Schubert, T., & Müller, H. J. (2012). The effect of task order predictability in audio-visual dual task performance: Just a central capacity limitation? Frontiers in Integrative Neuroscience, 6 (September), 75. https://doi.org/10.3389/fnint.2012.00075 .

Tombu, M., & Jolicœur, P. (2003). A central capacity sharing model of dual-task performance. Journal of Experimental Psychology: Human Perception and Performance, 29 (1), 3–18. https://doi.org/10.1037/0096-1523.29.1.3 .

Tsang, P. S. (2006). Regarding time-sharing with convergent operations. Acta Psychologica, 121 (2), 137–175. https://doi.org/10.1016/j.actpsy.2005.07.002 .

Tsang, S. N. H., & Chan, A. H. S. (2015). Tracking and discrete dual task performance with different spatial stimulus-response mappings. Ergonomics, 58 (3), 368–382. https://doi.org/10.1080/00140139.2014.978901 .

Van Roon, D., Caeyenberghs, K., Swinnen, S. P., & Smits-Engelsman, B. C. M. (2008). Development of feedforward control in a dynamic manual tracking task. Child Development, 79 (4), 852–865. https://doi.org/10.1111/j.1467-8624.2008.01163.x .

Van Rullen, R., & Thorpe, S. J. (2001). The time course of visual processing: From early perception to decision-making. Journal of Cognitive Neuroscience, 13 (4), 454–461. https://doi.org/10.1162/08989290152001880 .

Vu, K.-P.L., & Proctor, R. W. (2002). The prevalence effect in two-dimensional stimulus-response compatibility is a function of the relative salience of the dimensions. Perception & Psychophysics, 64 (5), 815–828. https://doi.org/10.3758/BF03194748 .

Wages, N. P., Beck, T. W., Ye, X., & Carr, J. C. (2016). Examination of a neural cross-over effect using resting mechanomyographic mean frequency from the vastus lateralis muscle in different resting positions following aerobic exercise. European Journal of Applied Physiology, 116 (5), 919–929. https://doi.org/10.1007/s00421-016-3351-9 .

Wahn, B., & König, P. (2015). Audition and vision share spatial attentional resources, yet attentional load does not disrupt audiovisual integration. Frontiers in Psychology, 6, 1084. https://doi.org/10.3389/fpsyg.2015.01084 .

Wahn, B., & König, P. (2017). Is attentional resource allocation across sensory modalities task-dependent? Advances in Cognitive Psychology, 13 (1), 83–96. https://doi.org/10.5709/acp-0209-2 .

Weir, D. J., Stein, J. F., & Miall, R. C. (1989). Cues and control strategies in visually guided tracking. Journal of Motor Behavior, 21 (3), 185–204. https://doi.org/10.1080/00222895.1989.10735477 .

Wickens, C. D. (1980). The structure of attentional resources. In R. Nickerson (Ed.), Attention and performance VII (pp. 239–257). New Jersey: Lawrence Erlbaum.

Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science, 3 (2), 159–177. https://doi.org/10.1080/14639220210123806 .

Wickens, C. D. (2008). Multiple resources and mental workload. Human Factors, 50 (3), 449–455. https://doi.org/10.1518/001872008X288394 .

Wickens, C. D., Goh, J., Helleberg, J., Horrey, W. J., & Talleur, D. A. (2003). Attentional models of multitask pilot performance using advanced display technology. Human Factors, 45 (3), 360–380. https://doi.org/10.1518/hfes.45.3.360.27250 .

Wickens, C. D., Gutzwiller, R. S., & Santamaria, A. (2015). Discrete task switching in overload: A meta-analyses and a model. International Journal of Human Computer Studies, 79, 79–84. https://doi.org/10.1016/j.ijhcs.2015.01.002 .

Wickens, & Colcombe, A. . (2007). Dual-task performance consequences of imperfect alerting associated with a cockpit display of traffic information. Human Factors, 49 (5), 839–850. https://doi.org/10.1518/001872007X230217 .

Wolpert, D. M., Diedrichsen, J., & Flanagan, J. R. (2011). Principles of sensorimotor learning. Nature Reviews Neuroscience, 12 (12), 739–751. https://doi.org/10.1038/nrn3112 .

Wolpert, D. M., Doya, K., & Kawato, M. (2003). A unifying computational framework for motor control and social interaction. Philosophical Transactions of the Royal Society B: Biological Sciences, 358 (1431), 593–602. https://doi.org/10.1098/rstb.2002.1238 .

Wolpert, D. M., & Kawato, M. (1998). Multiple paired forward and inverse models for motor control. Neural Networks, 11 (7–8), 1317–1329. https://doi.org/10.1016/S0893-6080(98)00066-5 .

Wulf, G., & Schmidt, R. A. (1997). Variability of practice and implicit motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23 (4), 987–1006. https://doi.org/10.1037/0278-7393.23.4.987 .

Zhou, X., Cao, X., & Ren, X. (2009). Speed-accuracy tradeoff in trajectory-based tasks with temporal constraint. In Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics) , 5726 LNCS(PART 1) (pp. 906–919). https://doi.org/10.1007/978-3-642-03655-2_99

Download references

Acknowledgements

We thank Dietmar Sach for programming the tracking software, Bettina Kretschmann and Ismael Pedraza for help with collecting data and Anita Todd for providing language help in earlier versions.

Open Access funding enabled and organized by Projekt DEAL. This research was supported by a grant within the Priority Program SPP 1772 from the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG), Grant No. RA 940/17-1 and KU 1557/3-1.

Author information

Authors and affiliations.

Institute of Psychology, German Sport University Cologne, Am Sportpark Müngersdorf 6, 50933, Cologne, Germany

Laura Broeker & Markus Raab

Institute of Sports Science, Augsburg University, Universitätsstraße 3, 86135, Augsburg, Germany

Harald Ewolds & Stefan Künzell

School of Applied Sciences, London South Bank University, 103 Borough Road, London, SE1 0AA, UK

Rita F. de Oliveira & Markus Raab

You can also search for this author in PubMed   Google Scholar

Contributions

LB made substantial contributions to the conception, design, acquisition, analysis, interpretation, and writing the work. RO made substantial contributions to the conception, design, and draft. HE made substantial contributions to the analysis and creation of the software. SK made substantial contributions to the conception and draft; MR made substantial contributions to the conception, interpretation, and revision. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Laura Broeker .

Ethics declarations

Ethical approval and consent to participate.

All experiments with human participants performed by the authors were approved by the University’s local ethics committee and were performed according to the Declaration of Helsinki 2008.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Broeker, L., Ewolds, H., de Oliveira, R.F. et al. The impact of predictability on dual-task performance and implications for resource-sharing accounts. Cogn. Research 6 , 1 (2021). https://doi.org/10.1186/s41235-020-00267-w

Download citation

Received : 19 September 2019

Accepted : 10 December 2020

Published : 04 January 2021

DOI : https://doi.org/10.1186/s41235-020-00267-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Multitasking
• Predictability
• Task integration

• Advanced search

Advanced Search

Brain Mechanisms of Serial and Parallel Processing during Dual-Task Performance

• Find this author on Google Scholar
• Find this author on PubMed
• Search for this author on this site
• Figures & Data
• Info & Metrics

The psychological refractory period (PRP) refers to the fact that humans typically cannot perform two tasks at once. Behavioral experiments have led to the proposal that, in fact, peripheral perceptual and motor stages continue to operate in parallel, and that only a central decision stage imposes a serial bottleneck. We tested this model using neuroimaging methods combined with innovative time-sensitive analysis tools. Subjects performed a dual-task visual–auditory paradigm in which a delay of 300 ms was injected into the auditory task either within or outside of the dual-task interference period. Event-related potentials indicated that the first ∼250 ms of processing were insensitive to dual-task interference, and that the PRP was mainly reflected in a delayed global component. By a clustering analysis based on time-resolved functional magnetic resonance imaging, we identified networks with qualitatively different timing properties: sensory areas tracked the objective time of stimulus presentation, a bilateral parietoprefrontal network correlated with the PRP delay, and an extended bilateral network that included bilateral posterior parietal cortex, premotor cortex, supplementary motor area, anterior part of the insula, and cerebellum was shared by both tasks during the extent of dual-task performance. The results provide physiological evidence for the coexistence of serial and parallel processes within a cognitive task.

• cognitive architecture
• brain dynamics
• Introduction

When two targets are presented at a short interval, processing the first target delays the processing of the second, a psychological phenomenon classically termed the “psychological refractory period” (PRP). According to a prominent theory, which emerged from numerous behavioral experiments, perceptual and response operations occur in parallel, and only a central decision stage, involved in coordinating sensory and motor operations, is delayed ( Pashler, 1994 ). The aim of the present study is to analyze the neurophysiology of dual-task performance into its component stages and clearly separate its parallel and serial steps, achieving a full decomposition of the dual task.

Several previous studies have investigated the cerebral basis of processing bottlenecks. Using event-related potentials (ERPs), some components such as the N2PC, P3, and lateralized readiness potentials were found to be reduced and/or delayed during the PRP ( Osman and Moore, 1993 ; Luck, 1998 ; Arnell and Duncan, 2002 ; Arnell et al., 2004 ; Dell'acqua et al., 2005 ; Brisson and Jolicoeur, 2007 ; Sessa et al., 2007 ). Using time-resolved functional magnetic resonance imaging (fMRI) ( Kim et al., 1997 ; Menon et al., 1998 ; Formisano and Goebel, 2003 ), Dux et al. (2006) showed delayed activity in prefrontal cortex (PFC) in a PRP paradigm, implying that a frontal network was one of the fundamental nodes responsible for the central bottleneck of information processing.

Achieving a more exhaustive decomposition of the dual-task situation into its processing stages to understand their parallel or serial nature requires the following: (1) estimating timing information invariantly across different brain regions, distinguishing changes in onset latency and in duration ( Bellgowan et al., 2003 ); and (2) clustering the timing information into distinct stages based on a precise model of task sequencing. In recent work, we showed how fMRI could be used to recover the precise timing of all the stages in a complex composite task ( Sigman et al., 2007 ). Here we conducted a PRP experiment using both time-resolved fMRI and high-density ERP recordings. The timing information from both imaging techniques was clustered into components, guided by a simple psychological model of the task sequence. This allowed us to parse the execution of the two tasks into a series of processing stages with different timing properties, to understand which nodes were involved in one or both tasks or in coordinating dual-task execution, and which stages proceeded in parallel with each other or imposed a serial bottleneck.

• Materials and Methods

Participants.

Twenty-one right-handed native French speakers took part in the ERP experiment (10 women and 11 men; mean age, 24, ranging from 20 to 33), and an additional 16 in the fMRI experiment (7 women, 9 men; mean age, 23, ranging from 20 to 28). All had normal or corrected-to-normal vision. All participants provided informed written consent to take part in the experiment. This study was included into a larger neuroimaging research program headed by Denis Le Bihan and approved by the Comité Consultatif pour la Protection des Personnes dans la Recherche Biomédicale, Hôpital de Bicêtre (Le Kremlin-Bicêtre, France).

Design and procedure.

Participants were asked to perform two tasks, with the clear instruction that they had to respond accurately and as fast as possible to each of them. The stimulus types and conditions were identical for the EEG and fMRI experiments. The stimulus onset asynchrony (SOA), i.e., the delay in the onset of the two tasks, changed randomly from trial to trial among four possible values: 0, 300, 900, and 1200 ms. The logic of this design was to have to SOA values separated at 300 ms within an interference regime and within a noninterference regime in which execution of both tasks does not overlap in time. Subjects responded to both tasks with key presses, with the right hand for the number-comparison task and with the left hand for the tone task. In the number-comparison task, a number, which varied randomly among four different values (28, 37, 53, and 62), was flashed in the center of the screen for 150 ms, and subjects had to respond whether the number was larger or smaller than 45. In the auditory task, subjects had to respond whether the tone was high (880 Hz) or low (440 Hz) frequency. Stimuli were shown on a black-and-white display on a 17 inch monitor with a refresh rate of 60 Hz. Subjects sat 1 m from the screen. Stimuli were always presented in the fovea, and their size was 1°. Auditory stimuli were pure tones of 150 ms duration and 440 or 880 Hz frequency. Auditory stimulation was provided through headphones.

Subjects were trained in the task before beginning the EEG recordings. During this practice period, response times were monitored on-line, and subjects did not start the imaging session until they completed 10 consecutive trials with both response times <1000 ms. During the EEG recording, intertrial intervals (ITIs) were jittered in the range from 3 to 4.2 s (mean ITI, 3.6 s). Subjects performed a total of 960 trials (240 for each SOA value), which were divided into five identical blocks of 192 trials. Subjects had a variable rest (maximum of 5 min between blocks).

Subjects were trained in the task before beginning the fMRI. The training criterion was identical to that in the EEG recordings. Three subjects participated in both experiments. During the fMRI recording, subjects performed one single-task block and five dual-task blocks.

The single-task block comprised 180 trials. Before the beginning of each trial, the fixation cross dimmed to subjects. Number (72 trials) and pitch (72 trials) were intermixed. Thirty-six trials were blanks, in which the fixation cross dim was followed by a blank. Intertrial intervals were jittered between 2.5 and 3 s (mean, 2.75 s). The entire block lasted 440 s (∼7 min).

In the dual-task condition, subjects performed a total of 160 trials (40 of each SOA value) divided into five identical blocks. Because our phase analysis is optimal for slow event-related designs ( Sigman et al., 2007 ), intertrial intervals were 12 s. Each block lasted 384 s, ∼6 min.

ERP methods.

ERPs were sampled at 250 Hz with a 128-electrode geodesic sensor net referenced to the vertex. We rejected voltage exceeding 200 μV, transients exceeding 100 μV, or electrooculogram activity exceeding 70 μV. The remaining trials were averaged in synchrony with T1 onset, digitally transformed to an average reference, bandpass filtered (0.3–30 Hz), and corrected for baseline over a 1000 ms window during fixation before the onset of T1.

ERP component analysis.

Fmri methods..

The experiments were performed on a 3T fMRI system (Bruker). Functional images sensitive to blood oxygenation level-dependent (BOLD) contrast were obtained with a T2*-weighted gradient echoplanar imaging sequence [repetition time (TR) = 1.5 s; echo time = 40 ms; angle = 90°; field of view (FOV) = 192 × 256 mm; matrix = 64 × 64]. The whole brain was acquired in 24 slices with a slice thickness of 5 mm. High-resolution images (three-dimensional gradient echo inversion-recovery sequence, inversion time = 700 mm; FOV = 192 × 256 × 256 mm; matrix = 256 × 128 × 256; slice thickness = 1 mm) were also acquired.

fMRI statistical analysis.

Data were analyzed with SPM2. To estimate the periodicity and phase of the event-related BOLD response, the data from each subject were submitted to a first-level model in which the signal from each trial (8 TRs of 1.5 s) was fitted with three regressors: a constant, a sine, and a cosine function at the above period. To facilitate intersubject averaging across possible differences in anatomical localization, the regression weights of the sines and cosines were smoothed (7 mm full-width at half-maximum). They were then transformed with the inverse tangent function to yield, for each trial, a phase lag expressed in seconds. As in Sigman et al. (2007) , phase and amplitude were calculated as φ j = arctan( A j y / A j x ) and A j = ( A j x ) 2 + ( A j y ) 2 , where A j x and A j y are, respectively, the regression weights of the cosine and sine functions for voxel j . The phase, originally between 0 and 2π, was converted into a fraction of the stimulation period of 12 s. A phase of 0 s thus indicates a peak activation synchronous with stimulus onset. We also computed the mean phase within each subject and each condition by using a circular average procedure.

To restrict our analysis to the network of voxels engaged in the task, we used phase information and estimated the fraction of measurements of the phase that lay within the expected response range (ERR). A total of 64 mean phase measurements were obtained corresponding to four conditions, each repeated for 16 subjects. The ERR was set to the interval from 2 to 10 s, based on previous characterizations of the hemodynamic response function and allowing a margin to account for region-to-region variability and changes across conditions. The probability that x out of 64 measurements lie within the ERR can be calculated following the binomial distribution ( Sigman et al., 2007 ). We kept for analysis only voxels with >48 measurements within the ERR, corresponding to a binomial p < 0.05. The corresponding network can be seen in Figure 5 , and corresponds to 27.9% of the whole brain.

Within this mask, the significance of the variations in phase with delay were assessed with a second-level SPM model that included all the single-trial phase measurements. Four regressors of interest modeled the four possible values of delays (0, 300, 900, and 1200 ms). Eight other regressors of no interest captured variations induced by the two response times (RT1 and RT2) within each of these delay conditions. For these variables, the mean RT within a given subject and delay was subtracted from the RT observed on each trial. Finally, 16 additional variables captured the within-subject changes in phase, reflecting the fact that the delay variable was a within-subject variable.

Two statistical tests were performed. First, we looked for linearly increasing phases as a function of delay (contrast −2 −1 1 2, taking into account the irregular spacing of the delays). Second, we searched for regions with a delay by regime type interaction (contrast 1 −1 −1 1), corresponding to a PRP effect. The same SPM model was also applied to measurements of single-trial response amplitude. All results are reported at voxel p < 0.001 and cluster-level p < 0.05 corrected for multiple comparisons across the brain volume.

Behavioral results and theoretical predictions

In the main dual-task blocks, subjects performed a sequence of two tasks: first a visual task of comparing an arabic numeral (target T1) to a fixed reference, with a right-hand response (response time 1 or RT1), and second an auditory task of judging the pitch of an auditory tone (target T2) with a left-hand response (response time 2 or RT2). The SOA between T1 and T2 was varied between 0, 300, 900, and 1200 ms.

Our analysis will be guided by precise quantitative behavioral dependences between the mean of RT1, RT2, and SOA. Thus, we first set to analyze critical aspects of these dependencies and test the hypotheses of the classical PRP model ( Pashler, 1984 , 1994 ; Pashler and Johnston, 1989 ; Pashler et al., 2001 ; Sigman and Dehaene, 2005 ) as illustrated in Figure 1 .

• Download figure
• Open in new tab
• Download powerpoint

Model of the psychological refractory period and main predictions. A , Model. The vertical axis indexes processing time. The column on the left indicates the first task, and each colored box within the column represents a different stage of processing: P component (blue), C component (green), and M component (black). The series of columns on the right indicate the processing time for task 2 at different SOAs (labeled on the x -axis). For each column, the three different boxes represent the three different stages of task 2: P component (red), C component (cyan), and M component (gray). As SOA progresses, the P component starts later. All components can be performed in parallel with task 1 components, except for the C component, which imposes a bottleneck. Seriality at the C level only results in the following predictions: (1) processing of the first task is independent of SOA; (2) RT2, measured from trial onset and represented by the black line, is unchanged for small SOA (within the interference regime) but increases linearly with a slope of 1 with SOA at long T1–T2 delays (noninterference regime). B , Predicted RT1 and RT2 (from trial onset) as a function of SOA, separately within and outside the interference regime. C , Observed average RT2 and RT1 as a function of SOA. D , The model also makes predictions concerning the activation delay of processes within each task. (1) All processes of task 1 should be unaffected by SOA; (2) P processes of task 2 should increase linearly with SOA in both regimes, whereas C processes should be independent of SOA within the interference regime and increase linearly with SOA outside the interference regime.

In most PRP studies, response times are usually measured from the onset of the corresponding stimulus (T1 or T2). The PRP effect is manifested as a linear decrease (with a slope close to −1) of RT2 with SOA for short SOA values. For long SOA values, when the two tasks are independent, RT2 is independent of SOA. Here, and throughout the paper, we follow a different convention in which response times to both tasks are reported from trial onset (i.e., onset of T1). The logic for such a choice is that, throughout the paper, we will seek to understand the dependence of different cerebral responses (of fMRI voxels or of components of the EEG response) with SOA. In this context, it is helpful to relate this timing to a single common onset for task 1 and task 2, hence we locked all measures to the beginning of the trial. When RTs are measured from trial onset, as we do here, the main PRP effect results in the following: (1) invariance of RT1 with SOA for all SOA values, (2) invariance of RT2 with SOA for short SOA values (intuitively this can be understood as a queuing process, whereby the second task cannot be completed until the first task is over, thus making the response to task 2 independent of the presentation time of the corresponding target T2), and (3) linear increase of RT2 with SOA for large SOA values. Note that changing from one representation to the other simply involves subtracting SOA to RT2. For consistency, in Table 1 , we report all RT values in both notations.

• View inline

RT1 and RT2 during the fMRI and ERP experiments

The experimental design was chosen so as to have two different SOA values within the interference regime (SOA = 0 or 300 ms), in which execution of both tasks overlaps in time, and two SOA values in the noninterference regime (SOA = 900 or 1200 ms), in which the onset of the second stimulus comes after the first task has been completed. The logic of the experimental design is to understand the effect of injecting a 300 ms delay in task 2 either within the interference or within the noninterference regime. Theoretical models of dual-task interference ( Pashler, 1984 ; Pashler and Johnston, 1998 ) based on behavioral results predict that whereas certain processes of a task can be performed in parallel with concurrent processes of another task, other processes can be only executed serially and thus establish a processing bottleneck ( Fig. 1 a ). The model predicts that RT1 should be unaffected by SOA manipulations. On the other hand, manipulations of SOA affect RT2 distinctively depending on whether this manipulation is done within the interference regime or not. Within the noninterference regime, increasing SOA should result in a direct proportional increase of RT2 (which by convention is taken from trial onset, i.e., the onset of the first stimulus T1). On the contrary, within the interference regime, increasing SOA should result in little or no change of RT2, because this response is no longer locked to T2 onset, but rather to the completion of the serial process of task 1 ( Fig. 1 b ).

The response times measured during ERP recordings followed this prediction ( Fig. 1 c , Table 1 ), thus replicating our previous results ( Sigman and Dehaene, 2005 ). To quantify this observation, we performed an ANOVA with two factors: regime type (interference or noninterference) and delay manipulation (short or long SOA within each regime) ( Table 2 ). As predicted by the model, we did not find any effect of SOA manipulations on RT1. Furthermore, we found a significant effect of both factors and of their interaction on RT2. Indeed, within the interference regime, there was only a moderate increase of RT2 with the 300 ms delay manipulation (RT2 SOA=0 = 990 ± 20 and RT2 SOA=300 = 1042 ± 19). This difference (54 ms) is small compared with the change observed in the noninterference regime (RT2 SOA=900 = 1420 ± 14 and RT2 SOA=1200 = 1680 ± 12), thus indicating the presence of a dual-task bottleneck at short but not long SOA. The fact that there was a residual nonzero difference merely indicates that for some trials in which the first task was responded fast, the second task may not have suffered from a full 300 ms bottleneck delay.

Statistical effects of SOA manipulations on response times and ERP component delays

In dual-task experiments, response grouping is sometimes reported. Grouping may indicate that the dual task is treated as a single compound stimulus S1 + S2 with a corresponding compound response R1 + R2 ( Welford, 1967 ). Alternatively, the first response may be deferred until the second response has been selected, so that the two responses are emitted simultaneously or in close succession ( Borger, 1963 ). Grouping would be problematic for present purposes, because it would imply that all of the processing stages of both tasks are performed before the execution of the first response, thus yielding predictions qualitatively distinct from what is expected in the sequential bottleneck model. It is therefore important to verify that subjects were not grouping responses in the present experiment. To this aim, we conducted two analyses on the response time data. First, experiments in which responses were voluntarily grouped have observed a very short delay between responses, typically <100 ms ( Pashler and Johnston, 1989 ; De Jong, 1993 ). In contrast, in our experiment, both in the fMRI and in the EEG data, <0.7% of the trials show a difference of 100 ms or less. Second, in grouped trials, the response to the first task is not executed until the second task is performed, which implies that RT1 should increase monotonically with SOA. In our data, we do not observe this behavior as indicated in Table 1 , in which no effect of SOA is observed on RT1.

Beyond the predictions for response time data, our simple theoretical model makes predictions concerning the dynamics of the different processes, depending on whether they can be carried in parallel with concurrent processes of another task or rather establish a serial bottleneck. The predictions are summarized in Figure 1 d . Here we draw attention to the main points. First, SOA should have no effect on any process of task 1. Second, the SOA manipulation is expected to separate, within task 2, two different types of cerebral processes. Processes locked to the onset of the second stimulus (parallel) should exhibit additive effects of regime and delay, but no interaction should be found. Processes that track the dual-task bottleneck and its consequences should reflect an effect of regime and of delay as well as, crucially, an interaction between these two factors. In what follows, to test these predictions, we will decompose the ERP data into distinct response components and study their latency as a function of SOA.

Decomposing the ERP data

To understand the dynamics of different brain processes involved in the dual-task condition, we first decomposed the ERP data using scalp templates identified from the ERP recorded at the largest SOA (SOA = 1200), in which the execution of both tasks does not overlap in time. To identify the main ERP components involved in each task at the group level, we integrated the absolute value of the voltages recorded over all electrodes from the grouped data at SOA = 1200. This simple measure resulted in two clearly distinguishable peaks ( Fig. 2 a ) after the presentation of each stimulus. The topographic distribution at the time of each maximum could easily be identified as the N1 and P3 components corresponding to each task. The peak latencies of the selected components were as follows: visual N1, 180 ms; visual P3, 340 ms; auditory N1, 110 ms; and auditory P3, 370 ms. Although more components could potentially have been identified by a more sophisticated analysis, our aim was not to identify all the independent processes within a task, but rather to understand the dynamics of these basic response components within the interference and noninterference regimes.

Decomposing the ERP data into four scalp templates. A , Integrated absolute value of voltages recorded over all electrodes from the grouped ERP data at the SOA = 1200 condition. After the presentation of the first (visual) stimulus (red bar) and the second (auditory) stimulus (blue bar), two peaks are clearly noticeable. At the latency of each peak, we estimated the voltage topography, which corresponds to well known N1 and P3 components. The P3 component of task 1 is lateralized to the right, and the P3 component of task 2 is lateralized to the left. This may result from motor preparation or setting of the stimulus response, because responses to the first task were made with the right hand and responses to the second task with the left hand. B , A histogram, across subjects, of the time at which the projection of the data from SOA = 1200 onto each component reached its maximum. The timing of N1 components is significantly more reliable across subjects than the timing of the P3 components. Colors correspond to a rainbow color scale between −4 and 4 μV.

To obtain response templates for each individual subject, and therefore estimate the components and their latencies at the individual level, we used multiple linear regression to project, for each individual subject and each time point, the ERP at SOA = 1200 (in the noninterference regime) to the four previously identified group scalp templates. We observed that on a subject-by-subject basis, the maximum of this projection did not coincide precisely with the timing of each group component. Thus, within a window of 400 ms centered in the timing of the group template, we searched for the maximum of this projection. For both tasks, N1 components were considerably more reliable in time than P3 components ( Fig. 2 b ). The scalp template of each individual subject was then estimated as the ERP map at the time point at which the projection to the group template was maximal.

Understanding the temporal dispersion of different components across subjects is informative, and, moreover, it allows a more appropriate definition of individual ERP components. However, it is important to note that no significant departures from the results reported here were observed when individual templates were defined at the same time point, based solely on the group timing. Indeed, very similar results were obtained even when the group templates were used as templates for the regressions of individual subjects.

Analyzing the latencies of the response components

To understand the dynamics of each scalp component as a function of the different SOA values, we used linear regression to decompose each ERP, at each SOA and each time point, into a linear combination of the four scalp templates. Thus, for each subject, we obtained a time course of the projection of the data onto each of the four ERP components (for simplicity, we refer to this simply as the “time course of the ERP component”). It is important to emphasize that the same four templates were used for all SOA values: this allowed us to understand how the dynamics of the corresponding brain processes changed in the different regimes. We observed that the time course of the components ( Fig. 3 ) fitted with predictions derived from our sequential model, if one supposes that the N1 components map onto perceptual processes and the P3 components onto central processes. The temporal course of the components of the first task (T1_N1 and T1_P3) was utterly unaffected by changes in SOA values ( Fig. 3 , first and second rows). This observation testifies to the efficiency of the decomposition procedure, which was able to identify the visual components of the T1 task even when they were superimposed with simultaneously occurring auditory components.

Time course of the four ERP components. Each row corresponds to the regression of the data onto one of the four ERP scalp templates (respectively, the N1 and P3 of task 1, then of task 2). Within each row, the different colors indicate the time course for a given SOA value. Components of the first task (first two rows) are unaffected by SOA. The N1 component of task 2 has a maximum at a fixed latency after target T2, both within and outside the interference regime. The P3 component of task 2 shows a minor effect of SOA within the interference regime and a shift proportional to the SOA change in the independent regime. Colors correspond to a rainbow color scale between −4 and 4 μV.

On the contrary, the temporal course of the components of the second task showed a very distinct pattern. The temporal course of the T2_N1 component was strictly time locked to T2 onset, as expected for a perceptual component of task 2. It had a maximum at a fixed latency after T2 presentation, both within and outside the interference regime, and thus increased linearly with SOA. The temporal course of the T2_P3 component of task 2, on the other hand, showed a minor effect of SOA within the interference regime and a shift proportional to the change in SOA in the noninterference regime, as expected for a central component of task 2.

To quantify these observations, we used the individual response components ( Fig. 4 a ). For each individual subject and SOA condition, we estimated the delay in the onset of a component as the time at which the projection of the data onto this component reached its maximum (within an interval of 1000 ms after stimulus presentation). We then estimated the mean for each component and condition by averaging the results across subjects ( Fig. 4 c ). The obtained results showed the pattern expected by the sequential model ( Fig. 4 b ). To test the significance of this observation, these values were submitted to the same ANOVA as response times, again with factors of regime type (interference or noninterference) and delay manipulation (short or long SOA within each regime) ( Table 2 ). As predicted, no effects were found for the components of task 1. Also, as predicted, we observed a main effect on both components of task 2 but an interaction only for the T2_P3 component.

Analyzing the latencies of the ERP components. A , Each of the four panels shows the time course of individual subjects' ERP components (first row, task 1; second row, task 2; first column, N1; second column, P3). Within each panel, the data are divided into four images corresponding to different SOA values. Within each image, each line corresponds to the time course for an individual subject, with amplitude coded by color. In the top panels, it can be seen that the projection is not sensitive to SOA, whereas in the bottom panels, it changes with SOA. Colors correspond to a rainbow color scale between −4 and 4. B , C , Predicted and observed mean latency of each component (averaged across subjects) as a function of SOA, within and outside the interference regime. As predicted by the model, task 1 components are independent of SOA. The N1 component of task 2 increases with SOA within both regimes, as predicted by a P component. The P3 component increases with SOA in the independent regime but not in the interference regime as predicted by a C component.

Summary of inferences from ERP recordings

Overall, the ERP results appear highly compatible with the proposed parallel–serial model. The fact that the T1 task components are entirely unchanged by the simultaneous presentation of a second auditory target T2 at various SOAs provides strong support for the serial hypothesis that subjects concentrate entirely on performing task 1 first. Contrary to response times, which only index task completion, ERPs track the complete time course of task 1 processing and suggest that task 1 is entirely unaffected by whether a second target is or is not waiting to be processed. This observation conflicts with an alternative model that postulates central capacity sharing during dual-task performance ( Tombu and Jolicoeur, 2005 ). At least in our specific experimental set up, with a fixed task order and with instructions to respond as fast and as accurate as possible to both tasks, we found no evidence for central time sharing between the T1 and T2 tasks.

As for task 2, we found that a perceptual stage, indexed by the N1, unfolded immediately after T2 presentation (and thus in parallel with the ongoing task 1), whereas a later stage, indexed by the P3, was systematically delayed in tight parallel to the dual-task processing delay inferred from response times ( Fig. 4 c ). The data suggest that the perceptual component unfolds as a series of damped oscillations over a period of ∼300 ms after T2 onset ( Fig. 3 , third row), whereas the central component starts ∼250 ms after T2 onset and peaks at 380 ms. This temporal decomposition fits well with a previous ERP-based decomposition of processing in related attentional-blink and masking experiments ( Sergent et al., 2005 ; Del Cul et al., 2007 ), in which the first ∼270 ms were attributed to perceptual processing and were followed by conscious access to a central distributed workspace involving prefrontal cortex as a central node.

Some details of the ERP analysis do suggest departure from the simple model proposed. Crucially, the N1 component of task 2 does not appear strictly invariant as a function of SOA. First, some amplitude attenuation is visible in Figure 3 , especially at the SOA of 300 ms (when T2 is presented while the subject is most fully engaged in task 1 processing). To test this quantitatively, we measured the amplitude of the peak of the N1 component of task 2 for the different SOA values, for each individual subject and then averaged across subjects. The peak amplitudes for the SOA values of 0, 300, 900, and 1200 ms were respectively 0.87 ± 0.05, 0.82 ± 0.07, 1.01 ± 0.08 μV, and 1.11 ± 0.07 μV. An ANOVA on peak amplitude showed a significant effect of regime type ( F = 66.1; p < 0.001; df = 1) and of delay manipulation ( F = 16.0; p < 0.001; df = 1), with no significant interaction. Second, for the largest SOA values, the temporal course of the N1 component ramps before stimulus presentation, probably reflecting task expectation and preparation ( Fig. 3 , third panel, especially for SOA = 1200 ms). Both the shifting baseline and increased auditory N1 for late SOA suggest that, contrary to the simple model, perceptual auditory processing was slightly modulated by task 1 engagement. This modulation probably reflected a change in attention (and the well known fact that attentional engagement in a visual task can lead to reduced auditory processing). Once subjects completed the visual task 1, which always came first, they were able to deploy auditory attention more fully, thus explaining the enhanced auditory N1 at long SOAs.

Indeed, the auditory P3 component provided direct evidence for a process of auditory anticipation. As seen in Figure 3 (fourth panel), an auditory P3 component emerged at long SOAs even before any auditory stimulus was presented. This anticipatory component peaked at ∼500 ms, thus coinciding nicely with the end of the visual P3 evoked by task 1. This ERP sequence is compatible with the hypothesis that as soon as they completed task 1, subjects reoriented their attention to prepare for task 2, thus enabling them to respond faster to T2 at long SOAs ( Fig. 4 c , lag between RT2 and T2_P3). That task 2 preparation contributes to the PRP phenomenon has often been postulated in previous behavioral work. For instance, it can account for the fact that task 2 is often completed faster than task 1 at long SOAs ( Logan and Gordon, 2001 ), or that a PRP cost is found even on trials in which RT1 is shorter than the SOA ( Jentzsch et al., 2007 ). The time course of the P3 component of task T2 clearly indicates the presence of an executive component of task 2 engaging, even before target 2 presentation, in agreement with a considerable amount of previous behavioral work ( De Jong, 1993 , 1995 ; Allport et al., 1994 ; Meiran et al., 2000 ; Logan and Gordon, 2001 ; Ruthruff et al., 2001 ; Sigman and Dehaene, 2006 ; Jentzsch et al., 2007 ).

fMRI experiment

To further our understanding of dual-task interference, it is essential to understand the cerebral underpinning of the central P3 component observed at the scalp level. Unfortunately, ERPs provide high temporal resolution, but they are notoriously imprecise for localization. Although distributed dipole models can be used to reconstruct an estimated time course of activation at each cortical location ( Sergent et al., 2005 ; Del Cul et al., 2007 ), this reconstruction is only approximate. Here, we took advantage of the fact that the PRP phenomenon induces large delays of several hundred milliseconds, which are measurable with fMRI. We recently described a Fourier-based method that results in a temporal resolution of ∼100–200 ms with whole-brain fMRI ( Sigman et al., 2007 ). The method is based on (1) a slow event-related design with long intertrial intervals, permitting measurement of the entire rise and fall of the hemodynamic response on each trial; (2) fitting of this response with sine and cosine functions, allowing estimation of its phase and amplitude; and (3) examination of how the phase and amplitude vary as a function of experimental parameters (here the injected SOA delay between T1 and T2). For a change in the onset of neural activation, only the phase of the hemodynamic response should vary, not the amplitude. For a change in duration of activation, both phase and amplitude should increase, with the slope of the phase change reflecting one-half of the actual change in the duration of neuronal activation ( Sigman et al., 2007 ).

We therefore recorded whole-brain fMRI images (TR = 1.5 s) while subjects performed, in the main blocks, 160 trials of the dual-task paradigm, spaced by 12 s, with the same four levels of SOA as above (0, 300, 900, or 1200 ms). The standard PRP model makes simple predictions about the impact of this delay on activation. As with the ERPs, these predictions will determine the analytic strategy to understand the fMRI data. For regions involved exclusively in task 1, activation should be identical and not delayed, hence the phase should be constant. For regions involved in the perceptual component of task 2, activation should be linearly delayed, and hence the phase should increase in direct proportion to the SOA. Finally, for regions involved in the central and motor components of task 2, activation should be delayed by the PRP bottleneck. Hence, a nonlinear phase change should be seen: the phase should be constant in the interference regime, but affected by SOA in the noninterference, thus resulting in an interaction of delay and regime type. For all these regions, if the effect of SOA is simply to alter the onset time of distinct processes, the amplitude of the fMRI activation should remain constant.

To test these predictions, we computed the phase and amplitude of the hemodynamic response on each trial, each subject, and each voxel, and submitted the resulting images to an ANOVA with delay as the main within-subject factor (see Materials and Methods). As shown in Figure 5 , a large network of brain areas exhibited phases consistently falling within the expected response latency for a task-induced activation (a liberal interval of 2–10 s). As expected for a complex dual-task experiment with visual and auditory stimuli, these regions included bilateral visual occipitotemporal cortices, bilateral superior temporal auditory cortices, motor, premotor, and cerebellar cortices, and a large-scale bilateral parietofrontal network ( Fig. 5 ).

Active network during dual-task performance. A , Fraction of measurements (a total of 64, corresponding to 16 subjects × 4 conditions) in which the mean phase value lies within the expected response range (set to a liberal interval of 2–10 s). Brain regions that show phases consistently within the expected regions (active regions) are shown in red, and brain regions whose phase is consistently out of the expected range (inactive regions) are shown in blue. As expected for a complex dual-task experiment with visual and auditory stimuli, active regions included bilateral visual occipitotemporal cortices, bilateral superior temporal auditory cortices, motor, premotor, and cerebellar cortices, and a large-scale bilateral parietofrontal network. Inactive regions involve a network that has been systematically shown to inactivate during task execution ( Raichle et al., 2001 ). B , Only a subset of the active network showed a significant increase in phase with SOA. C , A bilateral frontoparietal network showed a significant nonlinear effect (i.e., the effect of injecting a delay on the phase is distinct during the interference and noninterference regimes). L, Left; R, right.

Linear contrast for phase increase

We then examined which brain areas showed a significant increase in phase with SOA, using a linear contrast. Only a subset of the active network showed a significant phase increase ( Fig. 5 ). In particular, although the bilateral lateral occipitotemporal cortices, expected to contribute exclusively to task 1 (visual comparison), were highly active, their phase was essentially unaffected by delay ( Fig. 6 ). As could be predicted from the fact that task 2 involved an auditory decision, the most significant phase increase was seen in a large extent of bilateral auditory cortices, extending from Heschl's gyrus to the lateral superior temporal gyrus and surrounding lower bank of the supramarginal gyrus (peak Montreal Neurological Institute coordinates, −52, −28, 4; t = 11.0; and 54, −18, 6; t = 10.6). In these auditory areas, the extracted phase varied almost strictly linearly with the injected delay, and with a slope not significantly different from 1 ( Fig. 6 ). The linear effect was also significant in right midline precentral cortex at the level of the supplementary motor area (SMA; 10, −10, 59; t = 9.07), right motor cortex (26, −30, 54; t = 8.76), right central gyrus (44, −20, 44; t = 8.68), and left cerebellum (−20, −56, −20; t = 7.12) corresponding to the left hand used to respond to task 2.

Evidence for delayed activation in fMRI. A , Axial slices showing some of the brain areas in which the phase of activation increased with SOA. Only voxels with a highly significant linear contrast (voxel p < 0.05, familywise-error correction for multiple comparisons across the brain volume) are shown here for visualization purposes. Color encodes the value of the slope relating phase to SOA (a slope close to 1 is expected for regions in which activation is delayed in direct proportion to T2 presentation). B , Insets show the measured phase for each of the four SOAs (0, 300, 900, and 1200) and the 90% confidence interval as estimated by SPM2. For reference, two dotted lines are provided: slope of 0 (no variation in phase) and slope of 1 (pure delay). Phase dependencies with SOA are highly bilaterally symmetric and show very distinct profiles in active regions. For instance, the auditory cortex (middle row insets, z = 6 axial slice) shows a slope value very close to 1, whereas the extrastriate visual cortex (lateral occipital), although highly activated, has a phase value insensitive to SOA manipulations. L, Left; R, right.

Note that an effect of SOA on the phase was also seen in the left motor cortex (−60, −20, 46; t = 7.19) and right cerebellum (30, −58, −28; t = 6.51), perhaps corresponding to partial bilateral motor control. More surprisingly, phase also increased with delay in bilateral posterior lingual gyrus and cuneus (peaks at 10, −76, 0; t = 7.50; 0, −82, 18; t = 7.43). Additional smaller peaks were seen in bilateral thalami (8, −28, −2; t = 7.03; −14, −26, −2; t = 6.80), bilateral dorsolateral PFC (−42, 40, 28; t = 4.56; 48, 38, 26; t = 4.42), and bilateral inferior temporal cortex (−52, −64, 0; t = 4.41).

In sharp contrast with this major effect of delay on the measured phases, not a single brain region showed a significant linear increase in amplitude of the BOLD signal with delay, even when the threshold was lowered to a liberal voxelwise p < 0.01, uncorrected. This finding is important, because it indicates that dual-task interference corresponds to a pure delay effect: the lengthening of RT2 for short T1–T2 delays occurs without a change in the duration or the amplitude of the neural activation anywhere in the brain. This result, which replicates previous findings ( Jiang et al., 2004 ), argues in favor of the present queuing model ( Pashler, 1994 ; Sigman and Dehaene, 2005 ) and against models that attribute dual-task interference to a deployment of additional effort, resources, or central executive monitoring ( Meyer and Kieras, 1997 ).

Nonlinear profiles of phase

As seen in Figure 6 , within the areas selected for their significant increase in phase with delay, distinct temporal patterns were in fact observed. In auditory cortex, a purely linear variation was observed, whereas in right motor cortex, there seemed to be no increase of phase within the interference regime, reflecting the PRP effect.

To identify regions reflecting a phase dependence characteristic of the PRP effect, we searched for regions with a significant interaction between delay and regime type, as observed in RTs and in the P3 component of event-related potentials. The overall network of regions showing a significant nonlinear effect is shown in Figure 5 c . A bilateral parietofrontal network was seen, with its largest peak in right intraparietal and superior parietal cortex (36, −44, 40; t = 4.55; and 40, −48, 54; t = 4.42) and smaller clusters in left parietal cortex (−42, −48, 56; t = 4.44; −60, −38, 46; t = 4.09) and left dorsolateral PFC (−40, 44, 16; t = 4.38) ( Fig. 7 ). A symmetrical right dorsolateral PFC cluster showed a strong effect (40, 32, 32; t = 4.17), but its extent of 22 voxels was too small to reach significance ( Fig. 7 ). Finally, the right precentral gyrus also showed a nonlinear phase (34, −2, 66; t = 4.03). Again, all of these temporal effects occurred without any corresponding nonlinear change in BOLD signal amplitude, even at a relaxed p < 0.01.

Nonlinear temporal delays reflecting dual-task interference in fMRI. Axial slices show the regions in which a significant interaction between delay and interference regime was found, indicating a PRP interference pattern. This frontoparietal network includes the superior parietal cortex [peaks at (36, −44, 40) and (40, −48, 54)] and left and right dorsolateral PFC [peaks at (−40, 44, 16) (left) and a small cluster at (40, 32, 32) (right)]. L, Left; R, right.

Parsing of brain networks by their phase profile

The previous analysis separated three types of brain areas depending on their lack of variation, linear variation, or nonlinear profile of phase as a function of SOA. We next show that a quantitative analysis of the linear and nonlinear dependence of the phase can serve to identify regions involved in different processing stages of the dual task and thus to parse the active network into different processing stages according to their timing characteristics. Such a quantitative analysis is possible because the phase extracted from our fMRI analysis is a quantitative parameter, expressed in seconds, and the serial bottleneck model makes quantitative predictions as to how it should vary with SOA for different types of brain regions.

The simplest case is for regions devoted exclusively to the first task. Here, phase should not change with SOA. Thus, these regions should show neither a linear nor a nonlinear increase of phase with SOA. On a plane defined by the linear contrast on the x -axis, and the nonlinear contrast on the y -axis, such regions should fall close to the origin. Second, similarly, the phase of regions corresponding to parallel stages of task 2 should show a purely linear increase of phase with SOA, without any nonlinear dependence. Third, regions engaged solely in the bottleneck of task 2 (as well as postbottleneck task 2 processing) should show both a linear and a nonlinear effect, corresponding to the theoretical dependence exhibited by RT2 and described in Figure 1 .

In addition to these three phase profiles, we also expect more complicated patterns. Regions involved in both tasks should show an increase of phase corresponding approximately to one-half of those indicated by regions corresponding to task 2 (assuming that task 1 and task 2 cause approximately similar amplitudes of activation). Because the two tasks involved distinct modalities of input and output, we did not expect any regions to be shared at the perceptual or response level, but we did expect that areas responsible for the dual-task bottleneck might be shared between the two tasks, in which case they would show significant linear and nonlinear contrasts, but with a quantitative variation in phase approximately halved compared with the variation observed in RT2. Finally, because the ERP data had indicated some departures from the simple bottleneck model, here we investigated whether any regions showed a purely nonlinear component, i.e., a full crossover interaction with an increase in phase at the shortest SOA, as might be expected if this SOA requires a specific engagement of task-coordination processes.

Following this logic, we parsed the active network in regions with different timing characteristics by projecting all active voxels onto a two-dimensional plane defined by the value of the linear ( x -axis) and nonlinear ( y -axis) contrasts for phase as a function of SOA ( Fig. 8 ). The contrasts were scaled to permit quantitative predictions. For the linear contrast, a value of 0 indicated no variation in phase with delay, and a value of 1 indicated a 1:1 linear relation between injected delay and observed phase. For the interaction contrast, a value of 0 indicated no interaction (linear relation between phase and delay), a value of 1 indicated an interaction quantitatively equal to expectations from the PRP model for a central T2 process (no increase of phase within the interference regime, linear increase in phase outside the interference regime), and a value of 2 indicated a full crossover interaction. We then performed a Voronoi tessellation, determined by the distances to the five canonical points described previously: (1) no linear nor nonlinear response (blue), (2) purely linear response (yellow), (3) linear and nonlinear response with slope of 1 (red), (4) linear and nonlinear response with slope of ½ (cyan), and (5) purely nonlinear response with slope of 2 (green). This analysis defined five voxel types according to their timing properties, and their cerebral distribution was examined by color coding ( Fig. 8 ). For simplicity, we only considered the resulting spatial clusters that exceeded 200 voxels.

Clustering of brain regions by their temporal properties. To parse the active network in regions with different timing characteristics, all active voxels were projected onto a two-dimensional plane defined by the value of the linear ( x -axis) and nonlinear ( y -axis) contrasts for phase as a function of SOA and classified according to a Voronoi tessellation, determined by the distances to the five canonical points determined by the PRP model (see Results for details) in five categories: (1) no linear nor nonlinear response (blue) (expected profile of regions involved in execution of task 1); (2) purely linear response (yellow) (expected profile of regions of task 2 that can be active simultaneously during the execution of task 1); (3) linear and nonlinear response with slope of 1 (red) (expected profile of regions of task 2 that reflect a serial bottleneck); (4) linear and nonlinear response with slope of ½ (cyan) (expected profile of regions shared by both tasks); and (5) purely nonlinear response with slope of 2 (green) (expected profile of regions that show a delay during simultaneous presentation of auditory and visual stimuli). Insets, The anatomical projection as well as a representative phase profile of each cluster.

The functional neuroanatomy of the five parsed networks ( Fig. 8 ) was, for the most part, in tight accordance with the theoretical predictions. The first cluster (blue, no phase variation) comprised regions in extrastriate visual cortex, left motor cortex, and the most medial part of the posterior parietal cortex, as well as an extended subcortical network. This network plausibly corresponds to T1 task processing (visual number comparison with a right-hand response). The second cluster (yellow, slope 1 linear phase response) involved only bilateral auditory cortex, including Heschl's gyrus and more lateral regions of temporal cortex, a plausible network for the perceptual stages of the T2 task of auditory pitch judgment. The third cluster (red, slope 1 nonlinearity corresponding to T2-only bottleneck and postbottleneck areas) included the right motor cortex, right SMA (remember that target T2 is responded to with the left hand), and bilateral intraparietal activation. Interestingly, this cluster also included the most medial parts of the visual cortex. This unexpected finding might relate to the fact that subjects resumed attention to the fixation cross after conclusion of the two tasks.

The fourth and most theoretically relevant cluster (cyan; slope ½ nonlinearity corresponding to bottleneck areas shared by T1 and T2) involved an extended bilateral network that included the bilateral posterior parietal cortex, premotor cortex, SMA, anterior part of the insula, and the cerebellum. This cluster corresponds to the center of the Voronoi diagram, and hence there might be a bias for noisy and less reliable responses to be mapped by default to this cluster. Thus, we cannot exclude the possibility that some of these voxels might belong to other stages. However, most of this network showed precise phase dependence with a slope of ½, suggesting that a large amount of the dual-task network was shared by both tasks during dual-task performance.

Finally, the fifth cluster (green, purely nonlinear phase variation) involved exclusively a bilateral frontoparietal network. This network has been previously involved as responsible for processing bottleneck in dual-task performance ( Marois et al., 2000 ; Marois and Ivanoff, 2005 ; Dux et al., 2006 ), is engaged in effortful but not in automatic tasks ( Ashbridge et al., 1997 ), and is ubiquitously present in a large variety of goal-directed tasks ( Duncan and Owen, 2000 ). Although it is therefore not surprising to find these regions associated with the purest form of PRP interaction, it is not entirely clear why they should exhibit a slower response at SOA = 0 than at SOA = 300 ms, especially because such a pattern is not seen in response times. One possibility is that the SOA = 0 is special because it is the only condition in which the stimuli are not ordered. Thus, it may involve the deployment of additional higher-level control, both during and after T1 and T2 processing, to impose the appropriate task order ( Sigman and Dehaene, 2006 ).

Relation to single-task processing

Although our analysis identified subtle timing differences in dual-task execution, there are certain specific situations that it cannot disambiguate. Specifically, we cannot distinguish between timing patterns that yield the same dependence of the temporal center of mass of neural activation as a function of SOA. For instance the phase dependency of voxels in the fourth cluster (slope of ½, which we identified as voxels participating in both tasks) could also be found in a region with a phasic response at the end of task 1 (for instance, involved in task disengagement) and another phasic response at the beginning of task 2 (for instance, for task engagement). To resolve this ambiguity, we explored the relation between the dynamics in the dual-task experiment and the activations in single-task execution (of T1 and of T2). We asked whether the voxels identified by fMRI timing analysis as belonging to both tasks were indeed active during either task alone, or whether some voxels were solely active in relation to the requirements of dual-task execution. To this aim, we investigated the overlap of the five clusters identified by the fMRI timing analysis of the activation observed when subjects were engaged in only one task at a time, obtained in an independent fMRI run, in which the same subjects performed the number and sound tasks on separate trials. Number and sound trials alternated randomly in a fast event-related design. We also intermixed blank trials in which the fixation cross dimmed, to account for possible effects of the dimming of the fixation cross as well as of task-setting mechanisms.

Figure 9 A shows the regions differentially involved in the number and sound tasks (positive t values, number > sound; negative t values, sound > number). These were highly consistent with previous findings. Activation in motor cortex as well as in SMA was contralateral to the response hand, whereas activation in the cerebellum was ipsilateral. We observed a massive activation in superior temporal cortex, including primary auditory cortex, for the sound relative to the number task. In the converse direction, lateral occipital/fusiform cortex was more active in the number task, although no significant difference was seen in medial occipital cortex (area 17/18). This finding is consistent with the observation that the lateral occipital, but not the medial occipital cortex, showed phase invariance characteristic of exclusive task 1 performance.

Comparison of dual-task and single-task responses. A , Regions differentially involved in the number and sound tasks (positive t values, number > sound; negative t values, sound > number). Activation in motor cortex as well as in SMA was contralateral to the response hand, whereas activation in the cerebellum was ipsilateral. A massive activation is observed in superior temporal cortex, including primary auditory cortex, for the sound relative to the number task. In the converse direction, lateral occipital/fusiform cortex was more active in the number task, although no significant difference was seen in medial occipital cortex (area 17/18). B , To investigate the relation of each of the five clusters obtained in the dual-task analysis to these single-task activations, the voxels from each dual-task cluster were projected onto a scatter-plot plane in which the x -coordinate represents the single-task activation to the sound task (T2) and the y -coordinate represents the single-task activation to the number task (T1). C , Each scatter plot was collapsed to a histogram, counting the fraction of voxels within each individual cluster as a function of the t value of the number–sound contrast on the single-task runs. The vertical line indicates the mean of each distribution. The first cluster type (blue, no phase variation) had a major overlap with the T1 task (number task), including a subset of voxels strongly activated in the number task but virtually inactive in the sound task. The second cluster type (yellow, purely linear) involved exclusively regions strongly activated by the sound task but inactive during the number task. The third cluster type (red, linear and nonlinear variation corresponding to T2-only bottleneck and postbottleneck areas) also involved voxels with dominant activation for the T2 task (sound task). The fourth (cyan, slope ½ nonlinear corresponding to shared bottleneck areas) and fifth (green, purely nonlinear PRP effect) cluster types showed positive and positively correlated activations in both tasks.

We then investigated quantitatively the relation of each of the five clusters obtained in the dual-task analysis to single-task activations. We emphasize that the dual-task clusters were identified purely on the basis of their temporal profile: all dual-task activations occurred in response to a mixture of visual and auditory stimuli separated by 1.2 s at most. The PRP model predicts that voxels with different temporal properties during dual-task processing should be involved in different stages of the individual tasks. We explored this hypothesis by projecting the voxels from each dual-task cluster onto a new scatter-plot plane in which the x -coordinate represents the single-task activation to the sound task (T2) and the y -coordinate represents the single-task activation to the number task (T1) ( Fig. 9 B ). We also collapsed these data in histograms, in which we counted the fraction of voxels within each individual cluster as a function of the t value of the number–sound contrast on the single-task runs ( Fig. 9 C ).

As expected from the PRP model, we observed that the first cluster type (blue, no phase variation) had a major overlap with the T1 task (number task). The scatter plot, in particular, comprised a subset of voxels strongly activated in the number task but virtually inactive in the sound task. The second cluster type (yellow, purely linear) involved exclusively regions strongly activated by the sound task, but inactive during the number task. The third cluster type (red, linear and nonlinear variation corresponding to T2-only bottleneck and postbottleneck areas) also involved voxels with dominant activation for the T2 task (sound task), yet often with a positive activation for the T1 task too. Finally, the fourth (cyan, slope ½ nonlinear corresponding to shared bottleneck areas) and fifth (green, purely nonlinear PRP effect) cluster types showed positive and positively correlated activations in both tasks.

In summary, the cyan and green voxels in Figure 8 fulfill two independent criteria: they belong to the intersection of activations observed during the processing of either task in isolation, and they show a nonlinear phase relation characteristic of the PRP during dual-task performance. These areas, which form an extended parietoprefrontal network, are thus highly likely to play a causal role in dual-task interference.

Although an overall correspondence was found between the single-task and dual-task results, close examination of Figure 9 reveals several instances of deviation from the expected pattern (for instance, cyan voxels showing an interaction profile of phases during dual-task processing, yet no activity during either single task). Further experiments would be needed to understand these deviations, which could arise from a number of differences between the single- and dual-task conditions (e.g., random vs predictable order of the two targets).

We used time-resolved EEG and fMRI to probe the cerebral mechanisms of dual-task interference. Both methods converged to support a simple model that accounts for the major part of our observations: a central decision-related processing stage establishes a strictly serial bottleneck, whereas early perceptual processes occur in parallel as soon as a stimulus is delivered. Some specific observations, however, revealed important departures from the model and implied the involvement of additional executive components for task engagement and coordination.

Processing of T1 was unaffected by the joint presentation of T2 at various time intervals. fMRI also indicated that the total level of brain activation, whether evoked by T1 or T2, was unaffected by SOA. Thus, dual-task interference merely affected the temporal organization of brain activation. In both fMRI and ERPs, auditory activation evoked by T2 was time locked to the onset of auditory stimulation, even when T2 was presented during the T1 task. The fine temporal resolution of ERPs suggested that this perceptual processing stage lasted ∼250 ms. At this point, a large, P3-like component was rigidly delayed, reflecting a bottleneck and serial processing. fMRI related this effect to an extended network of distributed areas, mostly located in bilateral parietal and prefrontal cortices, within which subtle timing differences could be observed. Many of these areas were jointly activated by the T1 and T2 tasks performed in isolation, suggesting that they form a central processing network shared by various tasks and responsible for the dual-task bottleneck.

Our results extend previous ERP studies of the PRP, all of which agree that it does not delay the initial P1 and N1 sensory events, although their amplitude may be reduced ( Brisson et al., 2007 ). The N2PC, associated with attention deployment, is also sharply attenuated but with little or no delay ( Brisson and Jolicoeur, 2007 ; Brisson et al., 2007 ). On the contrary, the lateralized readiness potential, which indexes motor preparation, is rigidly delayed by an amount comparable to response time ( Osman and Moore, 1993 ; Jentzsch et al., 2007 ). Such a “bracketing” of the PRP effect on both sensory and motor sides is compatible with a main locus of interference at the stage of response selection, in agreement with psychological theorizing ( Pashler, 1984 ; Pashler and Johnston, 1989 ; Sigman and Dehaene, 2005 ). Yet previous studies have not identified a clear ERP component associated with this central stage. The novelty P3 component, a subtraction wave evoked by unexpected infrequent stimuli that forms a subset of the main P3, is delayed during the PRP, but only by ∼70 ms ( Luck, 1998 ; Arnell et al., 2004 ; Dell'acqua et al., 2005 ; Sessa et al., 2007 ), an interval considerable smaller than the delay observed in response times [although Dell'acqua et al. (2005) found the two delays to be correlated].

With respect to this background, our study is the first to examine the entire profile of the ERP rather than a reduced subtraction. This analysis demonstrated that the bulk of the late evoked potential, which is dominated by a P3-like late positive complex, is in fact delayed by an amount comparable to the PRP effect on RTs. Time-resolved fMRI confirms that the PRP delay on parietal and prefrontal activation can be as large as several hundreds of milliseconds. In fact, a surprising finding was that the fMRI delay, in dorsolateral prefrontal and intraparietal cortices, exceeded the PRP delay at the shortest SOA ( Fig. 7 ), which may indicate the deployment of additional higher-level control when T1 and T2 occur simultaneously, perhaps to impose the appropriate task order ( Sigman and Dehaene, 2006 ).

Recently, it has been proposed that the P3 may be related to access to a global workspace associated with conscious reportability ( Sergent et al., 2005 ; Del Cul et al., 2007 ). According to this theory, a distributed set of neurons with long axons provides a global “broadcasting” system enabling communication between arbitrary and otherwise not directly connected brain processors ( Baars, 1989 ; Dehaene et al., 1998 ; Dehaene and Naccache, 2001 ). Global workspace theory can explain why response selection generally imposes a dual-task bottleneck. In most psychological tasks, the relation between stimuli and responses is entirely arbitrary and thus requires the temporary mapping between otherwise independent processors. Establishing such a new arbitrary interconnection should involve central workspace mediation. Supporting this interpretation, interference is drastically reduced for highly practiced or nonarbitrary tasks ( Lien et al., 2002 , 2005 ; Greenwald, 2003 ).

Our fMRI results indicate that a large parietofrontal network shows delayed activity during the PRP. This is compatible with much fMRI literature on the brain mechanisms of central capacity limits ( Marois and Ivanoff, 2005 ). However, the majority of these previous neuroimaging studies are not time resolved and relied on indirect means of identifying the cerebral substrates of the PRP. When contrasting short versus long SOAs, no difference is seen in the total amount of fMRI activation ( Jiang et al., 2004 ), unless subjects engage more attentional resources to compensate for the interference, in which case a right inferior frontal increase is seen ( Jiang, 2004 ; Jiang et al., 2004 ). Such lack of activation differences is expected by the PRP model, which predicts only subtle timing differences. A single fMRI study has investigated the temporal delays in fMRI activation during the PRP ( Dux et al., 2006 ). Dux et al. (2006) used a design in which a difficult T1 task imposed an extended delay, which led to a detectable delayed peak of activation in left posterolateral prefrontal cortex associated with the PRP. However, the method was not very sensitive and only permitted analysis of few regions of interests. The present fMRI method goes beyond this previous work, because it permits whole-brain voxel-based analysis and provides highly sensitive between-subjects statistics. This allowed us to extend this previous finding, showing that the delay of the PRP holds for a broad array of bilateral parietal and frontal regions. The precise quantitative dependence of the temporal activation of this large cluster is in good agreement with the delay observed in the P3 component: no delay during the interference regime, and a delay of 300 ms during the noninterference regime. This finding suggests that these independent measures may be reflecting a common cerebral substrate.

More importantly, we achieved a full parsing of the responsive network, identifying distinct processing stages beyond the serial bottleneck. A massive cluster in the superior temporal cortex reflected perfect parallel processing, firmly constraining the extension of the cerebral locus of the bottleneck. Although this region was the natural candidate for a parallel sensory stage, the extent to which sensory areas may participate to the central bottleneck may vary according to task requirements. Recordings from the primary visual cortex of awake monkeys have shown that a visual stimulus evokes a first transient response, determined by stimulus properties and unaffected by attention, followed by a second wave of activity, which is modulated by stimulus visibility, affected by a concurrent stimulus ( Kovács et al., 1995 ; Lamme et al., 2002 ) and suppressed by anesthetics ( Lamme et al., 1998 ). This specificity suggests an engagement of the central workspace system, and thus, that the same neuron may be involved in distinct processing stages within the same task ( Gilbert and Sigman, 2007 ). Further experiments are required to determine whether this second wave of activity shows a dual-task delay characteristic of the serial processing bottleneck.

Cognitive theories differ as to the exact nature of the processes causing the PRP bottleneck. It might be a response selection process ( Pashler, 1984 ) or it might involve a more extended set of processes, including executive components of task engaging and disengaging ( Allport et al., 1994 ; Meiran et al., 2000 ; Logan and Gordon, 2001 ; Ruthruff et al., 2001 ; Sigman and Dehaene, 2006 ; Jentzsch et al., 2007 ) as well as delays in response initiation ( De Jong, 1993 , 1995 ; Sigman and Dehaene, 2006 ).

Although the vast majority of the data could be accounted for by a simple passive bottleneck model, with serial queuing on a “first-come, first-served” basis, some aspects of the data went beyond such a simple model. We therefore suggest that the “extended bottleneck” view is most compatible with our results, for at least two reasons. First, the extended array of areas affected by the PRP suggests that a broad array of processes cause the delay. Although this large set of areas might implement just a single cognitive stage of response selection, it seems more likely to correspond to the deployment of multiple hierarchically organized executive operations ( Koechlin et al., 2003 ; Koechlin and Jubault, 2006 ). Second, we observed an explicit T2-related ERP at the end of T1 processing, in advance of the T2 stimulus, and a modulation of the amplitude of the early T2-evoked perceptual components with SOAs. These findings suggest that at least part of the acceleration of RT2 with SOA is attributable to the deployment of active task-switching and stimulus expectancy processes. The time-resolved methods that we introduced here provide basic tools with which to further explore these open issues.

This work was supported by Inserm, Commissariat à l'Energie Atomique, and the Human Frontiers Science Program. We gratefully acknowledge the contributions of Martin Graziano and of members of the Cognitive Neuroimaging Unit, particularly Ghislaine Dehaene-Lambertz, Antoinette Jobert, Marco Buiatti, and Antoine Delcul. Denis Le Bihan provided essential administrative support.

• Correspondence should be addressed to Mariano Sigman, Integrative Neuroscience Laboratory, Physics Department, University of Buenos Aires, University City, Pabellon I, 1428 Buenos Aires, Argentina. sigman{at}df.uba.ar
• Moscovitch M
• Allport D ,
• Arnell KM ,
• Helion AM ,
• Hurdelbrink JA ,
• Ashbridge E ,
• Bellgowan PS ,
• Bandettini PA
• Brisson B ,
• Jolicoeur P
• Robitaille N ,
• Dehaene S ,
• Kerszberg M ,
• Changeux JP
• Del Cul A ,
• Baillet S ,
• Dell'acqua R ,
• Jolicoeur P ,
• Vespignani F ,
• Ivanoff J ,
• Asplund CL ,
• Formisano E ,
• Gilbert CD ,
• Greenwald AG
• Jentzsch I ,
• Leuthold H ,
• Kanwisher N
• Richter W ,
• Koechlin E ,
• Kouneiher F
• Spekreijse H
• Lamme VAF ,
• Proctor RW ,
• McCann RS ,
• Ruthruff E ,
• Luknowsky DC ,
• Pashler H ,
• Johnston JC
• Johnston JC ,
• Raichle ME ,
• MacLeod AM ,
• Snyder AZ ,
• Powers WJ ,
• Gusnard DA ,
• Pashler HE ,
• Sergent C ,
• Verleger R ,
• Dell'Acqua R
• Lebihan D ,

In this issue

• Table of Contents
• Table of Contents (PDF)
• About the Cover
• Index by author

Thank you for sharing this Journal of Neuroscience article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Citation Manager Formats

• EndNote (tagged)
• EndNote 8 (xml)
• RefWorks Tagged
• Ref Manager

• Tweet Widget
• Facebook Like
• Google Plus One

Jump to section

Responses to this article, jump to comment:, related articles, cited by..., more in this toc section.

• Memory Retrieval Has a Dynamic Influence on the Maintenance Mechanisms That Are Sensitive to ζ-Inhibitory Peptide (ZIP)
• Neurophysiological Evidence for a Cortical Contribution to the Wakefulness-Related Drive to Breathe Explaining Hypocapnia-Resistant Ventilation in Humans
• Monomeric Alpha-Synuclein Exerts a Physiological Role on Brain ATP Synthase

Behavioral/Systems/Cognitive

• Influence of Reward on Corticospinal Excitability during Movement Preparation
• Identification and Characterization of a Sleep-Active Cell Group in the Rostral Medullary Brainstem
• Gravin Orchestrates Protein Kinase A and β2-Adrenergic Receptor Signaling Critical for Synaptic Plasticity and Memory

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Dual-task interference in simple tasks: data and theory

Affiliation.

• 1 Department of Psychology 0109, University of California at San Diego, La Jolla 92093.
• PMID: 7972591
• DOI: 10.1037/0033-2909.116.2.220

People often have trouble performing 2 relatively simple tasks concurrently. The causes of this interference and its implications for the nature of attentional limitations have been controversial for 40 years, but recent experimental findings are beginning to provide some answers. Studies of the psychological refractory period effect indicate a stubborn bottleneck encompassing the process of choosing actions and probably memory retrieval generally, together with certain other cognitive operations. Other limitations associated with task preparation, sensory-perceptual processes, and timing can generate additional and distinct forms of interference. These conclusions challenge widely accepted ideas about attentional resources and probe reaction time methodologies. They also suggest new ways of thinking about continuous dual-task performance, effects of extraneous stimulation (e.g., stop signals), and automaticity. Implications for higher mental processes are discussed.

PubMed Disclaimer

Similar articles

• Dual-task interference: Bottleneck constraint or capacity sharing? Evidence from automatic and controlled processes. Wu Y, Wang Q. Wu Y, et al. Atten Percept Psychophys. 2024 Apr;86(3):815-827. doi: 10.3758/s13414-024-02854-1. Epub 2024 Feb 28. Atten Percept Psychophys. 2024. PMID: 38418805
• Dual-task interference: attentional and neurophysiological influences. Hiraga CY, Garry MI, Carson RG, Summers JJ. Hiraga CY, et al. Behav Brain Res. 2009 Dec 14;205(1):10-8. doi: 10.1016/j.bbr.2009.07.019. Epub 2009 Jul 23. Behav Brain Res. 2009. PMID: 19631693
• Ideomotor compatibility enables automatic response selection. Maquestiaux F, Lyphout-Spitz M, Ruthruff E, Arexis M. Maquestiaux F, et al. Psychon Bull Rev. 2020 Aug;27(4):742-750. doi: 10.3758/s13423-020-01735-6. Psychon Bull Rev. 2020. PMID: 32323163 Clinical Trial.
• [Aging and cognitive slowing: example of attentional processes--evaluation procedures and related questions]. Eusop E, Sebban C, Piette F. Eusop E, et al. Encephale. 2001 Jan-Feb;27(1):39-44. Encephale. 2001. PMID: 11294037 Review. French.
• Mind-wandering, cognition, and performance: a theory-driven meta-analysis of attention regulation. Randall JG, Oswald FL, Beier ME. Randall JG, et al. Psychol Bull. 2014 Nov;140(6):1411-1431. doi: 10.1037/a0037428. Epub 2014 Aug 4. Psychol Bull. 2014. PMID: 25089941 Review.
• No Evidence for Semantic Prediction Deficits in Individuals With Cerebellar Degeneration. King M, Bruinsma S, Ivry RB. King M, et al. Neurobiol Lang (Camb). 2024 Aug 15;5(3):635-651. doi: 10.1162/nol_a_00083. eCollection 2024. Neurobiol Lang (Camb). 2024. PMID: 39175790 Free PMC article.
• Dual-Task Performance in Individuals With Chronic Obstructive Pulmonary Disease: A Systematic Review With Meta-Analysis. Pasten JG, Aguayo JC, Aburto J, Araya-Quintanilla F, Álvarez-Bustos A, Valenzuela-Fuenzalida JJ, Camp PG, Sepúlveda-Loyola W. Pasten JG, et al. Pulm Med. 2024 Aug 10;2024:1230287. doi: 10.1155/2024/1230287. eCollection 2024. Pulm Med. 2024. PMID: 39157540 Free PMC article. Review.
• Implicit learning provides advantage over explicit learning for gait-cognitive dual-task interference. Nishimoto R, Inokuchi H, Fujiwara S, Ogata T. Nishimoto R, et al. Sci Rep. 2024 Aug 7;14(1):18336. doi: 10.1038/s41598-024-68284-z. Sci Rep. 2024. PMID: 39112521 Free PMC article. Clinical Trial.
• Presentation of multiple task instructional sets impacts auditory Stroop performance during dual task locomotion. Pitman J, Kissack B, Vallis LA. Pitman J, et al. Exp Brain Res. 2024 Aug 7. doi: 10.1007/s00221-024-06903-x. Online ahead of print. Exp Brain Res. 2024. PMID: 39110163
• Erroneous saccade co-execution during manual action control is independent of oculomotor stimulus-response translation ease. Kürten J, Raettig T, Huestegge L. Kürten J, et al. Psychol Res. 2024 Jul 30. doi: 10.1007/s00426-024-01989-y. Online ahead of print. Psychol Res. 2024. PMID: 39080022

Publication types

• Search in MeSH

Related information

• Cited in Books

Grants and funding

• 1-R29-MH45584/MH/NIMH NIH HHS/United States

LinkOut - more resources

Full text sources.

• American Psychological Association
• Ovid Technologies, Inc.

Other Literature Sources

• The Lens - Patent Citations
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Psychometric functions of dual-task paradigms for measuring listening effort

Yu-hsiang wu.

1 Department of Communication Sciences and Disorders, The University of Iowa

Elizabeth Stangl

Xuyang zhang, joanna perkins, emily eilers.

The purpose of the study was to characterize the psychometric functions that describe task performance in dual-task listening effort measures as a function of signal-to-noise ratio (SNR).

Younger adults with normal hearing (YNH, n = 24; Experiment 1) and older adults with hearing impairment (OHI, n = 24; Experiment 2) were recruited. Dual-task paradigms wherein the participants performed a primary speech recognition task simultaneously with a secondary task were conducted at a wide range of SNRs. Two different secondary tasks were used: an easy task (i.e., a simple visual reaction-time task) and a hard task (i.e., the incongruent Stroop test). The reaction time (RT) quantified the performance of the secondary task.

For both participant groups and for both easy and hard secondary tasks, the curves that described the RT as a function of SNR were peak shaped. The RT increased as SNR changed from favorable to intermediate SNRs, and then decreased as SNRs moved from intermediate to unfavorable SNRs. The RT reached its peak (longest time) at the SNRs at which the participants could understand 30% to 50% of the speech. In Experiments 1 and 2 the dual-task trials that had the same SNR were conducted in one block. To determine if the peaked shape of the RT curves was specific to the blocked SNR presentation order used in these experiments, YNH participants were recruited (n = 25; Experiment 3) and dual-task measures, wherein the SNR was varied from trial to trial (i.e., non-blocked), were conducted. The results indicated that, similar to the first two experiments, the RT curves had a peaked shape.

Conclusions

Secondary task performance was poorer at the intermediate SNRs than at the favorable and unfavorable SNRs. This pattern was observed for both YNH and OHI participants and was not affected by either task type (easy or hard secondary task) or SNR presentation order (blocked or non-blocked). The shorter RT at the unfavorable SNRs (speech intelligibility < 30%) possibly reflects that the participants experienced cognitive overload and/or disengaged themselves from the listening task. The implication of using the dual-task paradigm as a listening effort measure is discussed.

INTRODUCTION

Understanding speech involves both auditory and cognitive factors (e.g., Kiessling et al. 2003 ; Worrall & Hickson 2003 ; Pichora-Fuller & Singh 2006 ; Humes 2007 ). Listening effort, which is defined as “the mental exertion required to attend to, and understand, an auditory message” ( McGarrigle et al. 2014 ), has been recognized as an important dimension of speech perception and hearing enhancement device outcomes. When the target auditory message is speech, listening effort is often conceptualized as the cognitive resources allocated for speech processing ( Hick & Tharpe 2002 ; Fraser et al. 2010 ; Gosselin & Gagné 2010 ; Zekveld et al. 2010 ).

Among different methodologies that can objectively quantify listening effort, the dual-task paradigm is one of the most widely-used behavioral measures ( Gosselin & Gagné 2010 ). In this paradigm, listeners perform a primary speech recognition task concurrently with a secondary task. The former is referred to as the primary task because listeners are instructed to maximize speech recognition performance. The difficulty of the speech recognition task is systematically varied during the test session and the change in secondary task performance is taken as an index of the shift in allocation of cognitive resources for speech processing, i.e., listening effort. This interpretation assumes that (1) performance on each of the tasks requires some common cognitive resource allocation and (2) cognitive resources are limited ( Kahneman 1973 ). Dual-task paradigms have been used to investigate the effect of age ( Gosselin & Gagné 2011 ; Desjardins & Doherty 2013 ; Degeest et al. 2015 ), hearing loss ( Hick & Tharpe 2002 ), visual cues ( Fraser et al. 2010 ; Picou et al. 2013 ; Picou & Ricketts 2014 ), hearing aids ( Downs 1982 ; Hornsby 2013 ), noise reduction algorithms ( Sarampalis et al. 2009 ; Desjardins & Doherty 2014 ), and directional microphones ( Wu et al. 2014 ) on listening effort.

Although dual-task tests are widely used, it is less clear at what speech intelligibility level the test will be most sensitive to changes in listening effort (i.e., changes in secondary task performance). Specifically, researchers have speculated that dual-task measures conducted in conditions wherein the primary speech recognition task is too easy (e.g., quiet) or too difficult (e.g., high-level background noise) may not be sensitive (e.g., Picou et al. 2013 ). Gatehouse and Gordon (1990) measured listening effort using the reaction time (RT) to speech stimuli. They examined the effect of hearing aid amplification on listening effort in four conditions: when the unaided speech intelligibility was between 15 to 85%, close to 50%, and close to 85%, and when the benefit of amplification on speech intelligibility was less than 6%. These researchers found that their listening effort measure was more sensitive in the first two conditions. No prior study, however, has systematically examined the relationship between speech intelligibility and secondary task performance in dual-task listening effort measures.

To fill this gap, the objective of the current research was to characterize the psychometric functions that describe task performance in dual-task listening effort measures as a function of signal-to-noise ratio (SNR). It was hypothesized that speech recognition performance would decrease monotonically as SNR decreases. Based on the Ease of Language Understanding (ELU) model that describes a conceptual framework for speech understanding ( Rönnberg 2003 ; Rönnberg et al. 2008 ; Rönnberg et al. 2013 ), it was further hypothesized that secondary task performance would decrease monotonically as SNR (and speech intelligibility) decreases and might reach an asymptote level at very poor SNRs. Specifically, the ELU model suggests that speech information is rapidly and automatically bound in an episodic buffer and then compared to long-term memory. If the information in the buffer matches the long-term memory, speech will be recognized and there is no need for top-down processing. If the speech information input cannot immediately match the long-term memory, explicit and deliberate working memory top-down processes will be invoked to compensate for this mismatch. Because at poorer SNRs speech information will be more degraded and the mismatch will be more likely to happen, more top-down processing will be recruited for speech recognition and less processing will be available to the secondary task, resulting in poorer secondary task performance.

This hypothesis is supported by empirical data showing that the cognitive processing load of a task increases monotonically as the task becomes more demanding ( Peavler 1974 ; Cabestrero et al. 2009 ; Zekveld & Kramer 2014 ). For example, Cabestrero et al (2009) measured the cognitive processing load of an auditory digit span recall task using pupillometry. The number of the to-be-recalled digits was manipulated to create three load conditions (5, 8, and 11 digits). The results showed that pupil response systematically increased as more digits were presented. In the most difficult 11-digit condition, the pupil response reached an asymptote level at the ninth digit and remained stable until the last digit was presented. More recently, Zekveld and Kramer (2014) used pupil response to measure the cognitive processing load of a speech recognition task from a group of younger adults with normal hearing. The experiment consisted of one quiet and three noisy conditions (the first experiment of Zekveld & Kramer 2014 ). The mean speech intelligibility of the four conditions was 99%, 94%, 54%, and 0%, respectively. The result showed that pupil response increased linearly as SNR and speech intelligibility decreased.

There is evidence, however, suggesting that secondary task performance of dual-task listening effort measures may not decrease monotonically as SNR decreases ( Poock 1973 ; Granholm et al. 1996 ; Zekveld & Kramer 2014 ). Granholm et al (1996) measured pupil response of an auditory digit span recall task in three load conditions (5, 9, and 13 digits). The 5- and 9-digit conditions were low- and moderate-load conditions, respectively, and the 13-digit condition was considered overload as it exceeded the memory span and was almost impossible to recall completely. The study results showed that pupil response, instead of increasing monotonically as the recall task became more demanding, was smaller in the 13-digit condition than in the 5- and 9-digit conditions. Similar findings were observed by a second Zekveld and Kramer (2014) experiment where a speech recognition test was administered at a wide range of SNRs. The experiment results indicated that as SNR decreased from −4 dB (speech intelligibility: 80%) to −8 dB (intelligibility: 50%), pupil response increased. As SNR further decreased from −8 dB to −36 dB (intelligibility: 0%), however, pupil response decreased linearly. In other words, the function that described pupil response across SNRs had a peaked shape. Zekveld and Kramer (2014) suggested that their participants experienced cognitive overload (i.e., task demands exceed an individual’s ability) at poorer SNRs, as did the subjects in the overload condition of the study by Granholm et al (1996) . It is possible that people in the overload condition tend to give up on the task. Zekveld and Kramer (2014) asked subjects to report how often they gave up listening to speech and found that the frequency of giving up increased as speech intelligibility decreased.

The research finding showing that pupil response is smaller in the very demanding, overload condition than in the moderate-load condition is consistent with a study by Petersen et al (2015) , who measured working memory load using alpha oscillations of electroencephalogram. In the experiment, older adults with various degrees of hearing loss wore hearing aids and were asked to conduct an auditory recall task. The results showed that in low- and moderate-load conditions alpha power increased as the degree of hearing loss increased. In the most demanding condition, however, alpha power of subjects with moderate hearing loss was lower than that of subjects with mild hearing loss. Similarly, Sander et al. (2012) found that alpha power in high demanding conditions is reduced in older adults compared to children and younger adults. Therefore, this line of research suggests that (1) the psychometric function of secondary task performance in the dual-task listening effort measure would have a peaked shape (i.e., performance being poorer at intermediate SNRs than at favorable and unfavorable SNRs) and (2) the functions of younger adults with normal hearing (YNH) and older adults with hearing impairment (OHI) would have different shapes.

The current research consisted of three experiments. The first two experiments characterized the psychometric functions of the dual-task paradigm for YNH (Experiment 1) and OHI (Experiment 2) listeners. To determine if the trend of task performance across SNRs found in Experiments 1 and 2 was specific to the methodology used in the dual-task measures, Experiment 3 was conducted on YNH listeners using a different methodology. For all three experiments, dual-task paradigms wherein subjects performed a primary sentence recognition task simultaneously with a secondary task were used. Because the dual-task interference is dependent on factors such as task demand and degree to which overlapping resources are required ( Hazeltine et al. 2006 ; Wickens 2008 ), two different secondary tasks, one being simpler and one being more complex, were used in the study to examine the effect of secondary task on the shape of psychometric function. Because objective and subjective measures may assess different aspects of listening effort (e.g., Fraser et al. 2010 ; Zekveld et al. 2010 ), the current study also characterized the psychometric function of self-reported listening effort.

EXPERIMENT 1

The purpose of Experiment 1 was to characterize the pattern of task performance across SNRs in dual-task listening effort measures for YNH listeners.

Materials and Methods

Participants.

In total 26 younger adults were recruited. Most of them were college students. Two participants were unable to complete the study due to time conflict of their second laboratory visit (see below for experiment procedures). For the 24 YNH (12 males and 12 females) participants who completed the study, their ages ranged from 19 to 30 years with a mean of 23.7 years (SD = 3.6). The participants had pure-tone thresholds better than 25 dB HL at 0.5, 1, 2, and 4 kHz ( ANSI 2010 ). All participants are native speakers of English. The sample size was determined based on a pilot study, which indicated that a sample size of 24 were needed in order to detect the effect of SNR on task performance (assuming α = 0.05 and power = 0.8).

Dual-task paradigm

Two dual-task paradigms that included different secondary tasks were used. In the dual-task measure the participants performed a speech recognition task simultaneously with either an easy or hard secondary task.

Primary speech recognition task

The Hearing in Noise Test (HINT) ( Nilsson et al. 1994 ) was used as the speech material. In order to ensure that the dual-task measure was conducted across a wide range of speech intelligibility for each participant, individualized SNRs were used. Specifically, before the dual-task measure, an individual SNR-50 at which the participant could understand 50% of speech was determined using an adaptive SNR procedure. During the SNR-50 measure, the participants listened to the HINT sentences and repeated as much as possible. The speech level was fixed at 65 dB SPL. The level of the noise, which was the speech-shaped noise of the HINT, was adjusted adaptively depending on the participant’s responses using the one-down, one-up procedure in 2 dB steps. The correct response of each sentence was based on the repetition of the whole sentence, with minor exceptions such as a/the and is/was. Twenty HINT sentences were used and the SNRs of the last 16 sentences were averaged ( Nilsson et al. 1994 ). This averaged SNR minus 2 dB was defined as an individual’s SNR-50. According to the pilot study, at this SNR-50 a listener’s speech recognition performance would be close to 50% correct if the scoring of the HINT was based on words (which was the scoring method used in the dual-task measures).

For a given participant, 11 SNRs ranging from −10 to +10 dB in 2-dB steps relative to this participant’s SNR-50 were created and used in the speech recognition task of the dual-task measure. The speech presentation level was fixed at 65 dB SPL for all 11 SNR conditions. The HINT sentences (different sentences from those used in the SNR-50 measure) and speech-shaped noise were used. For each of the 11 SNR conditions, 20 trials (20 HINT sentences) were conducted. In each trial, the noise was presented 1 sec before the onset the sentence and ended approximately 1 sec after the offset of the sentence. The trials that had the same SNR were administered in one block. The order of SNR block was randomized. The participants were asked to repeat as much of each sentence as possible. The participants’ performance (i.e., the HINT score) at a given SNR was quantified by dividing the number of words the participants repeated correctly by the total number of words in the 20 sentences.

Secondary task

The visual stimuli of the Stroop test ( Stroop 1935 ) were used in the secondary task. During testing, visual stimuli of color words displayed in different font colors were shown in the middle of a computer monitor. Four color words and font colors were used: red, blue, green, and yellow. The combination of color word and font color was randomized, but the color word was always inconsistent with the font color. Below the stimulus word, the computer monitor showed four boxes containing “RED”, “BLUE”, “GREEN”, and “YELLOW,” respectively, to represent the four virtual response buttons. The font color of the words in the virtual button box was black.

Using the same visual stimuli, two tasks were created. For the easy task, listeners were asked to press the space bar on the keyboard as quickly as possible after stimulus word presentation, regardless of the word and the font color. In other words, the easy task was a simple visual reaction-time task. When the space bar was pressed, all four virtual buttons on the screen were highlighted to indicate the response.

In the hard task, the Stroop test paradigm was used. In particular, listeners were asked to respond to the font color, instead of the word, by pressing a keyboard button assigned to a given color as quickly as possible (i.e., the incongruent condition of the Stroop test). The keyboard buttons “D”, “C”, “M”, and “K” were assigned to font color red, blue, green, and yellow, respectively. To assist the participants in determining which keyboard button to press during the testing, the relative position of the four virtual buttons on the screen was arranged to spatially map that of the four keyboard buttons. When a given button was pressed, the corresponding virtual button on the screen was highlighted to indicate the response. Because the participants needed to inhibit the semantic meaning of the stimulus word and determine which button to push in the hard task, this task was more demanding and would interfere more with the speech recognition task than the easy task.

For both the easy and hard tasks, one stimulus word was presented with one HINT sentence in each trial. Because the total processing load for sentence understanding reaches a maximum at the end of the sentence ( Winn et al. 2015 ), the stimulus word was presented at a random time during the second half of each HINT sentence presentation. The RT from stimulus word onset until a keyboard button was pressed quantified the performance of the secondary task.

Subjective effort rating

The participants were asked to rate their perceived listening effort after listening to 20 HINT sentences in a given SNR block of the dual-task measure. The participants answered the question “how hard were you trying to understand the speech” using a 21-point scale ranging from 0, representing “not at all,” to 100, representing “very, very hard.” The question and the scale were adapted from the “effort” question of the National Aeronautics and Space Administration Task Load Index questionnaire ( Hart & Staveland 1988 ). Likely due to the wide SNR range used in the experiment, ceiling effect was observed on some subjects in the pilot study. In order to minimize the ceiling (and floor) effect, the participants were allowed to use larger numbers than 100 and negative numbers to rate listening effort ( Hällgren et al. 2005 ). For this reason, two arrowheads were placed at the two ends of the 21-point scale, one on each end, pointing away from the scale. Because the pilot study indicated that listeners did not report the highest effort at the poorest SNR, the participants were not trained to use the scale; i.e., they were not provided with samples of the most favorable and unfavorable SNRs to anchor the end points of the scale.

The study was approved by the Institutional Review Board of the University of Iowa. After agreeing to participate in the study and signing the consent form, the participants’ pure tone thresholds were measured, followed by the SNR-50 measure. Afterward, the dual-task tests were conducted. During the testing, the participants were instructed to repeat as much of each sentence as possible and respond to the visual stimulus as quickly (and accurately) as possible. The participants were asked to give priority to the speech recognition task, i.e., they should always try to maximize their speech recognition performance. The participants were also asked not to let the repetition of sentences affect their response speed to the visual stimuli of the secondary task, i.e., they should not repeat the sentence until they pushed the keyboard button. Prior to the experiment, a tutorial and practice session was given to familiarize participants with the tasks. For each secondary task, at least 60 trials were given in the practice. The dual-task experiment was conducted in 22 conditions (2 secondary tasks x 11 SNRs). Due to the limited length of the HINT, the HINT sentences were used twice in the experiment. Therefore, the experiment was completed in two laboratory visits: one for the easy task and one for the hard task. The interval between the two visits was at least one month in order to minimize the learning effect. The order of easy/hard task was randomized across the participants. Momentary compensation was provided to the participants at the completion of the study.

The experiment was conducted in a sound treated booth. All auditory stimuli (HINT sentences and noise) were presented via earphones. The auditory stimuli were generated by a computer and an M-Audio (Cumberland, Rhode Island) ProFire 610 sound interface, routed to a Grason-Stadler (Eden Prairie, Minnesota) GSI-61 audiometer, an Alesis (Cumberland, Rhode Island) DEQ830 digital equalizer, a Samson (Hauppauge, New York) Servo 120 amplifier, and then presented through a pair of Sennheiser (Wedemark, Germany) IE8 insert earphones. The output of the earphones was calibrated in a G.R.A.S. (Holte, Denmark) IEC 711 RA0045 Ear Simulator. The visual stimuli were presented using a 19 in. computer monitor, which was placed in front of the participants. The psychological testing software E-prime 2.0 (Psychology Software Tools, Inc., Sharpsburg, Pennsylvania) was used to present auditory and visual stimuli, collect participants’ responses, and measure the RT.

Data analysis

Data were first processed before analysis. For the HINT score (in percent), a logit transformation was conducted to linearize the relationship and homogenize the variance (logit-transformed score = log ((HINT score + 1) / (101 − HINT score))). For the easy secondary task, because the distribution of the RT across 20 trials at a given SNR was skewed, the median of the 20 RTs served as the RT of that SNR condition and was used in analysis. For the hard task (the Stroop test), the response accuracy was first examined. The overall accuracy across all conditions and participants was 95.3% (SD = 5%). The Friedman repeated measures analysis of variance on ranks indicated that SNR did not have an effect on response accuracy ( p = 0.78), suggesting that Stroop accuracy did not vary with speech intelligibility. The difference in Stroop accuracy between the participants’ first and last conditions was also found to be non-significant ( p = 0.13), indicating that the learning effect of the Stroop test was minimal. Because the Stroop accuracy was high and was stable across SNRs and time, the median RT across all 20 trials at a given SNR, regardless of accuracy, served as the RT of that SNR and was used in analysis. The distribution of the median RT across all participants, 11 SNRs, and two secondary tasks was then examined. Because the distribution was right-skewed, the RT was log-transformed before analysis (log-transformed RT = log (RT)). For the subjective effort rating, the distribution was first examined. Among all ratings, 12 ratings (2.2%) had values larger than 100 and the highest rating was 200. No rating was smaller than 0. Because some participants used larger numbers than 100 to rate listening effort, the subjective effort rating was linearly transformed so that the scale of the rating was the same across all participants. In particular, for a given participant, the subjective effort rating was linearly rescaled such that the maximum and minimum ratings across all 22 conditions (11 SNRs x 2 secondary tasks) were 100 and 0, respectively. Because the distribution of the rescaled subjective listening effort rating across all participants and test conditions was normal, no further transformation was conducted.

To characterize the psychometric function (i.e., the trend of performance across SNRs) of dual-task listening effort measures, a linear mixed model was used to analyze the repeated measure data. In particular, the model fits polynomial (linear, quadratic, and cubic) terms of SNR to the HINT score (logit-transformed), RT (log-transformed), or subjective effort rating data (rescaled) for each participant. The SNR quadratic and cubic terms were rescaled using quadratic term = (SNR/5) 2 and cubic term = (SNR/5) 3 to aid in convergence. In polynomial models, the linear term reflects an overall slope of the function; the quadratic term reflects the shape of the primary inflection point of the curve (positive and negative coefficients reflect concave-upward and -downward curves, respectively), and the cubic term reflects the steepness of the secondary inflection point. Fixed effects considered in the model were a secondary task term (easy/hard), three SNR polynomial terms (linear, quadratic, and cubic), and all two-way interactions. A random intercept, random SNR linear, quadratic and cubic terms were also included in the model. The random-effect term was then removed one by one to identify the model that had minimum Akaike Information Criterion (AIC). For the HINT score and RT, the model that included the random effect of intercept and SNR linear term had minimum AIC and was selected; for subjective effort rating, the model that included the random effect of intercept and SNR linear and quadratic terms had minimum AIC and was selected. The effects of other variables were estimated as fixed effects only. After the model was settled with respect to the random effect, the fixed-effect terms were examined. When an interaction term was not significant, it was excluded from the final model. The SNR cubic term was not included in the final model if it was not significant. The analysis was conducted using the Statistical Analysis Software (SAS Institute, Cary, North Carolina).

Speech recognition performance

Figure 1A shows the mean HINT scores across all participants for each secondary task as a function of SNR. The two sigmoidal curves are almost overlapped. The analysis first indicated that none of the interactions was significant. The results further showed that secondary task did not have a significant effect on HINT score ( β = 0.05, F 1, 23 = 0.80, p = 0.38). In contrast, SNR linear ( β = 0.65, F 1, 500 = 2599.1, p < 0.0001), quadratic ( β = −0.05, F 1, 500 = 6.57, p = 0.010), and cubic terms ( β = −0.26, F 1, 500 = 179.3, p < 0.0001) were all significant. These results indicated that HINT score increased as SNR increased and that this trend was similar for both the easy and hard dual-task tests.

Speech recognition score (1A), reaction time (RT) of the secondary task (1B and 1C), and subjective listening effort rating (D) averaged across participants as a function of signal-to-noise ratio (SNR) of Experiment 1. In Figure 1C the y-axes of the RT curves are rescaled so that the curves of the easy task (refer to the left y-axis) and the hard task (refer to the right y-axis) have similar peak heights in the figure. Error bars = 1 SE.

Secondary task performance

Figure 1B shows the RT (the median RT of a given SNR) averaged across all participants for each secondary task at each SNR. Longer RT represents poorer performance. For both tasks, the RT curves had a peaked shape. The RT increased as SNR changed from favorable (i.e., +8 and +10 dB) to intermediate SNRs (0 and −2 dB), and then decreased as SNRs moved from intermediate to unfavorable SNRs (−8 and −10 dB). To better compare the shape of the curves, in Figure 1C the y-axis of the two RT curves shown in Figure 1B was rescaled so that the two curves have similar peak heights in the figure. Figure 1C shows that the two curves are very similar in shape and the curves are fairly symmetrical around the peaks.

The analysis revealed that the interactions and the SNR cubic term were not significant. The effect of secondary task was significant ( β = −0.93, F 1, 23 = 2585.0, p < 0.0001). The SNR linear term was significant ( β = −0.005, F 1, 501 = 6.14, p = 0.014). The negative coefficient indicated that the right side of the curve is generally lower than the left side ( Figure 1C ). The SNR quadratic term was also significant ( β = −0.05, F 1, 501 = 51.7, p < 0.0001). The negative coefficient confirmed the peaked shape of the curves.

After listening to 20 HINT sentences at a given SNR block, the participants rated their listening effort. Higher ratings represented more listening effort. Figure 2D shows the rescaled subjective effort rating averaged across all participants as a function of SNR. Essentially, these curves also display a peaked shape. Compared to the RT curves shown in Figure 1C , the subjective effort rating curves are less symmetric around the peaks.

Average audiograms of study participants in Experiment 2. Error bars = 1 SD.

Mixed-effects analysis first revealed that the interactions and the SNR cubic terms were not significant. The effect of secondary task was significant ( β = 4.43, F 1, 23 = 5.65, p = 0.026), indicating that the participants reported that they tried harder to understand speech in the easy dual-task measure than in the hard dual-task measure. The results further indicated that, while the SNR linear term was not significant ( β = −0.78, F 1, 501 = 2.49, p = 0.12), the quadratic term was ( β = −6.24, F 1, 501 = 25.5, p < 0.0001).

The results of Experiment 1 showed that the RT of the secondary task in the dual-task measure had a non-linear trend over SNRs; that is, as SNR decreased, the RT first increased and then decreased ( Figures 1B and 1C ). More specifically, at favorable SNRs (speech intelligibility close to 100%), the RT was short, suggesting that speech recognition was easy and did not require much top-down processing. As SNR decreased, the RT increased, indicating that the participants used more top-down working memory processing to process the degraded speech signals. This was consistent with the ELU model ( Rönnberg et al. 2008 ). The RT reached its peak at −2 dB or 0 dB relative to SNR-50 (speech intelligibility = 30% to 50%). As the SNR kept decreasing, however, speech recognition performance became poorer while the RT became shorter. The shorter RT suggested that cognitive processing was shifted from the speech recognition task to the secondary task. The peaked shape of RT curve was consistent with Granholm et al (1996) and the second experiment of Zekveld and Kramer (2014) , suggesting that the participants might experience cognitive overload at the unfavorable SNRs.

Recall that the hard secondary task (Stroop test) required the participants to inhibit the semantic meaning of the stimulus word and determine which button to push, while the easy task was a simple visual reaction-time task. Therefore, it is not surprising that the RT of the hard task (0.85 sec, averaged across SNRs) was longer than that of the easy task (0.35 sec). Despite the large difference between the two secondary tasks, the RT curves of the easy and hard tasks had similar shapes, as indicated by the non-significant interaction between secondary task and SNR polynomial terms. The implication of this finding will be further discussed in the General Discussion section at the end of this paper.

Similar to the RT curve, the curve of subjective effort rating had a non-linear trend ( Figure 1D ). This result is not in line with the first experiment of Zekveld and Kramer (2014) , which found that both pupil response and subjective listening effort rating increased linearly as SNR and speech intelligibility decreased.

Of note, the participants reported higher listening effort in the easy than the hard dual-task measures. One speculation on this finding is that the self-reported rating reflected effort allocation between the primary and secondary tasks. Specifically, because the visual-reaction time task was less demanding than the Stroop test, the participants were able to exert more effort to the speech recognition task in the easy than the hard dual-task measure. As a result, when the participants were asked to answer the question “how hard were you trying to understand the speech,” they reported that they tried harder in the easy dual-task measure.

EXPERIMENT 2

The purpose of Experiment 2 was to replicate Experiment 1 using older listeners with hearing impairment (OHI).

In total 24 OHI (10 males and 14 females) participants were recruited and completed the study. Their ages ranged from 56 to 83 years with a mean of 69.9 years (SD = 5.8). The participants were eligible for inclusion in this study if their hearing loss met the following criteria: (1) postlingual bilateral downward-sloping sensorineural hearing loss (air-bone gap < 10 dB); (2) hearing thresholds no better than 20 dB HL at 500 Hz and no worse than 85 dB HL at 3 kHz ( ANSI 2010 ); and (3) hearing symmetry within 15 dB for all test frequencies. The mean pure tone thresholds are shown in Figure 2 . All participants were native speakers of English.

The stimuli, test conditions, equipment and procedures were identical to those used in Experiment 1. Experiment 2 differed from Experiment 1 in that the auditory stimuli (HINT sentences and noise) were spectrally shaped and linearly amplified before being routed to the earphones. The purpose of amplifying the stimuli was to ensure that speech intelligibility could approach 100% at favorable SNRs for all participants. The individual frequency shaping and amplification were based on each participants’ audiometric thresholds and the NAL-NL2 formula ( Keidser et al. 2011 ). Specifically, from an Audioscan Verifit hearing aid analyzer, the NAL-NL2 targets of real ear aided responses (REAR) to a 65-dB SPL speech input (the “carrot passage”) from 0.25 to 6 kHz for each participant were obtained. The REAR targets were used to configure the filter and gain settings of the DEQ830 multi-channel equalizer, one channel for each ear, to shape the one-third octave band spectra of the input signals such that, for the 65-dB “carrot passage” input, the outputs of the earphones met an individual’s NAL-NL2 REAR targets within ±3 dB across 0.25 to 6 kHz. Using the amplified auditory stimuli, the participant’s SNR-50 was measured and the 11 SNRs (−10 to +10 dB relative to SNR-50, 2-dB steps) were created. Before the testing, the participants were asked about their loudness perception of the stimuli. All participants reported that the sound level was appropriate.

Identical to Experiment 1, the response accuracy of the hard secondary task (the Stroop test) was examined before data analysis. The overall accuracy across all conditions and participants was high (97.9%). The accuracy did not vary with SNR ( p = 0.68) and was not different between the participants’ first and last condition ( p = 0.76). Therefore, for both the easy and hard tasks, the median RT across all 20 trials at a given SNR served as the RT of that SNR condition. For the subjective effort rating, the distribution was first examined. Among all ratings, 34 ratings (6.4%) had values larger than 100 and no rating was smaller than 0. The highest rating was 200. Identical to Experiment 1, the subjective effort rating was linearly transformed so that the scale of the rating was the same across all participants. Mixed-effects analysis was then used to determine the effect of secondary task and SNR polynomial terms on HINT score (logit-transformed), RT (log-transformed), or subjective effort rating (rescaled). The fixed and random effects that were included in the models were identical to those in Experiment 1.

The four panels in Figure 3 shows HINT score, RT, rescaled RT, and rescaled subjective effort rating as a function of SNR. For the HINT score ( Figure 3A ), analysis indicated that none of the interactions was significant. The effect of secondary task was not significant either ( β = 0.026, F 1, 23 = 0.21, p = 0.65). In contrast, the SNR linear ( β = 0.60, F 1, 500 = 1889.9, p < 0.0001), quadratic ( β = −0.13, F 1, 500 = 39.5, p < 0.0001), and cubic terms ( β = −0.22, F 1, 500 = 133.3, p < 0.0001) were all significant.

Speech recognition score (3A), reaction time (RT) of the secondary task (3B and 3C), and subjective listening effort rating (3D) averaged across participants as a function of signal-to-noise ratio (SNR) of Experiment 2. In Figure 3C the y-axes of the RT curves are rescaled so that the curves of the easy task (refer to the left y-axis) and the hard task (refer to the right y-axis) have similar peak heights in the figure. Error bars = 1 SE.

Similar to the YNH participants in Experiment 1, the RT curves of the OHI participants were peak shaped ( Figures 3B and 3C ). The peaks were located at −2 dB relative to SNR-50 for both the easy and hard tasks. Analysis results first revealed that none of the interactions was significant, nor was the SNR cubic term. The results further indicated that the easy task’s RT was significantly shorter than that of the hard task ( β = −0.92, F 1, 23 = 1555.1, p < 0.0001). While the SNR linear term was not significant ( β = 0.002, F 1, 501 = 2.07, p = 0.15), the quadratic term was ( β = −0.073, F 1, 501 = 78.7, p < 0.0001),

Although the OHI participants’ subjective effort rating curve had a peaked shape in the hard dual-task measure, the curve in the easy dual-task measure was more like a reversed sigmoidal shape ( Figure 3D ). Mixed-effects analysis showed that subjective listening effort rating was higher in the easy than the hard dual-task measure ( β = 7.55, F 1, 23 = 17.9, p = 0.0003). The SNR linear ( β = −4.54, F 1, 498 = 45.5, p < 0.0001), quadratic ( β = −7.04, F 1, 498 = 37.6, p < 0.0001), cubic terms ( β = 4.18, F 1, 498 = 10.8, p = 0.001) were also significant. The results further indicated that the interaction between secondary task and SNR linear term ( β = 2.25, F 1, 498 = 9.5, p = 0.002) and the interaction between secondary task and SNR cubic term ( β = −4.48, F 1, 498 = 14.5, p = 0.0002) were significant.

Generally, the results of the OHI participants in this experiment were consistent with the findings of the YNH subjects in Experiment 1. For both the easy and hard secondary tasks, the RT curve had a non-linear trend such that the RT initially increased as speech recognition became more difficult and decreased when the HINT score was lower than 30%. For the subjective effort rating, the participants reported more listening effort in the easy than the hard dual-task measure. However, inconsistent with Experiment 1, the significant interaction between secondary task and SNR polynomial terms indicated that the trend of subjective effort rating across SNRs was different between the easy and the hard dual-task measure. This difference mainly resulted from the large discrepancy in effort rating at the unfavorable SNRs ( Figure 3D ). It is unclear why the OHI participants reported high effort at the unfavorable SNRs in the easy dual-task measure but not in the hard dual-task measure, and why this pattern was not observed in the YNH participants of Experiment 1.

EXPERIMENT 3

In both Experiments 1 and 2, 20 dual-task trials that had the same SNR were administered in one block. Because the SNR was fixed across 20 trials, the participants could obtain a general idea about the test difficulty and their speech recognition performance level after the first few trials. As a result, at unfavorable SNRs the participants might easily decide to give up on the listening task in the rest of trials, resulting in shorter RTs. If the SNR was varied from trial to trial, the RT curve might have a different shape. Zekveld and Kramer (2014) have speculated that SNR presentation order (blocked vs. non-blocked) may explain why their two experiments generated inconsistent results regarding the trend of pupil response across SNRs. To investigate if the peaked shape of the RT curve was specific to the blocked SNR design used in Experiments 1 and 2, Experiment 3 characterized the RT curve in dual-task paradigms, wherein the SNR was varied from trial to trial, for YNH listeners.

In total 25 YNH (12 males and 13 females) participants were recruited and completed the study. Most of the participants were college students and their ages ranged from 19 to 30 years with a mean of 21.2 years (SD = 2.7). The participants had pure-tone thresholds better than 25 dB HL at 0.5, 1, 2, and 4 kHz ( ANSI 2010 ). All participants were native speakers of English.

The stimuli, dual-task paradigms, test SNRs, equipment, procedures, and data transformation and analysis were identical to those used in Experiment 1. Experiment 3 differed from Experiment 1 in that, for a given secondary task, the presentation SNR was randomized across the 220 HINT trials (20 sentences x 11 SNRs). Because SNR presentation order was randomized, the participants were not asked to report their perceived listening effort.

Figure 4 shows the results. For the HINT score ( Figure 4A ), analysis indicated that secondary task did not have a significant effect of HINT score ( β = 0.014, F 1, 24 = 0.09, p = 0.77), nor did the SNR quadratic term ( β = −0.016, F 1, 521 = 0.89, p = 0.35). The effect of SNR linear term ( β = 0.64, F 1, 521 = 3738.3, p < 0.0001) and cubic term ( β = −0.25, F 1, 521 = 243.3, p < 0.0001) was significant.

Speech recognition score (4A) and reaction time (RT) of the secondary task (4B and 4C) averaged across participants as a function of signal-to-noise ratio (SNR) of Experiment 3. In Figure 4C the y-axes of the RT curves are rescaled so that the curves of the easy task (refer to the left y-axis) and the hard task (refer to the right y-axis) have similar peak heights in the figure. Error bars = 1 SE.

The RT curves of both the easy and hard tasks were peak shaped ( Figures 4B and 4C ). The curve peaks of the easy and hard tasks were at −2 and 0 dB relative to SNR-50, respectively. Mixed-effects analysis revealed that none of the interactions was significant, nor was the SNR cubic term. In contrast, the secondary task term ( β = −1.04, F 1, 24 = 5018.8, p < 0.0001) and SNR linear ( β = −0.003, F 1, 522 = 3.91, p = 0.04) and quadratic terms ( β = −0.032, F 1, 522 = 37.6, p < 0.0001) were all significant. Similar to Experiment 1, the linear trend was negative (i.e., the right side of the curve was lower than the left side).

Individual difference

Averaged across the participants, the RT curve had a peaked shape and the peak was located at −2 to 0 dB relative to SNR-50 ( Figure 4C ). At the individual level, however, the shape of the RT curve varied considerably. To illustrate this point, Figure 5 shows the hard task RT curve of four participants in Experiment 3. In this figure, the top three curves have a peaked shape, but the width and location of the peak varies. The curve shown at the bottom of the figure has a reversed sigmoidal shape instead of a peaked shape. Although Figure 6 shows the results only from Experiment 3, large individual difference has been observed across all three experiments of the current study.

Reaction time (RT) of the hard secondary task as a function of signal-to-noise ratio (SNR) for four participants (S5, S4, S8, and S3) in Experiment 3. The curves are rescaled to have similar peak heights in the figure.

Reaction time (RT) of the secondary task averaged across participants as a function of signal-to-noise ratio (SNR) of Experiments 1, 2, and 3. YNH: younger adults with normal hearing; OHI: older adults with hearing impairment; Easy/Hard: the type of the secondary task; blocked/non-blocked: SNR presentation order; Exp: experiment.

Comparison across three experiments

To examine if SNR presentation order (blocked vs. non-blocked) had an effect on the trend of RT, analysis on the data collected from Experiments 1 and 3 was conducted. Because it is also of interest to compare the RT curve of YNH and OHI participants, the data of Experiment 2 were included in the analysis. Mixed-effects analysis was performed to investigate the effect of secondary task (easy/hard), SNR polynomial terms, and experiment (between-subject variable, Experiments 1/2/3) on RT (log-transformed). The model included the random effect of intercept and SNR linear term. The effects of other variables were estimated as fixed effects only. Figure 6 summarizes the RT curves of each experiment and each secondary task.

The results revealed that the main effects of secondary task ( F 1, 72 = 3144.8, p < 0.0001), experiment ( F 2, 70 = 19.0, p < 0.0001), and SNR quadratic term ( β = −0.023, F 1, 1525 = 166.3, p < 0.0001) were all significant, while the SNR linear term ( β = −0.003, F 1, 1525 = 3.35, p = 0.067) was not. The interaction between experiment and SNR linear term ( F 2, 1525 = 5.25, p = 0.005) and the interaction between experiment and SNR quadratic term ( F 2, 1525 = 9.56, p < 0.0001) were also significant. The post-hoc comparison showed that the SNR linear and quadratic terms of Experiment 1 did not significantly differ from those of Experiment 3 ( p = 0.45 and p = 0.12, respectively). In contrast, the SNR linear and quadratic terms were significantly different between Experiments 1 and 2 ( p = 0.002 and p = 0.006, respectively) and between Experiments 2 and 3 ( p = 0.017 and p < 0.0001, respectively).

The RT linear and quadratic terms did not differ between Experiments 1 and 3, suggesting that RT curve shape was not affected by SNR presentation order (blocked vs. non-blocked). Note that this result does not necessarily exclude the possibility that the participants actively quit listening at the unfavorable SNRs: the participants might quickly decide to give up on the listening task right after the onset of noise, which was presented 1 sec before the speech.

In contrast, the RT curves of YNH listeners (Experiments 1 and 3) and OHI listeners (Experiment 2) had different shapes. The difference in the SNR quadratic term was because the RT curve of the YNH participants was flatter than that of the OHI participants ( Figure 6 ). This may reflect that OHI listeners exert more effort on speech understanding than YNH listeners ( Desjardins & Doherty 2013 ; Degeest et al. 2015 ). The difference in the linear trend was because the RT curve showed a negative trend in Experiments 1 and 3 (YNH) but not in Experiment 2 (OHI). This difference indicated that YNH participants’ RTs at unfavorable SNRs were longer than that at favorable SNRs, while OHI listeners’ RTs were similar for both the unfavorable and favorable SNRs. The relatively short RT of OHI listeners at unfavorable SNRs may suggest that these listeners experienced more cognitive overload than YNH participants. This speculation was consistent with the study by Petersen et al (2015) , which found that alpha power breakdown is more likely to occur for listeners with more severe hearing loss in the most difficult condition.

GENERAL DISCUSSION AND CONCLUSIONS

The three experiments of the current study examined the task performance of dual-task listening effort measures across a wide range of SNR and speech intelligibility. The results suggested that RT had a non-linear trend across SNRs: RT was the longest at −2 dB or 0 dB relative to SNR-50 and was shorter when speech intelligibility was better than 50% or poorer than 30%. This pattern was observed for both YNH and OHI participants and was not affected by either the type of secondary task (easy or hard) or SNR presentation order (blocked or non-blocked). The result showing that RT reached its peak when speech intelligibility was between 30% and 50% was in line with the second experiment of Zekveld and Kramer (2014) , which found that pupil size was the largest when speech intelligibility was approximately 50%.

Why was the RT shorter at the unfavorable SNRs than the intermediate SNRs? As mentioned, this can be explained by the tendency of actively giving up listening in cognitive overload situations. Because the current study used dual-task paradigms, the peaked shape of the RT curve can also be explained by the adaptive gain theory ( Aston-Jones & Cohen 2005 ). In particular, this theory tries to explain the neurophysiological mechanism of the trade-off between an animal’s exploitative behavior (optimizing the performance of the current task) and exploratory behavior (searching for alternative sources of reward). The adaptive gain theory assumes that the trade-off between these two behaviors is driven by on-line assessments of task-relevant utility; that is, the costs and benefits associated with the task. It is likely that, in the dual-task measures of the current study, the utility of the primary speech recognition task was high at the favorable and intermediate SNRs and the participants expended effort on this task to optimize the performance. As the utility in the speech recognition task waned at very unfavorable SNRs, the participants disengaged themselves from the listening task and exerted more effort on the secondary task to pursue reward.

If the participants disengaged themselves from the listening task, can they “work harder” to improve their speech recognition performance? According to the ELU model, explicit and deliberate working memory top-down processes are invoked when the speech information input is degraded. Therefore, the longer RTs at intermediate SNRs suggest that the participants dedicated more working memory processes at these SNRs than at other SNRs. That is, even when their speech recognition performance was lower than 100% and there was room for improvement, at SNRs other than the intermediate SNRs (including more favorable and unfavorable SNRs) the participants did not allocate all available working memory resources to speech processing. It is unclear whether it is possible for listeners to deliberately dedicate more working memory processes to the task, and if in doing so can improve their speech understanding. Clarifying these issues may advance our understanding about the cognitive mechanism of speech listening in adverse conditions.

Recall that the motivation of the current study was to determine the optimal speech intelligibility level for dual-task listening effort measures. The study results indicate that, due to the RT nonlinear trend across SNRs, the dual-task paradigm should be used cautiously as a measure of listening effort. For example, if a hearing aid technology can improve SNR (e.g., directional microphones), dual-task measures could demonstrate that this technology improves, decreases, or has no effect on secondary task performance, depending on the test SNR and speech intelligibility level. If the change in secondary task performance is taken as an index of the change in listening effort, the result of dual-task measures could show that this hearing aid technology improves speech intelligibility while increasing listening effort. In order to avoid this paradoxical result, it is suggested that a dual-task listening effort measure is conducted at speech intelligibility level higher than 50%. If speech signal is highly degraded and the intelligibility is lower than 30%, individuals may experience cognitive overload and/or disengage the listening task. As a result, data interpretation will be more complex.

Although the peaked shape of the RT curve was consistently observed across the easy and hard secondary tasks used in the current study, this result may not generalize to all dual-task listening effort measures. For example, Pals et al (2013) used dual-task paradigms to measure the effect of spectral resolution (vocoder simulation) on listening effort for YNH listeners. Two different secondary tasks were used: a rhyme-judgment task and a mental rotation task. The results showed that, as the number of spectral channels decreased from 24 to 2 channels, speech intelligibility decreased from 100% to approximately 15% and RT of both secondary tasks increased monotonically. The RT was the longest in the lowest intelligibility (2-channel) condition. It is unclear why in Pals et al (2013) the RT trend across channel number did not show a peaked shape as the currently study. Possible explanations, which include the difference in speech signal degradation (vocoded speech vs. speech in noise) and secondary task, should be explored in future work.

It was observed that the shape of individual RT curve varied considerably across participants ( Figure 5 ). This variation can be regarded as a limitation of the study because the study result (i.e., the peak-shaped psychometric function) does not hold for all individuals. This individual difference, however, may reflect how people cope with adverse listening conditions. In particular, individuals who have peak-shaped RT curves may be more likely to experience cognitive overload and/or give up listening than those who have reversed-sigmoidal RT curves. Therefore, the former listeners may tend to use maladaptive strategies (e.g., pretending to understand the conversation) to avoid unpleasant situations, while the latter listeners may be more likely to use adaptive strategies (e.g., asking the talker to repeat) to improve communication ( Demorest & Erdman 1987 ). More research is needed to investigate these speculations.

Acknowledgments

Experiments 1 and 2 were supported by the New Century Scholars Research Grant from American Speech-Language-Hearing Foundation. Experiment 3 was supported by NIH/NIDCD R03DC012551 and R01-DC012769. The pilot study of this project was supported by a research grant from Siemens Hearing Instruments. The authors also thank Drs. Bob McMurray and Jacob Oleson for their valuable comments and suggestions on an early version of this paper.

Portions of this paper were presented at the annual meeting of the American Auditory Society, March, 2013, Scottsdale, Arizona, USA, and the annual ASHA Convention, November, 2015, Denver, Colorado, USA.

Conflicts of Interest and Source of Funding:

Yu-Hsiang Wu is currently receiving grants from NIH and the National Institute on Disability and Rehabilitation Research. For the remaining authors, none were declared. Experiments 1 and 2 were supported by the New Century Scholars Research Grant from American Speech-Language-Hearing Foundation. Experiment 3 was supported by NIH/NIDCD R03DC012551 and R01-DC012769. The pilot study of this project was supported by a research grant from Siemens Hearing Instruments.

• ANSI. Specification for Audiometers (ANSI S3.6) New York: American national standards institute; 2010. [ Google Scholar ]
• Aston-Jones G, Cohen JD. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci. 2005; 28 :403–450. [ PubMed ] [ Google Scholar ]
• Cabestrero R, Crespo A, Quirós P. Pupillary dilation as an index of task demands. Percept Motor Skill. 2009; 109 :664–678. [ PubMed ] [ Google Scholar ]
• Degeest S, Keppler H, Corthals P. The effect of age on listening effort. J Speech Lang Hear Res. 2015; 58 :1592–1600. [ PubMed ] [ Google Scholar ]
• Demorest ME, Erdman SA. Development of the communication profile for the hearing impaired. J Speech Hear Disord. 1987; 52 :129–143. [ PubMed ] [ Google Scholar ]
• Desjardins JL, Doherty KA. Age-related changes in listening effort for various types of masker noises. Ear Hear. 2013; 34 :261–272. [ PubMed ] [ Google Scholar ]
• Desjardins JL, Doherty KA. The effect of hearing aid noise reduction on listening effort in hearing-impaired adults. Ear Hear. 2014; 35 :600–610. [ PubMed ] [ Google Scholar ]
• Downs DW. Effects of hearing and use on speech discrimination and listening effort. J Speech Hear Disord. 1982; 47 :189–193. [ PubMed ] [ Google Scholar ]
• Fraser S, Gagné JP, Alepins M, et al. Evaluating the effort expended to understand speech in noise using a dual-task paradigm: the effects of providing visual speech cues. J Speech Lang Hear Res. 2010; 53 :18–33. [ PubMed ] [ Google Scholar ]
• Gatehouse S, Gordon J. Response times to speech stimuli as measures of benefit from amplification. Br J Audiol. 1990; 24 :63–68. [ PubMed ] [ Google Scholar ]
• Gosselin PA, Gagné JP. Use of a dual-task paradigm to measure listening effort. Canadian J Speech Lang Patho Audiol. 2010; 34 :43–51. [ Google Scholar ]
• Gosselin PA, Gagné JP. Older adults expend more listening effort than young adults recognizing speech in noise. J Speech Lang Hear Res. 2011; 54 :944–958. [ PubMed ] [ Google Scholar ]
• Granholm E, Asarnow RF, Sarkin AJ, et al. Pupillary responses index cognitive resource limitations. Psychophysiology. 1996; 33 :457–461. [ PubMed ] [ Google Scholar ]
• Hällgren M, Larsby B, Lyxell B, et al. Speech understanding in quiet and noise, with and without hearing aids. Int J Audiol. 2005; 44 :574–583. [ PubMed ] [ Google Scholar ]
• Hart SG, Staveland LE. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Hancock PA, Meshkati N, editors. Human Mental Workload. Amsterdam, The Netherlands: North Holland Press; 1988. pp. 139–183. [ Google Scholar ]
• Hazeltine E, Ruthruff E, Remington RW. The role of input and output modality pairings in dual-task performance: Evidence for content-dependent central interference. Cognitive Psychol. 2006; 52 :291–345. [ PubMed ] [ Google Scholar ]
• Hick CB, Tharpe AM. Listening effort and fatigue in school-age children with and without hearing loss. J Speech Lang Hear Res. 2002; 45 :573–584. [ PubMed ] [ Google Scholar ]
• Hornsby BWY. The effects of hearing aid use on listening effort and mental fatigue associated with sustained speech processing demands. Ear Hear. 2013; 34 :523–534. [ PubMed ] [ Google Scholar ]
• Humes LE. The contributions of audibility and cognitive factors to the benefit provided by amplified speech to older adults. J Am Acad Audiol. 2007; 18 :590–603. [ PubMed ] [ Google Scholar ]
• Kahneman D. Attention and Effort. Englewood Cliffs, NJ: Prentice-Hall; 1973. [ Google Scholar ]
• Keidser G, Dillon HR, Flax M, et al. The NAL-NL2 prescription procedure. Audiol Res. 2011; 1 :88–90. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kiessling J, Pichora-Fuller MK, Gatehouse S, et al. Candidature for and delivery of audiological services: special needs of older people. Int J Audiol. 2003; 42 :2, S92–101. [ PubMed ] [ Google Scholar ]
• McGarrigle R, Munro KJ, Dawes P, et al. Listening effort and fatigue: What exactly are we measuring? A British Society of Audiology Cognition in Hearing Special Interest Group ‘white paper’ Int J Audiol. 2014; 53 :433–445. [ PubMed ] [ Google Scholar ]
• Nilsson M, Soli SD, Sullivan JA. Development of the Hearing in Noise Test for the measurement of speech reception thresholds in quiet and in noise. J Acoust Soc Am. 1994; 95 :1085–1099. [ PubMed ] [ Google Scholar ]
• Pals C, Sarampalis A, Başkent D. Listening effort with cochlear implant simulations. J Speech Lang Hear Res. 2013; 56 :1075–1084. [ PubMed ] [ Google Scholar ]
• Peavler WS. Individual differences in pupil size and performance. In: Janisse MP, editor. Pupillary dynamics and behavior. Springer; 1974. pp. 159–175. [ Google Scholar ]
• Petersen EB, Wöstmann M, Obleser J, et al. Hearing loss impacts neural alpha oscillations under adverse listening conditions. Front Psychol. 2015; 6 :1–11. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Pichora-Fuller MK, Singh G. Effects of age on auditory and cognitive processing: implications for hearing aid fitting and audiologic rehabilitation. Trends Amplif. 2006; 10 :29–59. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Picou EM, Ricketts TA. The effect of changing the secondary task in dual-task paradigms for measuring listening effort. Ear Hear. 2014; 35 :611–622. [ PubMed ] [ Google Scholar ]
• Picou EM, Ricketts TA, Hornsby BW. How hearing aids, background noise, and visual cues influence objective listening effort. Ear Hear. 2013; 34 :e52–e64. [ PubMed ] [ Google Scholar ]
• Poock GK. Information processing vs pupil diameter. Percept Motor Skill. 1973; 37 :1000–1002. [ PubMed ] [ Google Scholar ]
• Rönnberg J. Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: A framework and a model. Int J Audiol. 2003; 42 :S68–S76. [ PubMed ] [ Google Scholar ]
• Rönnberg J, Lunner T, Zekveld AA, et al. The Ease of Language Understanding (ELU) model: theoretical, empirical, and clinical advances. Front Syst Neurosci. 2013; 7 :1–17. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Rönnberg J, Rudner M, Foo C, et al. Cognition counts: A working memory system for ease of language understanding (ELU) Int J Audiol. 2008; 47 :S99–S105. [ PubMed ] [ Google Scholar ]
• Sander MC, Werkle-Bergner M, Lindenberger U. Amplitude modulations and inter-trial phase stability of alpha-oscillations differentially reflect working memory constraints across the lifespan. Neuroimage. 2012; 59 :646–654. [ PubMed ] [ Google Scholar ]
• Sarampalis A, Kalluri S, Edwards B, et al. Objective measures of listening effort: effects of background noise and noise reduction. J Speech Lang Hear Res. 2009; 52 :1230–1240. [ PubMed ] [ Google Scholar ]
• Stroop J. Studies of interference in serial verbal reactions. J Exp Psychol. 1935; 18 :643–662. [ Google Scholar ]
• Wickens CD. Multiple resources and mental workload. Hum Factors. 2008; 50 :449–455. [ PubMed ] [ Google Scholar ]
• Winn MB, Edwards JR, Litovsky RY. The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation. Ear Hear. 2015; 36 :e153–e165. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Worrall LE, Hickson L. Communication disability in aging from prevention to intervention. New York: Thomson, Delmar Learning; 2003. [ Google Scholar ]
• Wu YH, Aksan N, Rizzo M, et al. Measuring listening effort: driving simulator versus simple dual-task paradigm. Ear Hear. 2014; 35 :623–632. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Zekveld AA, Kramer SE. Cognitive processing load across a wide range of listening conditions: Insights from pupillometry. Psychophysiology. 2014; 51 :277–284. [ PubMed ] [ Google Scholar ]
• Zekveld AA, Kramer SE, Festen JM. Pupil response as an indication of effortful listening: the influence of sentence intelligibility. Ear Hear. 2010; 31 :480–490. [ PubMed ] [ Google Scholar ]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 02 March 2014

Neural mechanisms of dual-task interference and cognitive capacity limitation in the prefrontal cortex

• Kei Watanabe 1 , 2   nAff3 &
• Shintaro Funahashi 2  

Nature Neuroscience volume  17 ,  pages 601–611 ( 2014 ) Cite this article

11k Accesses

135 Citations

19 Altmetric

Metrics details

• Cognitive control
• Working memory

Simultaneous performance of two tasks often leads to performance deficits in the component tasks. This effect, known as dual-task interference, is thought to be a proof of capacity limitation in cognition, and the lateral prefrontal cortex (LPFC) has been highlighted as its putative neural substrate. Here we recorded single-neuron activities in LPFC while monkeys performed dual tasks that required the simultaneous performance of a varying-load spatial attention task and a spatial memory task. We found that the performance of the monkeys exhibited dual-task interference, and prefrontal neuron activities showed a decreased ability to represent task-relevant information to a degree proportional to the increased demand of the concurrent counterpart task. The locus of the interference was shown to originate in the simultaneous, overloaded recruitment of the same LPFC neural population by the two tasks. These results provide direct neurophysiological evidence for, and constraints to, psychological models of dual-task interference and capacity limitation.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 12 print issues and online access

195,33 € per year

only 16,28 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Cross-hemispheric recruitment during action planning with increasing task demand

Shared mechanisms underlie the control of working memory and attention

Characterising the unity and diversity of executive functions in a within-subject fMRI study

Duncan, J. The demonstration of capacity limitation. Cognit. Psychol. 12 , 75–96 (1980).

Google Scholar  

Redelmeier, D.A. & Tibshirani, R.J. Association between cellular-telephone calls and motor vehicle collisions. N. Engl. J. Med. 336 , 453–458 (1997).

Strayer, D.L. & Johnston, W.A. Driven to distraction: dual-task studies of simulated driving and conversing on a cellular telephone. Psychol. Sci. 12 , 462–466 (2001).

Pashler, H. Dual-task interference in simple tasks: data and theory. Psychol. Bull. 116 , 220–244 (1994).

Marois, R. & Ivanoff, J. Capacity limits of information processing in the brain. Trends Cogn. Sci. 9 , 296–305 (2005).

Baddeley, A. & Hitch, G.J. Working memory. in The Psychology of Learning and Motivation Vol. 8 (ed. Bower, G.H.) 47–89 (Academic Press, New York, 1974).

Telford, C.W. The refractory phase of voluntary and associative responses. J. Exp. Psychol. 14 , 1–36 (1931).

Welford, A. The 'psychological refractory period' and the timing of high speed performance–a review and theory. Br. J. Psychol. Gen. Sect. 43 , 2–19 (1952).

Kahneman, D. Attention and Effort (Prentice Hall, Englewood Cliffs, New Jersey, 1973).

Wickens, C.D. The structure of attentional resources. in Attention and Performance VIII (ed. Nickerson, R.S.) 239–257 (Lawrence Erlbaum, Hillsdale, New Jersey, 1980).

Wickens, C.D. Multiple resources and performance prediction. Theor. Issues Ergon. 3 , 159–177 (2002).

Klingberg, T. Concurrent performance of two working memory tasks: potential mechanisms of interference. Cereb. Cortex 8 , 593–601 (1998).

Adcock, R.A., Constable, R.T., Gore, J.C. & Goldman-Rakic, P.S. Functional neuroanatomy of executive processes involved in dual-task performance. Proc. Natl. Acad. Sci. USA 97 , 3567–3572 (2000).

Bunge, S.A., Klingberg, T., Jacobsen, R.B. & Gabrieli, J.D.E. A resource model of the neural basis of executive working memory. Proc. Natl. Acad. Sci. USA 97 , 3573–3578 (2000).

Passingham, R.E. Attention to action. Phil. Trans. R. Soc. Lond. B 351 , 1473–1479 (1996).

Klingberg, T. & Roland, P.E. Interference between two concurrent tasks is associated with activation of overlapping fields in the cortex. Brain Res. Cogn. Brain Res. 6 , 1–8 (1997).

Baddeley, A., Della Sala, S., Papagno, C. & Spinnler, H. Dual-task performance in dysexecutive and nondysexecutive patients with a frontal lesion. Neuropsychology 11 , 187–194 (1997).

D'Esposito, M. et al. The neural basis of the central executive system of working memory. Nature 378 , 279–281 (1995).

Jiang, Y., Saxe, R. & Kanwisher, N. Functional magnetic resonance imaging provides new constraints on the theories of the psychological refractory period. Psychol. Sci. 15 , 390–396 (2004).

Dux, P.E., Ivanoff, J., Asplund, C.L. & Marois, R. Isolation of a central bottleneck of information processing with time-resolved fMRI. Neuron 52 , 1109–1120 (2006).

Funahashi, S., Bruce, C.J. & Goldman-Rakic, P.S. Dorsolateral prefrontal lesions and oculomotor delayed response performance: evidence for mnemonic “scotomas.”. J. Neurosci. 13 , 1479–1497 (1993).

Rossi, A.F., Bichot, N.P., Desimone, R. & Ungerleider, L.G. Top down attentional deficits in macaques with lesions of lateral prefrontal cortex. J. Neurosci. 27 , 11306–11314 (2007).

Zanto, T.P., Rubens, M.T., Thangavel, A. & Gazzaley, A. Causal role of the prefrontal cortex in top-down modulation of visual processing and working memory. Nat. Neurosci. 14 , 656–661 (2011).

Funahashi, S., Bruce, C.J. & Goldman-Rakic, P.S. Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex. J. Neurophysiol. 61 , 331–349 (1989).

Constantinidis, C., Franowicz, M.N. & Goldman-Rakic, P.S. The sensory nature of mnemonic representation in the primate prefrontal cortex. Nat. Neurosci. 4 , 311–316 (2001).

Everling, S., Tinsley, C.J., Gaffan, D. & Duncan, J. Filtering of neural signals by focused attention in the monkey prefrontal cortex. Nat. Neurosci. 5 , 671–676 (2002).

Lebedev, M.A., Messinger, A., Kralik, J.D. & Wise, S.P. Representation of attended versus remembered locations in prefrontal cortex. PLoS Biol. 2 , e365 (2004).

Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18 , 193–222 (1995).

Knudsen, E.I. Fundamental components of attention. Annu. Rev. Neurosci. 30 , 57–78 (2007).

Fougnie, D. & Marois, R. Distinct capacity limits for attention and working memory—evidence from attentive tracking and visual working memory. Psychol. Sci. 17 , 526–534 (2006).

Franconeri, S.L., Alvarez, G.A. & Cavanagh, P. Flexible cognitive resources: competitive content maps for attention and memory. Trends Cogn. Sci. 17 , 134–141 (2013).

Rao, S.G., Williams, G.V. & Goldman-Rakic, P.S. Destruction and creation of spatial tuning by disinhibition: GABAA blockade of prefrontal cortical neurons engaged by working memory. J. Neurosci. 20 , 485–494 (2000).

Wallis, J.D., Anderson, K.C. & Miller, E.K. Single neurons in prefrontal cortex encode abstract rules. Nature 411 , 953–956 (2001).

Fuster, J.M. & Alexander, G.E. Neuron activity related to short-term memory. Science 173 , 652–654 (1971).

Kubota, K. & Niki, H. Prefrontal cortical unit activity and delayed alternation performance in monkeys. J. Neurophysiol. 34 , 337–347 (1971).

Goldman-Rakic, P.S. Cellular basis of working memory. Neuron 14 , 477–485 (1995).

Curtis, C.E. & D'Esposito, M. Persistent activity in the prefrontal cortex during working memory. Trends Cogn. Sci. 7 , 415–423 (2003).

Wang, X.J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24 , 455–463 (2001).

Meyers, E.M., Freedman, D.J., Kreiman, G., Miller, E.K. & Poggio, T. Dynamic population coding of category information in inferior temporal and prefrontal cortex. J. Neurophysiol. 100 , 1407–1419 (2008).

Stokes, M.G. et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 78 , 364–375 (2013).

Zucker, R.S. & Regehr, W.G. Short-term synaptic plasticity. Annu. Rev. Physiol. 64 , 355–405 (2002).

Mongillo, G., Barak, O. & Tsodyks, M. Synaptic theory of working memory. Science 319 , 1543–1546 (2008).

Katsuki, F. & Constantinidis, C. Unique and shared roles of the posterior parietal and dorsolateral prefrontal cortex in cognitive functions. Front. Integr. Neurosci. 6 , 17 (2012).

Suzuki, M. & Gottlieb, J. Distinct neural mechanisms of distractor suppression in the frontal and parietal lobe. Nat. Neurosci. 16 , 98–104 (2013).

Rissman, J., Gazzaley, A. & D'Esposito, M. The effect of non-visual working memory load on top-down modulation of visual processing. Neuropsychologia 47 , 1637–1646 (2009).

Bourke, P.A., Duncan, J. & Nimmo-Smith, I. A general factor involved in dual-task performance decrement. Q. J. Exp. Psychol. A 49A , 525–545 (1996).

Watanabe, K., Igaki, S. & Funahashi, S. Contribution of prefrontal cue-, delay-, and response-period activity to the decision process of saccade direction in a free-choice ODR task. Neural Netw. 19 , 1203–1222 (2006).

Robinson, D.A. A method of measuring eye movement using a scleral search coil in a magnetic field. IEEE Trans. Biomed. Eng. 10 , 137–145 (1963).

Bruce, C.J., Goldberg, M.E., Bushnell, M.C. & Stanton, G.B. Primate frontal eye fields. II. Physiological anatomical correlates of electrically evoked eye movements. J. Neurophysiol. 54 , 714–734 (1985).

Page, E.B. Ordered hypotheses for multiple treatments: a significance test for linear ranks. J. Am. Stat. Assoc. 58 , 216–230 (1963).

Kutner, M.H., Nachtsheim, C.J., Neter, J. & Li, W. Applied Linear Statistical Models (McGraw-Hill, New York, 2005).

Buschman, T.J., Siegel, M., Roy, J.E. & Miller, E.K. Neural substrates of cognitive capacity limitations. Proc. Natl. Acad. Sci. USA 108 , 11252–11255 (2011).

Kline, R.B. Beyond Significance Testing: Reforming Data Analysis Methods in Behavioral Research (American Psychological Association, Washington DC, 2004).

Download references

Acknowledgements

We thank M. Buckley, J. Duncan, M. Kusunoki and M. Stokes for their comments on the manuscript and R. Akaishi, K. Mochizuki and A. Tanaka for their helpful discussions. This work was supported by Grant-in-Aids for Scientific Research (21240024 and 25240021) from the Ministry of Education, Culture, Sports, Science and Technology (MEXT), Japan to S.F. and by Research Fellowships for Young Scientists from the Japan Society for the Promotion of Science to K.W. (20-8015). Two monkeys used for this experiment were supplied from the National Bioresource Project (Japanese Monkeys) supported by MEXT.

Author information

• Kei Watanabe

Present address: Present address: Department of Experimental Psychology, University of Oxford, Oxford, UK.,

Authors and Affiliations

Japan Society for the Promotion of Science (JSPS), Tokyo, Japan

Kokoro Research Center, Kyoto University, Kyoto, Japan

Kei Watanabe & Shintaro Funahashi

You can also search for this author in PubMed   Google Scholar

Contributions

K.W. designed the experiment, collected and analyzed the data and wrote the manuscript. S.F. designed the experiment, supervised all aspects of the project and wrote the manuscript.

Corresponding author

Correspondence to Shintaro Funahashi .

Ethics declarations

Competing interests.

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary figure 1 event sequence of example trials..

( a ) Example trials in the standard dual-task. The upper row depicts an example dual-task trial in which the attention task is performed as a short trial. The bottom row shows a trial in which the attention task is performed as a long trial. Note that, in the long trial, there were two possible patterns in the temporal order of catch change and memory cue presentation. ( b ) Same as in panel ( a ), but for the easy dual-task. Note that, in the long trial (bottom row), catch change was scheduled and executed as an ‘empty event’. ( c ) Example trials in the single memory task (SMT). The time course of the task was matched with that in the standard and easy dual-tasks. However, while all attention task events were scheduled, they were executed as ‘empty events’. Trials were automatically initiated by the appearance of FR after an intertrial interval (4.0–7.0 s).

Supplementary Figure 2 Additional evidence supporting the presence of dual-task interference effect in the DMT conditions.

( a ) Mean percent correct rates in the SMT and four DMT conditions plotted separately for the trials with short (< 2.0 s), medium (2.0–4.0 s) and long (> 4.0 s) memory delay periods. In monkey S (left), a two-way mixed-design ANOVA showed significant main effects of both Task condition and Delay length ( P < 10 –4 ), and a nonsignificant interaction effect ( P = 0.27). In monkey A (right), there were significant main effects of both Task condition and Delay length ( P < 10 –4 ), and an interaction effect ( P = 0.02). Asterisks indicate the result of the simple effect ANOVA for the factor Delay length. ( b ) Time course of FB error rates relative to memory cue onset averaged across all sessions for monkeys S (left) and A (right). Inset bar graphs indicate the mean FB error rate during the 1-s period following memory cue onset. Error bars indicate s.e.m. During this period, monkey S showed a significant increase in the FB error rate in DMT-Up and DMT-Down compared with SMT ( post-hoc Steel-Dwass test, P < 10 –4 ; omnibus Kruskal-Wallis test, P < 10 –4 ), indicating that the oculomotor aspect of memory task performance was substantially interfered with by the concurrent attention task. ( c , d ) Trajectories and end points of FB eye movements that occurred during the 1-s period following memory cue onset in monkeys S ( c ) and A ( d ). Three unfilled black circles indicate the ring stimuli for the attention task. Colors are assigned to each memory cue location (square) and each FB eye movement trajectory so that the color of a given FB eye movement trajectory indicates the location of the memory cue presentation that preceded this FB error. End points of FB eye movements are shown as black dots. Numbers shown on each colored square indicate the cumulative number of FB errors across sessions in each memory cue location. The total number of trials (N) in which the memory cue was presented is shown at the bottom of each panel. In the DMT-Up and DMT-Down conditions in monkey S, regardless of memory cue location, FB eye movements after memory cue onset were predominantly directed toward the attention target ring, rather than the memory cue location as observed in SMT. However, importantly, at the time of memory cue onset, the attention cue had been removed from the monitor for 1.6–5.1 s, and the monkeys were simply viewing the still images of three rings. This indicates that around the time of memory cue presentation, information regarding the position of the target ring for the attention task was occupying monkey S’s processing capacity, suggesting that the level of readiness for memory cue encoding was severely disrupted. Monkey A’s FB errors were characterized by short eye movements clustered on the vertical axis, regardless of the memory task conditions. The similarity in FB eye movement trajectories between the SMT and DMT conditions indicates that in all of the four DMT conditions, the preparatory state for memory cue encoding was not disrupted by the concurrent attention task. Thus, we concluded that, for monkey A, dual-task interference on the oculomotor aspect memory task performance was minimal.

Supplementary Figure 3 Behavioral performance in the modified standard dual-task.

( a ) Schematic diagram of the event sequence for non-cued trials of the attention task that were randomly inserted among the Up, Down, FR std conditions (cued trials). A catch change was scheduled but executed as an ‘empty event’ without actual changes in the display items. ( b ) Distribution of the session-by-session percent correct rates in the attention task in the modified standard dual-task (consecutive 25 sessions in each monkey) in monkeys S (left) and A (right). Only data from cued trials are shown. The results in FR easy ( Fig. 2a ) are also shown in the rightmost box plot. The statistical testing procedure and conventions were the same as in Fig. 2a . The results in FR easy ( Fig. 2b ) are also shown in the rightmost box plot. ( c ) Distribution of the session-by-session median RTs in the attention task for monkeys S (left) and A (right). ( d ) Distribution of the session-by-session percent correct rates in the three DMT conditions for monkeys S (left) and A (right). The results in SMT ( Fig. 2c ) are also shown in the rightmost box plot. The dotted line indicates the mean percent correct rates after “corrected-for-guessing” transformation. P -values were adjusted for three multiple comparisons between the SMT and three DMT conditions. ( e ) Comparison of the session-by-session percent correct rates between cued (C) and non-cued (NC) trials in the attention task. In monkey S, the percent correct rates in cued trials were significantly higher than those in non-cued trials (two-way repeated-measures ANOVA: main effect of Cueing, P = 2 × 10 –4 ; Attention condition, P < 10 –4 ; interaction, P < 10 –4 ) (simple main effect of Cueing: Up, P < 10 –4 ; FR std , P = 0.006; both C > NC). ( f ) Comparison of the session-by-session median lever-release RTs between cued (C) and non-cued (NC) trials in the attention task. In both monkeys, the RTs in cued trials were significantly shorter than those in non-cued trials (monkey S: main effect of Cueing, P = 5 × 10 –4 ; Attention condition, P < 10 –4 ; interaction, P = 0.06; monkey A: main effect of Cueing, P < 10 –4 ; Attention condition, P = 0.17; interaction, P = 0.09) (simple main effect of Cueing: P < 0.009 for the Up and Down conditions in monkey S; P < 2×10 –4 for all three conditions in monkey A; all NC > C).

Supplementary Figure 4 Cue-period activity of example neurons.

( a ) Activity of a single neuron (monkey S, right hemi.) recorded in the SMT pre , DMT-Up, DMT-Down, DMT-FR std (standard dual-task), and SMT post conditions. Conventions as in Fig. 4 . ( b ) Same as in panel ( a ), but for a neuron recorded from monkey A (left hemi.) This neuron exhibited significant spatial selectivity in all three DMT conditions in the standard dual-task. However, the strength of cue-period activity in the maximum response location (270°) is significantly attenuated in DMT-Up and DMT-Down compared to that in SMT pre . ( c ) Activity of a single neuron (monkey A, left hemi.) recorded in the SMT pre , DMT-FR easy (easy dual-task), and SMT post conditions. In DMT-FR easy , attenuation of both the magnitude and selectivity of cue-period activity was absent.

Supplementary Figure 5 Delay-period activity of example neurons.

( a ) Activity of a single neuron (monkey S, right hemi.) recorded in the SMT pre , DMT-Up, DMT-Down, DMT-FR std , DMT-FR easy , and SMT post conditions. Conventions as in Fig. 4 . From left to right, seven memory task conditions, including two SMT post blocks (SMT post -1 and SMT post -2), are shown in the order of recording, except for DMT-Up, DMT-Down and DMT-FR std . For this and three other neurons (monkey S) with spatially-selective delay-period activity in SMT pre , activities were obtained in both the standard and easy dual-tasks. ( a ) Activity of a single neuron (monkey A, left hemi.) recorded in SMT pre , DMT-Up, DMT-Down, DMT-FR std , and SMT post . Note that in DMT-Down, the activity level was elevated in all memory cue locations because the attention cue had been presented near the neuron’s maximum response location (315°). Nevertheless, spatial selectivity of delay-period activity was lost in this condition ( P = 0.75). ( b ) Activity of a single neuron (monkey A, left hemi.) recorded in SMT pre , DMT-FR easy , and SMT post . This neuron exhibited delay-period activity similar to that in panel ( b ). However, attenuation of both the magnitude and selectivity of delay-period activity was absent in DMT-FR easy .

Supplementary Figure 6 Population cue-period activities of individual monkeys.

( a , b ) Data are for monkeys S ( a ) and A ( b ). Upper row: population cue-period activity in the maximum (blue line) and minimum (red line) response locations across the six memory task conditions. In both monkeys, significant interaction effects [Task condition × Cue location] were observed among cue-period activity (monkey S: main effect of Task condition, F 5,187 = 10.85, P < 10 –4 ; Cue location, F 1,187 = 114.88, P < 10 –4 ; interaction, F 5,187 = 10.52, P < 10 –4 ; monkey A : main effect of Task condition, F 5,185 = 0.54, P = 0.76; Cue location, F 1,185 = 154.90, P < 10 –4 ; interaction, F 5,185 = 3.57, P = 0.004; two-way mixed design ANOVA). Bottom row: population spatial tuning during the cue-period. Conventions as in Fig. 5 . Although, in both monkeys, cue-period activity exhibited a significant attenuation of spatial selectivity under DMT, the degree of attenuation was smaller in monkey A, whose cue-period activity exhibited a robust increase following memory cue onset. This suggests that, in monkey A, information processing of the memory task was rather unaffected in the initial phase (time period immediately following memory cue presentation), whereas in monkey S, processing of the memory task was considerably disrupted from this phase. In agreement with this notion, comparison of the behavioral performance between the two monkeys showed that monkey A exhibited a more moderate dual-task interference on memory task performance than monkey S ( Fig. 2c ). This was particularly evident in the trials that had short memory delay period (< 2.0 s) ( Supplementary Fig. 2a ). In addition, monkey A’s oculomotor behavior following memory cue presentation did not show signs of dual-task interference, whereas monkey S’s oculomotor behavior in the same epoch clearly exhibited interference by the attention task, as indicated by a significant increase in FB error rates in DMT-Up and DMT-Down relative to SMT ( Supplementary Fig. 2b–d ).

Supplementary Figure 7 Population delay-period activities of individual monkeys.

( a, b ) Data are for monkeys S ( a ) and A ( a ). Upper row: population delay-period activity in the maximum (blue line) and minimum (red line) response locations across the six memory task conditions. In both monkeys, significant interaction effects [Task condition × Cue location] were observed among delay-period activity (monkey S: main effect of Task condition, F 5,126 = 4.10, P = 0.002; Cue location, F 1,126 = 66.05, P < 10 –4 ; interaction, F 5,126 = 5.13, P = 3 × 10 –4 ; monkey A : main effect of Task condition, F 5,165 = 0.64, P = 0.67; Cue location, F 1,165 = 65.64, P < 10 –4 ; interaction, F 5,165 = 3.77, P = 0.003). Bottom row: population spatial tuning during the delay-period. Conventions as in Fig. 5 . In contrast to the cue-period activity, the degree of selectivity attenuation among delay-period activity was comparable between the two monkeys in the DMT conditions, suggesting that, in both monkeys, memory task processing was substantially disrupted in the later stage by the presence of the concurrent attention task. In accordance with this notion, behavioral results showed that, in both monkeys, prominent dual-task interference was observed in the trials that had long memory delay-period (> 4.0 s) ( Supplementary Fig. 2a ). The close correspondence between the individual variability among behavioral performance and that among response patterns of cue- and delay-period activities further supports the notion that the attenuation of neuronal selectivity for the memory cue location under DMT is a direct neural correlate of the behavioral cost of dual-task performance.

Supplementary Figure 8 Comparison of cue- and delay-period activities in SMT between neurons assigned to the recording in the standard dual-task and the easy dual-task.

(a,b) Upper row: population cue-period activities in the SMT pre ( a ) and SMT post ( b ) conditions for neurons assigned to the recording in the standard dual-task (left) and the easy dual-task (right). Conventions are the same as in Fig. 5 . In both conditions, the activity patterns were highly similar between the groups of neurons assigned to the standard dual-task (left) and the easy dual-task (right) (SMT pre : main effect of Task assignment, F 1,96 = 0.83, P = 0.36; Cue location, F 1,96 = 170.30, P < 10 –4 ; interaction, F 1,96 = 0.06, P = 0.81; SMT post : main effect of Task assignment, F 1,77 = 0.40, P = 0.53; Cue location, F 1,77 = 67.03, P < 10 –4 ; interaction, F 1,77 = 0.07, P = 0.79; two-way mixed-design ANOVA).Bottom row: population spatial tuning during the cue-period in SMT pre ( a ) and SMT post ( b ). In both SMT pre and SMT post , the tuning slopes and intercepts did not differ between the assigned task (SMT pre : slope, P = 0.92; intercept, P = 0.44; SMT post : slope, P = 0.56; intercept, P = 0.60). ( c , d ) Same as in ( a ) and ( b ), but for delay-period activity (SMT pre : main effect of Task assignment, F 1,73 = 0.16, P = 0.69; Cue location, F 1,73 = 92.18, P < 10 –4 ; interaction, F 1,73 = 0.28, P = 0.60; SMT post : main effect of Task assignment, F 1,56 = 0.78, P = 0.38; Cue location, F 1,56 = 43.21, P < 10 –4 ; interaction, F 1,56 = 1.54, P = 0.22). The tuning slopes and intercepts did not differ between the assigned task (SMT pre : slope, P = 0.65; intercept, P = 0.26; SMT post : slope, P = 0.49; intercept, P = 0.10).

Supplementary Figure 9 Comparison of single-neuron PEV values between the SMT and DMT conditions.

( a ) Upper row: scatter diagrams comparing PEV values of cue-period activity in SMT pre to those in the four DMT and SMT post conditions. Integration time window for PEV calculation was 0.4 s (0.1–0.5 s from memory cue onset). Blue dashed lines indicate the mean PEV values in SMT pre . Red dashed lines indicate the same neurons’ mean PEV values in the corresponding conditions for comparison. Fractions show the number of neurons that showed a decrease in PEV relative to SMT pre , divided by the number of neurons that exhibited spatially-selective cue-period activity in SMT pre . Bottom row: histograms comparing the distribution of PEV values between SMT pre (blue bars) vs. each of the four DMT and SMT post conditions (inverted red bars). ( b ) Same as in ( a ), but for delay-period activity. Integration time window was 1.0 s (0–1.0 s from memory cue offset). ( c ) Summary of the five paired-comparisons shown in ( a ). Note that n = 91 for SMT pre . The center of a notched bar indicates the median value, edges are CI 68% , and the error bar is the CI 95% of the median (bootstrap method). Open black circles indicate mean values. PEV values for cue-period activity were significantly different across memory task conditions (Kruskal-Wallis test, P = 4 × 10 –4 ), and SMT pre showed a significantly greater PEV value than DMT-Up, DMT-Down, and DMT-FR std ( post-hoc Steel-Dwass test, P < 0.02). All six memory task conditions gave median PEV values significantly larger than zero (one-sample Wilcoxon signed-rank test). ( d ) Summary of the five paired-comparisons in ( b ). Note that n = 71 for SMT pre . PEV values in delay-period activity were significantly different across memory task conditions (Kruskal-Wallis test, P = 0.001). SMT pre showed a significantly greater PEV value than DMT-Up, DMT-Down ( post-hoc Steel-Dwass test, P < 0.03) and a substantially greater PEV value than DMT-FR std ( P = 0.06). All six memory task conditions gave median PEV values significantly larger than zero.

Supplementary Figure 10 Comparison of memory task-related activity between the three-ring and one-ring layout types in the modified single memory task.

( a ) Spatially-selective cue-period activity of a representative neuron that exhibited almost identical activities in the two layout types. Conventions as in Fig. 3 . ( b ) Population activity in the 3-ring (top left) and 1-ring layout types (top right) for 13 spatially-selective cue neurons. A scatter diagram (bottom) shows a comparison of the strength of cue-period activity in the maximum (blue) and minimum (red) response locations that were selected from the five cue locations that were also used in DMT. Dotted lines indicate the mean cue-period activity across the population. The strength of cue-period activity was comparable between the two ring layout types at both the maximum ( P = 0.31) and minimum ( P = 0.19) response locations (Wilcoxon signed-rank test). ( c ) Same as in panel ( b ), but for 10 spatially-selective delay neurons. There was no significant difference in the strength of delay-period activity between the two layout types at both the maximum ( P = 0.92) and minimum ( P = 0.43) response locations.

Supplementary Figure 11 Temporal dynamics of neuronal signals representing attention and memory task information in the standard and easy dual-tasks.

( a ) Time course of neuronal signals of an example neuron (the neuron shown in Supplementary Fig. 5b ) representing the location of the attention cue (PEV attention , magenta), the memory cue (PEV memory , blue), and their interaction (PEV interaction , green) in the standard dual-task. Dashed cyan line indicates the same neuron’s PEV memory in SMT pre . Conventions are the same as in Fig. 8a . ( b ) Time course of neuronal signals of an example neuron (the neuron shown in Supplementary Fig. 5c ) representing the location of the memory cue (PEV memory , blue) in the easy dual-task. Dashed cyan line indicates the same neuron’s PEV memory obtained in SMT pre . ( c ) Population-averaged time course of PEV memory in the easy dual-task (solid blue line, n = 24). Shaded areas indicate s.e.m. The same neurons’ population-averaged PEV memory time series in SMT pre are plotted as a solid cyan line. Dashed blue line and dashed cyan line indicate population-averaged PEV memory time series in the standard dual-task and SMT pre , respectively for 51 neurons analyzed in Fig. 8 (the curves are the same as those shown in Fig. 8a ). ( d ) Time course of the proportion of neurons that exhibited significant information ( P < 0.05) about the memory cue location (solid blue line). The same neurons’ results in SMT pre are plotted as a solid cyan line. Dashed blue line and dashed cyan line indicate the results of the same analysis in the standard dual-task and SMT pre , respectively (both n = 51, the curves are the same as in Fig. 8f ). Horizontal dashed lines indicate the proportion expected by chance (5%).

Supplementary Figure 12 Comparison of spatial selectivity between the SMT and DMT conditions.

( a ) Comparison of behavioral performance between the SMT pre sessions with low percent correct rates and the DMT (standard dual-task) sessions with high percent correct rates. To perform this analysis, session-by-session percent correct rates in each memory task condition were rank-ordered and split at the median. The bottom half of SMT pre sessions and the top half of DMT sessions were selected. This analysis included 51 sessions where spatially-selective delay-period activity was recorded in SMT pre . Data from the three DMT conditions in the standard dual-task (DMT-Up, DMT-Down and DMT-FR std ) were collapsed. The sessions from the individual monkey were separately rank-ordered to avoid a biased subsampling from one monkey. The subsampled sessions gave highly similar percent correct rates between SMT pre and DMT (SMT pre : 95.4%, n = 26; DMT: 96.2%, n = 26; Wilcoxon rank-sum test, P = 0.99). Conventions as in Fig. 2c . ( b ) Time course of PEV attention (magenta), PEV memory (blue), and PEV interaction (green) in the standard dual-task for the 26 subsampled sessions. The magnitude of PEV memory during the delay-period ( D ) was significantly attenuated relative to that in SMT pre ( n = 26, dashed cyan line) (Wilcoxon rank-sum test, P = 0.03). Conventions as in Fig. 8 . ( c ) Comparison of PEV memory between the pre-T col change period and the follow-up fixation period in the standard dual-task. Following the conclusion of the attention task events, PEV memory in the standard dual-task exhibited significant reawakening. ( d ) Comparison of PEV memory and PEV attention during the follow-up fixation period in the standard dual-task. The reawakening of PEV memory during the follow-up fixation period coincided with the reprioritization of task processing between the attention and memory tasks. ( e ) Normalized population-averaged delay-period activity (grey shaded area) in the maximum (blue line) and minimum (red line) response locations in SMT pre and the three DMT conditions in the standard dual-task for the subsampled sessions. For comparing delay-period activity across the four conditions, behavioral performance in DMT-Up, DMT-Down and DMT-FR std were rank-ordered separately. The subsampled sessions gave similar percent correct rates across the four conditions ( P = 0.14, n = 26 for each of the four conditions,). For each neuron, firing rate in each 50-ms bin was divided by the peak delay-period firing rate at the maximum response location in the SMT pre condition. Compared with SMT pre , the difference in activity between the maximum and minimum response locations was remarkably attenuated in DMT-Up, DMT-Down and DMT-FR std (main effect of Task condition, F 3,100 = 0.15, P = 0.93; Cue location, F 1,100 = 52.95, P < 10 –4 ; interaction, F 3,100 = 6.25, P = 6 × 10 –4 ; two-way mixed design ANOVA). Conventions as in Fig. 5c . ( f ) Comparison of PEV values of the delay-period activity between the SMT and three DMT conditions. The three DMT conditions in the standard dual-task exhibited attenuation in spatial selectivity relative to SMT pre (Kruskal-Wallis test, P = 0.03). Conventions as in Supplementary Fig. 9d . Similar result was obtained when the rank-order of sessions was done over monkey-collapsed data (dotted line, P = 0.03).

Supplementary information

Supplementary text and figures.

Supplementary Figures 1–12 (PDF 11028 kb)

Source data

Source data to fig. 1, source data to fig. 2, source data to fig. 3, source data to fig. 4, source data to fig. 5, rights and permissions.

Reprints and permissions

About this article

Cite this article.

Watanabe, K., Funahashi, S. Neural mechanisms of dual-task interference and cognitive capacity limitation in the prefrontal cortex. Nat Neurosci 17 , 601–611 (2014). https://doi.org/10.1038/nn.3667

Download citation

Received : 12 September 2013

Accepted : 03 February 2014

Published : 02 March 2014

Issue Date : April 2014

DOI : https://doi.org/10.1038/nn.3667

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Dataset of human-single neuron activity during a sternberg working memory task.

• Michael Kyzar
• Jan Kamiński
• Ueli Rutishauser

Scientific Data (2024)

Change in activity patterns in the prefrontal cortex in different phases during the dual-task walking in older adults

• Chang Yoon Baek
• Hyeong Dong Kim
• Jang Woo Lee

Journal of NeuroEngineering and Rehabilitation (2023)

Knowledge generalization and the costs of multitasking

• Kelly G. Garner
• Paul E. Dux

Nature Reviews Neuroscience (2023)

Cycles of goal silencing and reactivation underlie complex problem-solving in primate frontal and parietal cortex

• Mikiko Kadohisa
• John Duncan

Nature Communications (2023)

Modulation of inhibitory communication coordinates looking and reaching

• Maureen A. Hagan
• Bijan Pesaran

Nature (2022)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Dual-Task Performance in Motor Learning

• Reference work entry
• pp 1048–1050
• Cite this reference work entry

• Sarah A. Fraser 2 &
• Karen Z. H. Li 3  

1241 Accesses

Divided attention

Dual-task performance requires an individual to perform two tasks (i.e., Task A and Task B) simultaneously. Typically this type of performance is contrasted with single-task performance in which the individual only has to perform one task at a time (Task A or B).

Motor learning occurs when an individual demonstrates relatively enduring improvements in their capability to perform a motor task after practice.

Theoretical Background

Motor learning proceeds in stages. Historically, three stages of motor learning were proposed (Fitts and Posner 1967 ). The first stage was named the cognitive stage, the second the associative stage, and the third the autonomous stage. One of the reasons for naming the first stage the cognitive stage is that cognitive processes are highly involved in this stage of learning. In particular, attention to the instructions and to the demands of the motor task to be learned is crucial during this stage of learning. In contrast,...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Durable hardcover edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Fitts, P. M., & Posner, M. I. (1967). Human performance . Belmont: Brooks/Cole.

Google Scholar  

Kahneman, D. (1973). Attention and effort . Englewood Cliffs: Prentice-Hall.

Download references

Author information

Authors and affiliations.

Department of Psychology, Centre de recherche de l’institut universitaire de gériatrie de Montréal, Université du Québec à Montréal, 4565 Queen Mary, Montréal, Québec, H3W 1W5, Canada

Dr. Sarah A. Fraser

Department of Psychology, Center for Research in Human Development, Concordia University, 7141 Sherbrooke Street West, Montréal, Québec, H4B 1R6, Canada

Dr. Karen Z. H. Li

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sarah A. Fraser .

Editor information

Editors and affiliations.

Faculty of Economics and Behavioral Sciences, Department of Education, University of Freiburg, 79085, Freiburg, Germany

Norbert M. Seel

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer Science+Business Media, LLC

About this entry

Cite this entry.

Fraser, S.A., Li, K.Z.H. (2012). Dual-Task Performance in Motor Learning. In: Seel, N.M. (eds) Encyclopedia of the Sciences of Learning. Springer, Boston, MA. https://doi.org/10.1007/978-1-4419-1428-6_1703

Download citation

DOI : https://doi.org/10.1007/978-1-4419-1428-6_1703

Publisher Name : Springer, Boston, MA

Print ISBN : 978-1-4419-1427-9

Online ISBN : 978-1-4419-1428-6

eBook Packages : Humanities, Social Sciences and Law Reference Module Humanities and Social Sciences Reference Module Education

Share this entry

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research