eNeuro

eNeuro. 4(6): ENEURO.0293-16.2017

Back to Pupillometry: How Cortical Network State Fluctuations Tracked by Pupil Dynamics Could Explain Neural Signal Variability in Human Cognitive Neuroscience

http://orcid.org/0000-0003-4162-2298Miriam Schwalm¹², http://orcid.org/0000-0001-5002-3823Eduardo Rosales Jubal¹³

1. Focus Program Translational Neuroscience (FTN) and Institute for Microscopic Anatomy and Neurobiology, Johannes Gutenberg-University Mainz, Mainz D-55128, Germany

2. GRADE Brain, Goethe Graduate Academy and FB 15, Goethe University Frankfurt, Frankfurt D-60438, Germany

3. Instituto de Ciencias Biomédicas, Universidad Autónoma de Chile, Santiago, Chile

The authors declare no competing financial interests.

Author contributions: XXX designed research; XXX performed research; XXX contributed unpublished reagents/analytic tools; XXX analyzed data; M.S. and E.R.J. wrote the paper.

Correspondence should be addressed to Miriam Schwalm, Focus Group Translational Neuroscience (FTN) and Institute for Microscopic Anatomy and Neurobiology, Johannes Gutenberg-University Mainz, Hanns-Dieter-Hüsch-Weg 19, Mainz D-55128, Germany, E-mail: mschwalm@uni-mainz.de.

Copyright © 2017 Schwalm and Rosales Jubal

DOI: 10.1523/ENEURO.0293-16.2017

Published online: 26 December 2017

Visual Abstract

[Image: enu006172491r001] View larger version

Abstract

The mammalian thalamocortical system generates intrinsic activity reflecting different states of excitability, arising from changes in the membrane potentials of underlying neuronal networks. Fluctuations between these states occur spontaneously, regularly, and frequently throughout awake periods and influence stimulus encoding, information processing, and neuronal and behavioral responses. Changes of pupil size have recently been identified as a reliable marker of underlying neuronal membrane potential and thus can encode associated network state changes in rodent cortex. This suggests that pupillometry, a ubiquitous measure of pupil dilation in cognitive neuroscience, could be used as an index for network state fluctuations also for human brain signals. Considering this variable may explain task-independent variance in neuronal and behavioral signals that were previously disregarded as noise.

Significance Statement

The mammalian thalamocortical system generates intrinsic activity reflecting different states of excitability, arising from changes in the membrane potentials of underlying neuronal networks. Fluctuations between these states occur spontaneously, regularly, and frequently throughout awake periods and influence stimulus encoding, information processing, and neuronal and behavioral responses. Changes of pupil size have recently been identified as a reliable marker of underlying neuronal membrane potential and thus can encode associated network state changes in rodent cortex. This suggests that pupillometry, a ubiquitous measure of pupil dilation in cognitive neuroscience, could be used as an index for network state fluctuations also for human brain signals. Considering this variable may explain task-independent variance in neuronal and behavioral signals that were previously disregarded as noise.

Neural networks undergo constant state fluctuations provoked by changes of internally generated activity, even in the absence of external stimulation. These changing patterns of activity vary on slow and rapid time scales and shape the ongoing signal as well as neuronal responses on incoming sensory information.

At the behavioral level, these states can be associated with different levels of arousal and attention, or other nonsensory signals directly related to behavioral performance (Hall et al., 2014; Reimer et al., 2014). For example, alternating states of excitability are associated with changes in global network activity, as between sleep and wakefulness (Steriade et al., 2001; Destexhe et al., 2007), from inattention to vigilance (Boly et al., 2007; McGinley et al., 2015a) or from resting to locomotion (Poulet and Petersen, 2008; Eggermann et al., 2014).

Network states provide a rich experimental variable, which can explain a multitude of neuronal response properties. In rodents, sensory responses have been found to be noticeably shaped by the dominant network activity pattern (Gao et al., 2009; Stroh et al., 2013; Schwalm et al., 2017). Specifically, neuronal responses can be altered in magnitude and signal-to-noise ratio (McGinley et al., 2015a), latency (Lima et al., 2011; Vinck et al., 2015; Schwalm et al., 2017), neuronal variability (Zagha et al., 2013; McGinley et al., 2015a; Schölvinck et al., 2015), and noise correlations (Poulet and Petersen, 2008; Vinck et al., 2015). Studies using macroscopic measures of ongoing signals as resting state functional magnetic resonance imaging (rs-fMRI) measurements usually reveal transient changes in brain activity related to vigilance drifts, complicating the analysis of such data (Schneider et al., 2016). Those changes in the rs-fMRI signal were shown to correspond to fluctuations of intracortical electrophysiological recordings while tracking behaviorally relevant arousal changes in macaques (Chang et al., 2016). In humans, functional connectivity has been linked to global state changes associated with awareness (Godwin et al., 2015) and similarly, fluctuations of spontaneous activity were demonstrated to correlate with connectivity differences in resting state networks (Scheinost et al., 2016). Network state fluctuations thus, have the potential to explain variance in neural and behavioral responses, specifically regarding measures of task performance, response latencies and neuronal gain, and thereby can enhance the reliability and predictability of the brain’s processing of information (McGinley et al., 2015b).

Network states can be electrophysiologically measured and are highly correlated with changes in pupil diameter (Reimer et al., 2014; McGinley et al., 2015a; Vinck et al., 2015; Reimer et al., 2016), since both, network states and pupil diameter, are controlled by the release of acetylcholine (ACh) and noradrenalin (NA; Aston-Jones and Cohen, 2005; Goard and Dan, 2009; Reimer et al., 2016). A recent demonstration that pupillometry reflects fluctuations of spontaneous neuronal activity and correlates, similar to the network state variable, with sensory evoked responses and performance measures (McGinley et al., 2015a) provides crucial evidence for the association between network state and behavioral outcome in the awake rodent brain. This was found to be the case similarly for measures of intracellular calcium of two-photon imaging data in awake mice. Lu et al. (2017) showed that a substantial fraction of neurons exhibited calcium transients which were strongly correlated with the pupil size of the animal. This positive correlation was particularly consistent for vasoactive intestinal peptide interneurons, a specialized cell class potentially serving to facilitate increases in cortical activity (Jackson et al., 2016) and heightened attention (Reimer et al., 2014). Thus, pupil diameter appears to be a reliable index for underlying network state, which would have a direct application for studies where pupillometry is routinely recorded and direct network state readouts remain challenging.

Physiologically, the correlation of pupil diameter with heart rate and galvanic skin reflex (Tursky et al., 1969) indicates its tight coupling to the peripheral nervous system. Pupil diameter is defined by the interaction of two muscles: the iris sphincter, which receives parasympathetic innervation mainly through cholinergic transmission, controls pupil constriction; and the radial muscle of iris, which receives NA releasing sympathetic fibers, commanding pupil dilation. Recently it was shown that pupil dilation and cortical network state fluctuations may be directly linked through their control by cholinergic and adrenergic projections (Reimer et al., 2016). Besides adapting to changes in luminance, spontaneous pupil diameter changes have been shown to be correlated with changes in arousal, attention, and perception (Einhäuser et al., 2008; Gilzenrat et al., 2010; Murphy et al., 2011, 2014b).

In cognitive neuroscience, pupil diameter has additionally been reported to correlate with mental effort in decision-making tasks (Kahneman and Beatty, 1966; Fiedler and Glöckner, 2012; de Gee et al., 2014; Murphy et al., 2014a), as well as with learning dynamics (Nassar et al., 2012; Lavín et al., 2014). Pupil diameter changes on manipulations of task-effort (Fig. 1A; Kahneman and Beatty, 1966), with timing of mental decisions (Fig. 1B; Einhäuser et al., 2010), perceptual changes as in binocular (Einhäuser et al., 2008) or interauricular (Kang and Wheatley 2015) rivalry, as well as with surprise (Preuschoff et al., 2011) or uncertainty in decision-making tasks (Fig. 1C; Lavín et al., 2014; Geng et al., 2015). Although very recently pupillometry has been employed to follow changes in tonic alertness during rs-fMRI experiments (Schneider et al., 2016), pupil dilation as a direct measure for underlying neuronal excitability states has not yet been considered routinely in human cognitive neuroscience.

In this review, we propose that using pupillometry measures as an index for network state fluctuations in humans may be a useful approach to explain unaccounted variance in macroscopic neural and behavioral signals.

Figure 1.View larger versionFigure 1. Pupil diameter in human cognitive neuroscience and as an accurate predictor of rapid variations in parameters related to brain state and arousal in the mouse model. A, In a short-memory task, pupil dilation was modulated by the amount of items under active processing at any time: average pupil diameter (mm) for five subjects during auditory presentation (before “pause” period) and recall (after pause period) of digit strings of varying lengths (three to seven digits). The authors found that the pupil dilates when items are presented and constricts during report. The rate of change of these functions was related to task difficulty. From Kahneman and Beatty (1966). B. Pupil dilation departs from baseline ∼1 s before a volitional button press, peaks at 420 ms after the response, and relaxes back to baseline after ∼2 s, thus revealing the time of decision making. Authors interpreted pupil dilation as a marker of NE release from LC and as evidence for the latter role in consolidation of cognitive decisions. From Einhäuser et al. (2010). C, Pupil dilation displays an anticipatory response to uncertainty levels associated with options in a strategic gambling task (Iowa Gambling Task, IGT) where subjects are asked to maximize their profits by choosing the best drawing strategy from a set of four card decks, each of them delivering gains and losses following a pattern unknown for participants. Greater pupil dilation was observed in conditions with a low probability of incoming negative feedback (NF), as compared to conditions where NF had an enhanced probability to occur. Authors interpreted these results as evidence of pupil dilation signaling LC response to decision making in unfamiliar contexts. From Lavín et al. (2014). All axes in A–C depict pupil dilation (y-axis) versus time (x-axis). D, upper panel, Simultaneous recording of pupil diameter and membrane potential of a cortical neuron in layer 2/3 of the mouse primary visual cortex. Pupil diameter exhibits spontaneous variations in size even in the synchronized state and in the absence of locomotion. Note the strong relationship between slow (2–10 Hz) rhythmic synaptic activity and constriction, and the suppression of this activity with dilation. Lower panel, Comparison of pupil diameter and density of low-frequency (<10 Hz) rhythmic synaptic activity in a layer 5 pyramidal cell in the auditory cortex. Increases in pupil diameter are associated with prominent suppression of low-frequency synaptic activity (desynchronized state). From McGinley et al. (2015b). E, Whole-cell recordings from a layer 5 pyramidal neuron in auditory cortex of an awake mouse while simultaneously monitoring pupil diameter and locomotion. Brief dilations of the pupil (microdilations), are highlighted in gray and are associated with a suppression of the low-frequency activity and a depolarization of this neuron independent of locomotion. From McGinley et al. (2015b).

Different Types of Network States

Alternating states of excitability in the waking brain have first been observed in intracellular and local field potential (LFP) recordings of awake rodents. They show large low-frequency fluctuations during periods of quiet resting (Crochet and Petersen, 2006; Poulet et al., 2012; McGinley et al., 2015a). The initiation of whisking or locomotion suppresses the recurrent, slow (<10 Hz) component of the LFP and increases the power of higher frequency oscillations (Poulet et al., 2012; Eggermann et al., 2014; McGinley et al., 2015a). Based on these observations, a classification distinguishing states within waking periods emerged: synchronized states show bimodal, slow rhythmic network activity, and desynchronized states show unimodal persistent, fast network activity. Since network states are continuous and transient phenomena, rather than discrete, categorical conditions, assigning such a dichotomous classification may appear overly simplistic. Nevertheless, classifying data for these substates of waking could explain neuronal response variability and behaviorally relevant correlates in rodents (McGinley et al., 2015a), highlighting its applicability. While the bimodal, synchronized activity was usually related to resting (but see Hall et al., 2014), inattentive behavior, and slower neuronal responses, the persistent desynchronized activity was demonstrated to occur during locomotion or task engagement, showing faster and temporally more precise neuronal responses (Crochet and Petersen, 2006; Pachitariu et al., 2014; McGinley et al., 2015a; Schwalm et al., 2017). In humans, intracortical electrode recordings in the hippocampus of awake subjects undergoing surgical treatment for refractory epilepsy showed similar results: during resting states, slow ripples appeared coordinated in hippocampal areas, whereas in active states during cognitively demanding tasks, high frequency activity emerged in hippocampus and parahippocampal cortex (Billeke et al., 2017). Macroscopic measures similarly demonstrated fluctuating substates in awake human brain activity, during rest (Scheinost et al., 2016; Custo et al., 2017) or during task engagement (Coon et al., 2016; Godwin et al., 2015).

Pupil Diameter as a Reliable Proxy Marker for Network State

The mentioned network substates, synchronized versus desynchronized activity, can only directly be revealed by invasive readouts reflecting synaptic activity (LFP or subdural electrode array recordings), action potentials (intracellular electrophysiological or calcium recordings), or membrane potentials (whole-cell recordings). The recent introduction of pupil diameter as a reliable peripheral marker for network state fluctuations (Reimer et al., 2014, 2016; McGinley et al., 2015a,b) would allow for classification of network states by means of a noninvasive measure. These studies demonstrated a direct correlation of pupil diameter changes and alterations of intracellular membrane and LFP recordings from cortical and hippocampal neurons in the awake, head-fixed mouse (Fig. 1D,E). Importantly, in these experiments, pupil diameter has proven to be remarkably accurate in following state changes of cortical network activity, even in the absence of locomotion or other types of movement (Fig. 1E; Reimer et al., 2014). Pupil dilation was found to be associated with increases of cortical activation and suppression of slow waves, while pupillary constriction was related to an increase of cortical low-frequency activity. Additionally, microdilations of the pupil were associated with the initiation of brief cortical up states (Fig. 1E). The generality of this effect was emphasized by the finding that pupil diameter was correlated to the rate of hippocampal sharp-wave ripples (McGinley et al., 2015a). Furthermore, in humans, pupil dilation can predict fluctuations of fMRI network structures, a measure for global state change (Shine et al., 2016).

As neuronal activity patterns can be highly conserved across species (Buzsáki et al., 2013; Sanchez-Vives et al., 2017) and the neurophysiology of parasympathetic and sympathetic control of pupil dilation seems to be similar in rodents and humans, it is highly likely that the direct relationship between network state and pupil dilation likewise exists in the human brain.

Physiology of Neuromodulation, Pupil Dilation, and Network State Fluctuations

Neuromodulatory pathways, especially the central cholinergic and noradrenergic pathways, have been shown to be directly involved in shifts of ongoing cortical activity and network responsiveness (Lee and Dan, 2012; Constantinople and Bruno, 2011; Castro-Alamancos and Gulati, 2014; Eggermann et al., 2014; Chen et al., 2015). Importantly, ACh released by the basal forebrain and NA released by the locus coeruleus (LC) are the same neurotransmitters used by the sympathetic and parasympathetic pathways to control pupil diameter (McGinley et al., 2015b). ACh is linked to vigilance and attention and transient prefrontal ACh release can control detection of behaviorally relevant sensory stimuli on multiple time scales (Parikh et al., 2007). Furthermore, the discharge of both basal forebrain cholinergic and LC noradrenergic neurons is increased by attention to external stimuli, arousal, and locomotion (Aston-Jones and Cohen, 2005; Eggermann et al., 2014). The role of the LC has been confirmed in attentional modulation and the regulation of goal-directed versus exploratory behaviors (Usher et al., 1999). LC stimulation can mimic the effects associated with arousal, such as increased amplitudes and precision of sensory evoked responses, as well as increased learning-induced plasticity (Martins and Froemke, 2015; McGinley et al., 2015b) and the discharge of LC neurons is tightly locked to pupil dilation (Aston-Jones and Cohen, 2005; Varazzani et al., 2015).

LC activation is robustly associated with pupil dilation dynamics. Indeed, LC firing reliably anticipates spontaneously occurring, or stimulus-triggered, changes in pupil diameter (Joshi et al., 2016). Likewise, BOLD activity in human LC covaries with pupil dilation (Murphy et al., 2014a). The LC responds to the outcome of task-related decision processes (Clayton et al., 2004), as it also receives synaptic input from the frontal cortex that may be involved in arousal responses to stimuli requiring high-level cognition (Lee and Dan, 2012). The effect of LC firing on cortical state dynamics has been linked to single-trial sensory processing, revealing that the temporal structure of noradrenergic modulation may selectively and dynamically enhance cortical stimulus responses. This suggests a coupling between LC and cortex, that can amplify low-frequency fluctuations and thereby enhance cortical responses to stimuli by tightly timed, phasic LC bursts which account for state-dependent, trial-to-trial variability (Safaai et al., 2015). Recently, the physiologic processes linking network states and pupil variations have been directly linked to activity in noradrenergic and cholinergic projections to the cortex. While long-lasting pupil dilations, such as during locomotion or task engagement, are accompanied by sustained cholinergic activity, brief dilations of the pupil are associated with phasic noradrenergic activity (Reimer et al., 2016).

Network State Fluctuations Underpin Arousal and Performance

The inverted-U relationship between arousal and performance (percentage correct, latency, discriminability) proposed by Yerkes and Dodson (1908) situates optimal behavioral performance at intermediate levels of arousal, with extremes of low and high arousal leading to poor performance, due to disengagement and exhaustion respectively. Performance peaks at intermediate arousal levels, particularly in behavioral tasks involving high-level, prefrontal cortex-dependent cognition. This relationship depends on the activation level of receptors for the aforementioned neuromodulators, which are also involved in pupil diameter changes.

The inverted-U relationship was confirmed between task performance in a go/no-go task and neuronal activity in the mouse auditory cortex (McGinley et al., 2015a). Moreover, this relationship held true for correct behavioral responses, pupil diameter, and evoked neuronal responses. Hit rates and neuronal gain were highest and response latencies shortest at intermediate pupil diameters. In macaques, the ability to perform a delayed saccade to target task was found to be optimal at intermediate pupil diameters (Ebitz et al., 2014). In humans, variations in spontaneous activity have similarly been associated with changes in sensory perception and task performance (Schroeder and Lakatos, 2009) as revealed by fMRI (Boly et al., 2007), electroencephalography (EEG; Hesselmann et al., 2008), and magnetoencephalography (MEG; Baumgarten et al., 2015). Furthermore, Murphy et al. (2011) showed response latency for correct stimulus detection in an auditory oddball task to be shortest, and P3 potential in EEG to be maximal, at medium pupil diameters.

Thus, variations in the arousal of awake human subjects contain an optimal zone for neuronal and behavioral responses which can be determined by tracking cortical state fluctuations either directly, through electrophysiological recordings, or indirectly, by using pupil dilation measurements. Moreover, network states have been shown to influence sensory processing efficiency and response times and might further extend to higher-order effects on learning and memory-related tasks.

Concerning stimulus-response properties, during the desynchronized state, mouse barrel cortex neurons show a lower stimulus detection threshold, higher response fidelity, and shorter response latency (Fazlali et al., 2016). In auditory cortex, noise correlations decrease and neuronal responses to tones become temporally more precise and reliable during the desynchronized state (Pachitariu et al., 2014). This supports the notion of desynchronized activity being required to accurately process sensory information (McGinley et al., 2015b; Kaneko et al., 2017).

Since human behavioral response timing is highly variable from trial to trial, including the timing of neuronal activity as a variable would be helpful to explain this variability. Tracking neuronal population activity across the human cortex demonstrated how variations in the timing of neural activity relate to variations in the timing of behavior in a modified Posner visual-attention cueing task (Posner 1980; Coon et al., 2016). Whereas the authors interpreted their results in terms of oscillatory phase modulation of the information’s transmission speed in cortical networks, and its resulting behavior, they critically noticed a broadband activity surge preceding behavioral responses, which can be interpreted as a change in network state. In rs-fMRI data, it was shown that the human brain alternates between functional states, linking cognitive performance and the dynamic reorganization of the network structure reflected by pupil dilation (Shine et al., 2016). Similar to the data obtained in rodents (Reimer et al., 2014; McGinley et al., 2015a), in this study, faster and more effective cognitive performance (measured as shorter nondecision time during an N-back task) was related to increases in pupil diameter.

Putting these previous findings into perspective regarding underlying neural activity, it seems worthwhile exploring such datasets while employing pupil diameter as a direct measure for network state. As there is already work showing the influence of network state on basic behavioral response probabilities and preliminary evidence for a role in higher cognitive function, we propose to explore the influences of network state indexed by pupil dilation on cognitive and decision-making tasks.

Network States and Pupil Dilation in Decision-Making Tasks

Pupil diameter has been used as a proxy measure for arousal in effortful decision-making tasks (Kahneman and Beatty, 1966; Fiedler and Glöckner, 2012; de Gee et al., 2014; Murphy et al., 2014a). Changes of pupil dilation have been investigated in a financial-choice paradigm showing sustained pupil dilation throughout decision formation (Fiedler and Glöckner, 2012). Task-independent shifts of arousal state, indicated by enhanced pupil diameter, have been related to higher trial-by-trial variability in the rate of evidence accumulation during perceptual decision making (Murphy et al., 2014b). Along the same lines, pupil dilation has been linked to cognitive variables influencing the decision-making process (Gilzenrat et al., 2010; Preuschoff et al., 2011; Cavanagh et al., 2014). Such decision processes involve the calculation of unexpected uncertainty associated with LC-NA activity (Aston-Jones and Cohen, 2005; Yu and Dayan, 2005; Lavín et al., 2014; Geng et al., 2015).

Although it has been proposed that changes in network state interact with biased decision making in the face of uncertainty (de Gee et al., 2014), the relationship between pupil dilation and decision-making processes has previously been studied from the perspective of task-evoked changes in pupil diameter. From this perspective, environmental changes (e.g. stimuli signaling uncertainty) are assumed to trigger a change in LC firing mode which leads to a release of ACh and NA, which in turn induce a change of pupil diameter. Whereas the association between pupil dilation and arousal state has been indirectly assumed in many decision-making studies (Nassar et al., 2012; de Gee et al., 2014; Murphy et al., 2014b), until now a direct link between task-independent brain state fluctuations and behavioral performance and learning in decision making has not been sufficiently investigated. Also, in the absence of environmental stimulation, global changes in brain state can lead to an alteration of LC firing and neuromodulator levels, which are reflected in pupil diameter change but also have their correlate in differential neuronal responses and differential behavioral outcome.

As explained above, pupil dilation has routinely been measured in decision-making studies and was also related to cognitive variables, such as arousal, and to neuromodulatory fluctuations. However, the finding of pupil dilation directly reflecting underlying excitatory states of the neural network opens another dimension of explanatory value. Introducing a network-state related variable and separating behavioral and neural data based on pupil dilation measures would, through post hoc, dedicated analyses, help explain previously unaccounted variance in decision-making studies. Hence, this approach could provide a novel, neurophysiologically defined technique of exploring the underpinnings of everyday choices.

Considering Pupil-Indexed Network State as a Variable in Cognitive Neuroscience

Considering the well-established physiologic evidence outlined above, internal brain dynamics which have previously been shown to underlie neuronal responses may similarly influence high-level cognition, decision-making outcomes and learning. This influence on high-level brain functions has already been explored in rodents and primates (Harris and Thiele, 2011), the timing of LC activity has been shown to track behavioral responses more closely than stimulus presentation (Clayton et al., 2004), and LC firing has an effect on cortical state (Safaai et al., 2015). However, evidence linking pupil-indexed network state to performance and learning in humans has not been reported., Hence, it is reasonable to use physiologic measures of underlying network state fluctuations, such as LC-controlled pupil dilation (Gilzenrat et al., 2010; Murphy et al., 2014b), to predict performance and learning in cognitive tasks in humans.

First steps in the exploration of learning dynamics regulated by pupil-linked arousal systems have already been made: brief changes in pupil dilation predicted the reliability of responses in a predictive inference task (Nassar et al., 2012). These authors also predicted task-independent manipulations of pupil diameter to alter behavior, since higher learning rates were associated with small baseline pupil diameter and lower learning rates were associated with large baseline diameter. This finding is consistent with the hypothesis that larger pupil diameters are associated with desynchronized states of the network and smaller pupil diameter indicate underlying synchronized states (McGinley et al., 2015a,b).

Implementing Pupil-Indexed Network State Measures by Pupillometry and Data Analysis

Establishing a link between spontaneous network fluctuations, pupil diameter, and behavioral performance is achievable in experiments recording whole-brain activity. Macroscopic measures (EEG, MEG, or fMRI) can be related to pupillometry, and to behavioral performance, learning, or decision-making outcomes.

It is well established that pupil diameter fluctuates in response to changes in environmental luminance. Thus, the first step in any study aiming to record meaningful pupillometry data are to ensure that all visual stimulation is isoluminant and no sudden transitions in visual stimulation are included in the experimental paradigm, since these may induce pupillary responses triggered by visual novelty. Additionally, subjects should be screened for preserved central and peripheral innervation to pupil muscles (i.e., consensual response in photopupillary reflex).

The recording of pupillometry is achieved noninvasively by camera-based, infrared eyetrackers. Commercial alternatives are matched by open-source systems, which in some cases include their own analysis software (Zimmermann et al., 2016) making them particularly user friendly and adaptable to the desired experimental paradigm. Usually, the pupil area or diameter is reported in arbitrary units. The time series obtained from the eyetracker system can be z-transformed, and missing data caused by blink periods can be interpolated using linear or spline methods.

The recorded pupillometry datasets can be sorted offline for sustained dilation or constriction periods, indicative of desynchronized (“active”) and synchronized (“resting”) network states respectively. Here, exact thresholds for pupil dilation, microdilation and constriction remain to be established, but tracking the first derivative of pupil area would allow for detection of rapid changes based on the signal's slope. Once sorted, the data (e.g., EEG signals) recorded during putative desynchronized or synchronized states can be analyzed accordingly and response times or learning and decision-making parameters can be compared between states. Specific data analysis techniques which would be appropriate include linear regression or representational similarity analysis (Kriegeskorte et al., 2008), among a plethora of approaches aiming to predict the neuronal or the behavioral signal on the network state periods previously classified by pupil dilation. With this approach, also for the human brain, potential network state influences can be revealed and previously unaccounted variance in the ongoing signal or in trial-to-trial variability might be explained.

Conclusion

We encourage employing pupillometry as a noninvasive, robust proxy measure of ongoing network states and highlight the need to verify the corresponding predictions derived from animal models in humans.

Fluctuations between cortical states occur constantly and frequently throughout awake periods (Buzsáki, 2006; Poulet and Petersen, 2008; McGinley et al., 2015a). These continuous changes can contribute to the variability of experimental results in awake recordings and may appear as noise if they are disregarded in the statistical modeling of neural signals. Including a fine-grained classification of network state as independent variable (i.e., covariate, regressor, or entirely new factor) could explain a considerable portion of variance in neuronal and behavioral responses, resulting in an increase of signal-to-noise ratio and more reliable readouts of human brain activity. Hence, considering network state fluctuations opens the possibility to explore and predict neural mechanisms of behavior, sensory coding, decision-making and motor responses more accurately and eventually reveal trial-to-trial variability to be more predictable than previously thought.

Investigating the direct interrelation of underlying global EEG, MEG, or fMRI activity reflecting fluctuations of brain state and pupil diameter changes, and relating both measures to behavioral performance, learning and decision making, would put previous findings into perspective, eventually demonstrating a dependency of higher-order cognitive operations on internally generated states of the brain. Spontaneous fluctuations of ongoing neuronal activity likely influence behavioral outcomes and learning, but can equally be altered by environmental stimulation, and thus represent an important mediator variable which should be considered in a broad range of neurophysiological measurements. The possibility to track these state changes within human cognitive experiments by pupil dilation measures could greatly enhance the current knowledge about the dependency of human decision-making processes on neural network states.

The work reviewed here suggests a direct relationship of changes in pupil diameter and cortical state fluctuations in the awake brain. As discussed above, pupil diameter has been commonly associated with psychological constructs, notably arousal and cognitive effort, where a consistent relationship with NA neurons in LC has gained ample empirical support. However, until recently, the close, phase-locked relationship with brain state reflected by membrane and local LFPs was not known. The question remains whether pupil dilation also proves as a reliable biomarker for the described substates of wakefulness in human subjects, and whether it can be used to predict neuronal responses, task performance and learning, beyond sensory perception paradigms. It is plausible that results in humans will follow those in animal models, given how conserved state variations are across mammalian brains (Buzsáki et al., 2013). Ultimately, the identification of a direct, unequivocal brain signature could lead to the refinement of constructs such as arousal or cognitive effort in favor of a precise neurophysiological variable provided by a pupil dilation-defined network state prediction.

Articles from eNeuro are provided here courtesy of Society for Neuroscience

PMC Copyright Notice

The articles available from the PMC site are protected by copyright, even though access is free. Copyright is held by the respective authors or publishers who provide these articles to PMC. Users of PMC are responsible for complying with the terms and conditions defined by the copyright holder.

Users should assume that standard copyright protection applies to articles in PMC, unless an article contains an explicit license statement that gives a user additional reuse or redistribution rights. PMC does not allow automated/bulk downloading of articles that have standard copyright protection.

See the copyright notice on the PMC site, https://www.ncbi.nlm.nih.gov/pmc/about/copyright/, for further details and specific exceptions.

References

Aston-Jones G, Cohen JD (2005) An integrative theory of locus coeruleus norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci 28:403–450. 10.1146/annurev.neuro.28.061604.13570916022602

Baumgarten TJ, Schnitzler A, Lange J (2015) Beta oscillations define discrete perceptual cycles in the somatosensory domain. Proc Natl Acad Sci USA 112:12187–12192. 10.1073/pnas.150143811226324922

Billeke P, Ossandon T, Stockle M, Perrone-Bertolotti M, Kahane P, Lachaux JP, Fuentealba P (2017) Brain state-dependent recruitment of high-frequency oscillations in the human hippocampus. Cortex 94:87–99. 28728081

Boly M, Balteau E, Schnakers C, Degueldre C, Moonen G, Luxen A, Phillips C, Peigneux P, Maquet P, Laureys S (2007) Baseline brain activity fluctuations predict somatosensory perception in humans. Proc Natl Acad Sci USA 104:12187–12192. 10.1073/pnas.061140410417616583

Buzsáki G (2006) Rhythms of the brain. New York: Oxford University Press.

Buzsáki G, Logothetis N, Singer W (2013) Scaling brain size, keeping timing: evolutionary preservation of brain rhythms. Neuron 80:751–764. 10.1016/j.neuron.2013.10.00224183025

Castro-Alamancos MA, Gulati T (2014) Neuromodulators produce distinct activated states in neocortex. J Neurosci 34:12353–12367. 10.1523/JNEUROSCI.1858-14.2014 25209276

Cavanagh JF, Wiecki TV, Kochar A, Frank MJ (2014) Eye tracking and pupillometry are indicators of dissociable latent decision processes. J Exp Psychol Gen 143:1476–1488. 10.1037/a0035813 24548281

Chang C, Leopold DA, Schölvinck ML, Mandelkow H, Picchioni D, Liu X, Ye FQ, Turchi JN, Duyn JH (2016) Tracking brain arousal fluctuations with fMRI. Proc Natl Acad Sci USA 113:4518–4523. 10.1073/pnas.1520613113 27051064

Chen N, Sugihara H, Sur M (2015) An acetylcholine-activated microcircuit drives temporal dynamics of cortical activity. Nat Neurosci 18:892–902. 10.1038/nn.400225915477

Clayton EC, Rajkowski J, Cohen JD, Aston-Jones G (2004) Phasic activation of monkey locus ceruleus neurons by simple decisions in a forced choice task. J Neurosci 24:9914–9920. 10.1523/JNEUROSCI.2446-04.200415525776

Constantinople CM, Bruno RM (2011) Effects and mechanisms of wakefulness on local cortical networks. Neuron 69:1061–1068. 10.1016/j.neuron.2011.02.040 21435553

Coon WG, Gunduz A, Brunner P, Ritaccio AL, Pesaran B, Schalk G (2016) Oscillatory phase modulates the timing of neuronal activations and resulting behavior. NeuroImage 133:294–301. 10.1016/j.neuroimage.2016.02.08026975551

Crochet S, Petersen CC (2006) Correlating whisker behavior with membrane potential in barrel cortex of awake mice. Nat Neurosci 9:608–610. 10.1038/nn1690 16617340

Custo A, van der Ville D, Wells WM, Tomescu IM, Michel C (2017) EEG resting-state networks: microstates' source localization. Brain Connect 7:671–682. 10.1089/brain.2016.0476 28938855

de Gee JW, Knapen T, Donner TH (2014) Decision-related pupil dilation reflects upcoming choice and individual bias. Proc Natl Acad Sci USA 111:618–625. 10.1073/pnas.1317557111

Destexhe A, Hughes SW, Rudolph M, Crunelli V (2007) Are corticothalamic ‘up’ states fragments of wakefulness? Trends Neurosci 30:334–343. 10.1016/j.tins.2007.04.006 17481741

Ebitz RB, Pearson JM, Platt ML (2014) Pupil size and social vigilance in rhesus macaques. Front Neurosci 8:100. 10.3389/fnins.2014.00100 24834026

Eggermann E, Kremer Y, Crochet S, Petersen CC (2014) Cholinergic signals in mouse barrel cortex during active whisker sensing. Cell Rep 9:1654–1660. 10.1016/j.celrep.2014.11.005 25482555

Einhäuser W, Stout J, Koch C, Carter O (2008) Pupil dilation reflects perceptual selection and predicts subsequent stability in perceptual rivalry. Proc Natl Acad Sci USA 105:1704–1709. 18250340

Einhäuser W, Koch C, Carter OL (2010) Pupil dilation betrays the timing of decisions. Front Hum Neurosci 4:18. 10.3389/fnhum.2010.00018 20204145

Fazlali Z, Ranjbar-Slamloo Y, Adibi M, Arabzadeh E (2016) Correlation between cortical state and locus coeruleus activity: implications for sensory coding in rat barrel cortex. Front Neural Circuits 10:14 10.3389/fncir.2016.0001427047339

Fiedler S, Glöckner A (2012) The dynamics of decision making in risky choice: an eye-tracking analysis. Front Psychol 3:335. 10.3389/fpsyg.2012.00335 23162481

Gao L, Meng X, Ye C, Zhang H, Liu C, Dan Y, Poo MM, He J, Zhang X (2009) Entrainment of slow oscillations of auditory thalamic neurons by repetitive sound stimuli. J Neurosci 29:6013–6021. 10.1523/JNEUROSCI.5733-08.200919420268

Geng JJ, Blumenfeld Z, Tyson TL, Minzenberg MJ (2015) Pupil diameter reflects uncertainty in attentional selection during visual search. Front Hum Neurosci 9:435 10.3389/fnhum.2015.0043526300759

Gilzenrat MS, Nieuwenhuis S, Jepma M, Cohen J (2010) Pupil diameter tracks changes in control state predicted by the adaptive gain theory of locus coeruleus function. Cogn Affect Behav Neurosci 10:252–269. 10.3758/CABN.10.2.25220498349

Goard M, Dan Y (2009) Basal forebrain activation enhances cortical coding of natural scenes. Nat Neurosci 12:1444–1449. 10.1038/nn.240219801988

Godwin D, Barry RL, Marois R (2015) Breakdown of the brain’s functional network modularity with awareness. Proc Natl Acad Sci USA 112:3799–3804. 10.1073/pnas.1414466112 25759440

Hall TM, de Carvalho F, Jackson A (2014) A common structure underlies low-frequency cortical dynamics in movement, sleep, and sedation. Neuron 83:1185–1199. 10.1016/j.neuron.2014.07.02225132467

Harris KD, Thiele A (2011) Cortical state and attention. Nat Rev Neurosci 12:509–523. 10.1038/nrn308421829219

Hesselmann G, Kell CA, Eger E, Kleinschmidt A (2008) Spontaneous local variations in ongoing neural activity bias perceptual decisions. Proc Natl Acad Sci USA 105:10984–10989. 10.1073/pnas.071204310518664576

Jackson J, Ayzenshtat I, Karnani MM, Yuste R (2016) VIP+ interneurons control neocortical activity across brain states. J Neurophysiol 115:3008–3017. 10.1152/jn.01124.201526961109

Joshi S, Li Y, Kalwani RM, Gold JI (2016) Relationships between pupil diameter and neuronal activity in the locus coeruleus, colliculi, and cingulate cortex. Neuron 89:221–234. 10.1016/j.neuron.2015.11.02826711118

Kahneman D, Beatty J (1966) Pupil diameter and load on memory. Science 154:1583–1585. 5924930

Kaneko M, Fu Y, Stryker MP (2017) Locomotion induces stimulus-specific response enhancement in adult visual cortex. J Neurosci 37:3532–3543. 28258167

Kang O, Wheatley T (2015) Pupil dilation patterns reflect the contents of consciousness. Conscious Cogn 35:128–135. 10.1016/j.concog.2015.05.00126002764

Kriegeskorte N, Mur M, Bandettini P (2008) Representational similarity analysis - connecting the branches of systems neuroscience. Front Syst Neurosci 2:4.19104670

Lavín C, San Martín R, Rosales Jubal E (2014) Pupil dilation signals uncertainty and surprise in a learning gambling task. Front Behav Neurosci 7:218. 10.3389/fnbeh.2013.0021824427126

Lee S-H, Dan Y (2012) Neuromodulation of brain states. Neuron 76:209–222. 10.1016/j.neuron.2012.09.012 23040816

Lima B, Singer W, Neuenschwander S (2011) Gamma responses correlate with temporal expectation in monkey primary visual cortex. J Neurosci 31:15919–15931. 10.1523/JNEUROSCI.0957-11.201122049435

Lu R, Sun W, Liang Y, Kerlin A, Bierfeld J, Seelig JD, Wilson DE, Scholl B, Mohar B, Tanimoto M, Koyama M, Fitzpatrick D, Orger MB, Ji N (2017) Video-rate volumetric functional imaging of the brain at synaptic resolution. Nat Neurosci 20:620–628. 10.1038/nn.451628250408

Martins AR, Froemke RC (2015) Coordinated forms of noradrenergic plasticity in the locus coeruleus and primary auditory cortex. Nat Neurosci 18:1483–1492. 10.1038/nn.409026301326

McGinley MJ, David SV, McCormick DA (2015a) Cortical membrane potential signature of optimal states for sensory signal detection. Neuron 87:179–192. 26074005

McGinley MJ, Vinck M, Reimer J, Batista-Brito R, Zagha E, Cadwell CR, Tolias AS, Cardin JA, McCormick DA (2015b) Waking state: rapid variations modulate neuronal and behavioral responses. Neuron 87:1143–1161. 26402600

Murphy PR, Robertson IH, Balsters JH, O’connell RG (2011) Pupillometry and P3 index the locus coeruleus-noradrenergic arousal function in humans. Psychophysiology 48:1532–1543. 10.1111/j.1469-8986.2011.01226.x21762458

Murphy PR, O’Connell RG, O’Sullivan M, Robertson IH, Balsters JH (2014a) Pupil diameter covaries with BOLD activity in human locus coeruleus. Hum Brain Mapp 35:4140–4154. 10.1002/hbm.22466 24510607

Murphy PR, Vandekerckhove J, Nieuwenhuis S (2014b) Pupil-linked arousal determines variability in perceptual decision making. PLoS Comput Biol 10:e1003854 10.1371/journal.pcbi.100385425232732

Nassar MR, Rumsey KM, Wilson RC, Parikh K, Heasly B, Gold JI (2012) Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15:1040–1046. 10.1038/nn.3130 22660479

Pachitariu M, Lyamzin DR, Sahani M, Lesica NA (2014) State-dependent population coding in primary auditory cortex. J Neurosci 35:2058–2073.

Parikh V, Kozak R, Martinez V, Sarter M (2007) Prefrontal acetylcholine release controls cue detection on multiple timescales. Neuron 56:141–154. 10.1016/j.neuron.2007.08.02517920021

Posner MI (1980) Orienting of attention. Q J Exp Psychol 32:3–25. 10.1080/003355580082482317367577

Poulet JF, Petersen CC (2008) Internal brain state regulates membrane potential synchrony in barrel cortex of behaving mice. Nature 454:881–885. 10.1038/nature0715018633351

Poulet J, Fernandez LMJ, Crochet S, Petersen CCH (2012) Thalamic control of cortical states. Nat Neurosci 15:370–372. 10.1038/nn.3035 22267163

Preuschoff K, ‘t Hart BM, Einhäuser W (2011) Pupil dilation signals surprise: evidence for noradrenaline’s role in decision making. Front Neurosci 5:115. 10.3389/fnins.2011.00115 21994487

Reimer J, Froudarakis E, Cadwell CR, Yatsenko D, Denfield GH, Tolias AS (2014) Pupil fluctuations track fast switching of cortical states during quiet wakefulness. Neuron 84:355–362. 10.1016/j.neuron.2014.09.03325374359

Reimer J, McGinley M, Liu Y, Rodenkirch C, Wang Q, McCormick DA, Tolias AS (2016) Pupil fluctuations track rapid changes in adrenergic and cholinergic activity in cortex. Nat Commun 7:13289 10.1038/ncomms1328927824036

Safaai H, Neves R, Eschenko O, Logothetis N, Panzeri S (2015) Modeling the effect of locus coeruleus firing on cortical state dynamics and single-trial sensory processing. Proc Natl Acad Sci USA 112:12834–12839. 10.1073/pnas.151653911226417078

Sanchez-Vives MV, Massimini M, Mattia M (2017) Shaping the default activity pattern of the cortical network. Neuron 94:993–1001. 10.1016/j.neuron.2017.05.015 28595056

Scheinost D, Tokoglu F, Shen X, Finn ES, Noble S, Papademetris X, Constable RT (2016) Fluctuations in global brain activity are associated with changes in whole-brain connectivity of functional networks. IEEE Trans Biomed Eng 63:2540–2549. 27541328

Schneider M, Hathway P, Leuchs L, Sämann PG, Czisch M, Spoormaker VI (2016) Spontaneous pupil dilations during the resting state are associated with activation of the salience network. NeuroImage 139:189–201. 10.1016/j.neuroimage.2016.06.01127291493

Schölvinck ML, Saleem AB, Benucci A, Harris KD, Carandini M (2015) Cortical state determines global variability and correlations in visual cortex. J Neurosci 35:170–178. 25568112

Schroeder CE, Lakatos P (2009) Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci 32:9–18. 10.1016/j.tins.2008.09.01219012975

Schwalm M, Schmid F, Wachsmuth L, Backhaus H, Kronfeld A, Aedo Jury F, Prouvot PH, Fois C, Albers F, van Alst T, Faber C, Stroh A (2017) Cortex-wide BOLD fMRI activity reflects locally-recorded slow oscillation-associated calcium waves. Elife 6.

Shine JM, Bissett PG, Bell PT, Koyejo O, Balsters JH, Gorgolewski KJ, Moodie CA, Poldrack RA (2016) The dynamics of functional brain networks: integrated network states during cognitive task performance. Neuron 92:544–554. 10.1016/j.neuron.2016.09.01827693256

Steriade M, Timofeev I, Grenier F (2001) Natural waking and sleep states: a view from inside neocortical neurons. J Neurophysiol 85:1969–1985. 11353014

Stroh A, Adelsberger H, Groh A, Rühlmann C, Fischer S, Schierloh A, Deisseroth K, Konnerth A (2013) Making waves: initiation and propagation of corticothalamic Ca2+ waves in vivo. Neuron 77:1136–1150. 10.1016/j.neuron.2013.01.031 23522048

Tursky B, Shapiro D, Crider A, Kahneman D (1969) Pupillary, heart rate, and skin resistance changes during a mental task. J Exp Psychol 79:164–167. 5785627

Usher M, Cohen JD, Servan-Schreiber D, Rajkowski J, Aston-Jones G (1999) The role of locus coeruleus in the regulation of cognitive performance. Science 283:549–554. 9915705

Varazzani C, San-Galli A, Gilardeau S, Bouret S (2015) Noradrenaline and dopamine neurons in the reward/effort trade-off: a direct electrophysiological comparison in behaving monkeys. J Neurosci 35:7866–7877. 10.1523/JNEUROSCI.0454-15.201525995472

Vinck M, Batista-Brito R, Knoblich U, Cardin JA (2015) Arousal and locomotion make distinct contributions to cortical activity patterns and visual encoding. Neuron 86:740–754. 25892300

Yerkes RM, Dodson JD (1908) The relation of strength of stimulus to rapidity of habit-formation. J Comp Neurol Psychol 18:459–482. 10.1002/cne.920180503

Yu AJ, Dayan P (2005) Uncertainty, neuromodulation, and attention. Neuron 46:681–692. 10.1016/j.neuron.2005.04.026 15944135

Zagha E, Casale AE, Sachdev RN, McGinley MJ, McCormick DA (2013) Motor cortex feedback influences sensory processing by modulating network state. Neuron 79:567–578. 10.1016/j.neuron.2013.06.00823850595

Zimmermann J, Vazquez Y, Glimcher PW, Pesaran B, Louie K (2016) Oculomatic: high speed, reliable, and accurate open-source eye tracking for humans and non-human primates. J Neurosci Methods 270:138–146. 10.1016/j.jneumeth.2016.06.01627339782

[Back]

[Back]

[Back]

Figure 1.

Pupil diameter in human cognitive neuroscience and as an accurate predictor of rapid variations in parameters related to brain state and arousal in the mouse model. A, In a short-memory task, pupil dilation was modulated by the amount of items under active processing at any time: average pupil diameter (mm) for five subjects during auditory presentation (before “pause” period) and recall (after pause period) of digit strings of varying lengths (three to seven digits). The authors found that the pupil dilates when items are presented and constricts during report. The rate of change of these functions was related to task difficulty. From Kahneman and Beatty (1966). B. Pupil dilation departs from baseline ∼1 s before a volitional button press, peaks at 420 ms after the response, and relaxes back to baseline after ∼2 s, thus revealing the time of decision making. Authors interpreted pupil dilation as a marker of NE release from LC and as evidence for the latter role in consolidation of cognitive decisions. From Einhäuser et al. (2010). C, Pupil dilation displays an anticipatory response to uncertainty levels associated with options in a strategic gambling task (Iowa Gambling Task, IGT) where subjects are asked to maximize their profits by choosing the best drawing strategy from a set of four card decks, each of them delivering gains and losses following a pattern unknown for participants. Greater pupil dilation was observed in conditions with a low probability of incoming negative feedback (NF), as compared to conditions where NF had an enhanced probability to occur. Authors interpreted these results as evidence of pupil dilation signaling LC response to decision making in unfamiliar contexts. From Lavín et al. (2014). All axes in A–C depict pupil dilation (y-axis) versus time (x-axis). D, upper panel, Simultaneous recording of pupil diameter and membrane potential of a cortical neuron in layer 2/3 of the mouse primary visual cortex. Pupil diameter exhibits spontaneous variations in size even in the synchronized state and in the absence of locomotion. Note the strong relationship between slow (2–10 Hz) rhythmic synaptic activity and constriction, and the suppression of this activity with dilation. Lower panel, Comparison of pupil diameter and density of low-frequency (<10 Hz) rhythmic synaptic activity in a layer 5 pyramidal cell in the auditory cortex. Increases in pupil diameter are associated with prominent suppression of low-frequency synaptic activity (desynchronized state). From McGinley et al. (2015b). E, Whole-cell recordings from a layer 5 pyramidal neuron in auditory cortex of an awake mouse while simultaneously monitoring pupil diameter and locomotion. Brief dilations of the pupil (microdilations), are highlighted in gray and are associated with a suppression of the low-frequency activity and a depolarization of this neuron independent of locomotion. From McGinley et al. (2015b).

[Back]