Elsevier

Psychoneuroendocrinology

Volume 53, March 2015, Pages 268-280
Psychoneuroendocrinology

The interaction of acute and chronic stress impairs model-based behavioral control

https://doi.org/10.1016/j.psyneuen.2014.12.017Get rights and content

Highlights

  • Computational Modeling reveals that the interaction of acute and chronic stress impairs model-based control.

  • Acute stress does not affect model-based (goal-directed) vs. model-free (habitual) control.

  • Cortisol reactivity relates to a lower degree of model-based decision-making.

  • Subjective stress reactivity is associated with a higher degree of model-based control.

Summary

It is suggested that acute stress shifts behavioral control from goal-directed, model-based toward habitual, model-free strategies. Recent findings indicate that interindividual differences in the cortisol stress response influence model-based decision-making. Although not yet investigated in humans, animal studies show that chronic stress also shifts decision-making toward more habitual behavior. Here, we ask whether acute stress and individual vulnerability factors, such as stress reactivity and previous exposure to stressful life events, impact the balance between model-free and model-based control systems. To test this, 39 male participants (21–30 years old) were exposed to a potent psychosocial stressor (Trier Social Stress Test) and a control condition in a within-subjects design before they performed a sequential decision-making task which evaluates the balance between the two systems. Physiological and subjective stress reactivity was assessed before, during, and after acute stress exposure. By means of computational modeling, we demonstrate that interindividual variability in stress reactivity predicts impairments in model-based decision-making. Whereas acute psychosocial stress did not alter model-based behavioral control, we found chronic and acute stress to interact in their detrimental effect on decision-making: subjects with high but not low chronic stress levels as indicated by stressful life events exhibited reduced model-based control in response to acute psychosocial stress. These findings emphasize that stress reactivity and chronic stress play an important role in mediating the relationship between stress and decision-making. Our results might stimulate new insights into the interplay between chronic and acute stress, attenuated model-based control, and the pathogenesis of various psychiatric diseases.

Introduction

Making effective decisions is particularly relevant in stressful situations and may depend on individual responsiveness during acute stress as well as on the long-term stress load. Dual-system theories of decision-making postulate a goal-directed system and a habitual system to compete for behavioral control (Balleine and Dickinson, 1998, Balleine and O’Doherty, 2010). Recently, computational modeling accounts of reinforcement learning have amended these theories (Daw et al., 2005): here, goal-directed, model-based behavior is seen as a flexible, albeit computationally complex strategy, which builds an internal mental model of the environment. Thereby, future actions and their potential outcomes are planned in a forward manner. In contrast, habitual, model-free control is seen as a retrospective and therefore more rigid strategy driven by past rewards which neglects environmental structure for the advantage of computational efficiency. Crucially, human decision-making involves both control systems with considerable interindividual variability (Daw et al., 2011). However, it remains an intriguing question how control over actions is allocated between the two systems depending on the particular situation and on interindividual trait differences (Dolan and Dayan, 2013).

Among situational factors that influence this allocation of control, stress is a key candidate for biasing the balance of the two systems toward habitual decision-making (Schwabe and Wolf, 2009, Schwabe and Wolf, 2011, Schwabe and Wolf, 2013). At the neurobiological level, cortisol, the endproduct of the hypothalamus-pituitary-adrenal (HPA-) axis, might affect prefrontal executive capacities, which may thus limit the degree of control exerted by the more sophisticated, model-based system. On the behavioral level, stress has been shown to influence decision-making, e.g. in terms of dysfunctional strategy use, automatic responding, goal implementation, response conflicts, risk taking, feedback processing per se and reward vs. punishment sensitivity (Petzold et al., 2010, Plessow et al., 2011, Plessow et al., 2012, Starcke and Brand, 2012). In a recent study, Otto et al. (2013b) compared acutely stressed and non-stressed participants and did not observe between-group differences in the balance of behavioral control. However, interindividual differences in physiological stress response, as measured by cortisol increase, were negatively correlated with the degree of model-based control across both groups. Importantly, this points to the direction that interindividual differences in stress reactivity, rather than a stress-eliciting condition per se, might impact decision-making.

Beyond acute stress, animal studies suggest that chronic stress shifts decision-making toward more habitual strategies: Dias-Ferreira et al. (2009) observed that chronically stressed rats became insensitive to outcome devaluation, a key characteristic of habitual behavior. In humans, the effect of chronic stress and the interplay between previous stress experience and acute stress on model-based decision-making has not yet been investigated.

Here, we utilized a within-subjects design to assess the influence of a potent acute psychosocial stressor on the balance between model-based and model-free control as assessed via sequential decision-making (Daw et al., 2011). By means of computational modeling, we first asked if acute psychosocial stress diminishes the degree of model-based control within individuals. Second, we tested if interindividual variations in physiological and subjective stress reactivity predict the balance of behavioral control per se. Finally, we examined the interaction of chronic and acute stress levels in human decision-making.

Section snippets

Participants

Thirty-nine healthy male subjects recruited by Internet advertisements completed the study (mean age: 25.2, SD = 2.73, range: 21–30 years). All participants except for one had obtained university entrance qualification, one held the general certificate of secondary education. The average years of education (including school, university etc.) was 16.32 (SD = 3.21), the average duration of unemployment counted 0.19 years (SD = 0.44). Exclusion criteria comprised presence or history of any neurological

Cortisol and heart rate response

Seven participants did not display a cortisol increase of at least 2.5 nmol/l and thus were considered as non-responders (Schommer et al., 2003, Van Cauter and Refetoff, 1985). The responder rate (∼82%) was in line with other studies using the TSST (Kudielka et al., 2007, Petzold et al., 2010). Cortisol response in n = 32 responders was analyzed using a repeated-measures ANOVA with the within-subjects factors time (t1–t5) and acute stress (stress vs. control). A significant main effect of acute

Discussion

This study investigated the influence of stress on the balance of model-based and model-free behavioral control during a two-step decision task in a within-subjects design. First, after inducing acute psychosocial stress, we did not observe a shift toward model-free behavioral control across the entire sample. Second, we showed that variability in physiological and psychological stress reactivity is associated with interindividual differences in the balance between model-free and model-based

Conclusion

We show that interindividual differences in acute subjective and physiological stress response impact the degree of model-based behavioral control. Furthermore, a reduction in model-based control in response to acute stress was only observed in subjects with higher levels of chronic stress as indicated by a higher score in the Life Stress scale for the last 24 months.

By defining interindividual differences in acute stress response and in chronic stress experience as crucial factors in

Role of the funding source

This study was supported by grants from the German Research Foundation awarded to FS (DFG SCHL1969/1-1 and DFG SCHL 1969/2-1). The study was supported by the Max Planck Society.

Conflict of interest

None declared.

Acknowledgements

The authors thank K. Hudl, M. Kerkemeyer, L. Lüttgau, J. Schott and T. Wilbertz for their assistance in the TSST protocol and H. Schmidt for her help in designing the figures. The authors thank M. Gaebler for helpful comments on the analysis of heart rate data.

References (55)

  • L. Schwabe et al.

    Stress and multiple memory systems: from ‘thinking’ to ‘doing’

    Trends Cogn. Sci.

    (2013)
  • P. Smittenaar et al.

    Disruption of dorsolateral prefrontal cortex decreases model-based in favor of model-free control in humans

    Neuron

    (2013)
  • K. Starcke et al.

    Decision making under stress: a selective review

    Neurosci. Biobehav. Rev.

    (2012)
  • K. Starcke et al.

    Does stress alter everyday moral decision-making?

    Psychoneuroendocrinology

    (2011)
  • K.E. Stephan et al.

    Bayesian model selection for group studies

    Neuroimage

    (2009)
  • K. Wunderlich et al.

    Dopamine enhances model-based over model-free choice behavior

    Neuron

    (2012)
  • B.W. Balleine et al.

    Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action

    Neuropsychopharmacology

    (2010)
  • A. Bechara et al.

    Deciding advantageously before knowing the advantageous strategy

    Science

    (1997)
  • A. Bechara et al.

    Failure to respond autonomically to anticipated future outcomes following damage to prefrontal cortex

    Cereb. Cortex

    (1996)
  • J. Cohen

    Statistical Power Analysis for the Behavioral Sciences

    (1988)
  • S. Cohen et al.

    A global measure of perceived stress

    J. Health Soc. Behav.

    (1983)
  • D.R. Critchley

    Genetic, biochemical and structural approaches to talin function

    Biochem. Soc. Trans.

    (2005)
  • C. Dalbert

    ASTS – Aktuelle Stimmungsskala

    (1992)
  • A.R. Damasio

    The Feeling of What Happens: Body and Emotion in the Making of Consciousness

    (1999)
  • N.D. Daw et al.

    Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control

    Nat. Neurosci.

    (2005)
  • E. Dias-Ferreira et al.

    Chronic stress causes frontostriatal reorganization and affects decision-making

    Science

    (2009)
  • B. Eppinger et al.

    Of goals and habits: age-related and individual differences in goal-directed decision-making

    Front. Neurosci.

    (2013)
  • Cited by (76)

    • Neural systems and the emotion-memory link

      2021, Neurobiology of Learning and Memory
    View all citing articles on Scopus
    1

    Both these authors contributed equally to this work.

    View full text