Skip to main content
Log in

Decoding six basic emotions from brain functional connectivity patterns

  • Research Paper
  • Published:
Science China Life Sciences Aims and scope Submit manuscript

Abstract

Although distinctive neural and physiological states are suggested to underlie the six basic emotions, basic emotions are often indistinguishable from functional magnetic resonance imaging (fMRI) voxelwise activation (VA) patterns. Here, we hypothesize that functional connectivity (FC) patterns across brain regions may contain emotion-representation information beyond VA patterns. We collected whole-brain fMRI data while human participants viewed pictures of faces expressing one of the six basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise) or showing neutral expressions. We obtained FC patterns for each emotion across brain regions over the whole brain and applied multivariate pattern decoding to decode emotions in the FC pattern representation space. Our results showed that the whole-brain FC patterns successfully classified not only the six basic emotions from neutral expressions but also each basic emotion from other emotions. An emotion-representation network for each basic emotion that spanned beyond the classical brain regions for emotion processing was identified. Finally, we demonstrated that within the same brain regions, FC-based decoding consistently performed better than VA-based decoding. Taken together, our findings revealed that FC patterns contained emotional information and advocated for paying further attention to the contribution of FCs to emotion processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adolphs, R. (2008). Fear, faces, and the human amygdala. Curr Opin Neurobiol 18, 166–172.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Anzellotti, S., and Coutanche, M.N. (2018). Beyond functional connectivity: investigating networks of multivariate representations. Trends Cogn Sci 22, 258–269.

    Article  PubMed  Google Scholar 

  • Baczkowski, B.M., Johnstone, T., Walter, H., Erk, S., and Veer, I.M. (2017). Sliding-window analysis tracks fluctuations in amygdala functional connectivity associated with physiological arousal and vigilance during fear conditioning. NeuroImage 153, 168–178.

    Article  PubMed  Google Scholar 

  • Barrett, L.F., Bliss-Moreau, E., Duncan, S.L., Rauch, S.L., and Wright, C.I. (2007). The amygdala and the experience of affect. Soc Cogn Affect Neurosci 2, 73–83.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ceravolo, L., Frühholz, S., Pierce, J., Grandjean, D., and Péron, J. (2021). Basal ganglia and cerebellum contributions to vocal emotion processing as revealed by high-resolution fMRI. Sci Rep 11, 10645.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Cha, J., DeDora, D., Nedic, S., Ide, J., Greenberg, T., Hajcak, G., and Mujica-Parodi, L.R. (2016). Clinically anxious individuals show disrupted feedback between inferior frontal gyrus and prefrontal-limbic control circuit. J Neurosci 36, 4708–4718.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Chen, J., Wang, Z., Li, Z., Peng, D., and Fang, Y. (2021). Disturbances of affective cognition in mood disorders. Sci China Life Sci 64, 938–941.

    Article  PubMed  Google Scholar 

  • Cole, M.W., Ito, T., Cocuzza, C., and Sanchez-Romero, R. (2021). The functional relevance of task-state functional connectivity. J Neurosci 41, 2684–2702.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Denisko, D., and Hoffman, M.M. (2018). Classification and interaction in random forests. Proc Natl Acad Sci USA 115, 1690–1692.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Desikan, R.S., Ségonne, F., Fischl, B., Quinn, B.T., Dickerson, B.C., Blacker, D., Buckner, R.L., Dale, A.M., Maguire, R.P., Hyman, B.T., et al. (2006). An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. NeuroImage 31, 968–980.

    Article  PubMed  Google Scholar 

  • Ekman, P. (1992). An argument for basic emotions. Cognition Emotion 6, 169–200.

    Article  Google Scholar 

  • Fang, F., and Hu, H. (2021). Recent progress on mechanisms of human cognition and brain disorders. Sci China Life Sci 64, 843–846.

    Article  PubMed  Google Scholar 

  • Giordano, B.L., Whiting, C., Kriegeskorte, N., Kotz, S.A., Gross, J., and Belin, P. (2021). The representational dynamics of perceived voice emotions evolve from categories to dimensions. Nat Hum Behav 5, 1203–1213.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hamann, S. (2012). Mapping discrete and dimensional emotions onto the brain: controversies and consensus. Trends Cogn Sci 16, 458–466.

    Article  PubMed  Google Scholar 

  • Henriksson, L., Khaligh-Razavi, S.M., Kay, K., and Kriegeskorte, N. (2015). Visual representations are dominated by intrinsic fluctuations correlated between areas. NeuroImage 114, 275–286.

    Article  PubMed  Google Scholar 

  • Hu, X., Chen, J., Wang, F., and Zhang, D. (2019). Ten challenges for EEG-based affective computing. Brain Sci Adv 5, 1–20.

    Article  Google Scholar 

  • Hutchison, R.M., Womelsdorf, T., Allen, E.A., Bandettini, P.A., Calhoun, V.D., Corbetta, M., Della Penna, S., Duyn, J.H., Glover, G.H., Gonzalez-Castillo, J., et al. (2013). Dynamic functional connectivity: promise, issues, and interpretations. NeuroImage 80, 360–378.

    Article  PubMed  Google Scholar 

  • Ito, T., Hearne, L., Mill, R., Cocuzza, C., and Cole, M.W. (2020). Discovering the computational relevance of brain network organization. Trends Cogn Sci 24, 25–38.

    Article  PubMed  Google Scholar 

  • Jiang, F., Jin, H., Gao, Y., Xie, X., Cummings, J., Raj, A., and Nagarajan, S. (2022). Time-varying dynamic network model for dynamic resting state functional connectivity in fMRI and MEG imaging. NeuroImage 254, 119131.

    Article  PubMed  Google Scholar 

  • Kragel, P.A., and LaBar, K.S. (2016). Decoding the nature of emotion in the brain. Trends Cogn Sci 20, 444–455.

    Article  PubMed  PubMed Central  Google Scholar 

  • Langs, G., Menze, B.H., Lashkari, D., and Golland, P. (2011). Detecting stable distributed patterns of brain activation using Gini contrast. NeuroImage 56, 497–507.

    Article  PubMed  Google Scholar 

  • Liang, Y., Liu, B., Xu, J., Zhang, G., Li, X., Wang, P., and Wang, B. (2017). Decoding facial expressions based on face-selective and motion-sensitive areas. Hum Brain Mapp 38, 3113–3125.

    Article  PubMed  PubMed Central  Google Scholar 

  • Liang, Y., Liu, B., Li, X., and Wang, P. (2018). Multivariate pattern classification of facial expressions based on large-scale functional connectivity. Front Hum Neurosci 12, 94.

    Article  PubMed  PubMed Central  Google Scholar 

  • Liang, Y., and Liu, B. (2020). Cross-subject commonality of emotion representations in dorsal motion-sensitive areas. Front Neurosci 14, 567797.

    Article  PubMed  PubMed Central  Google Scholar 

  • Liu, C., Song, S., Guo, X., Zhu, Z., and Zhang, J. (2018a). Image categorization from functional magnetic resonance imaging using functional connectivity. J Neurosci Methods 309, 71–80.

    Article  PubMed  Google Scholar 

  • Liu, C., Li, Y., Song, S., and Zhang, J. (2020). Decoding disparity categories in 3-dimensional images from fMRI data using functional connectivity patterns. Cogn Neurodyn 14, 169–179.

    Article  PubMed  Google Scholar 

  • Liu, J., Zhang, F., Liu, X., Zhuo, Z., Wei, J., Du, M., Chan, Q., Wang, X., and Wang, D. (2018b). Altered small-world, functional brain networks in patients with lower back pain. Sci China Life Sci 61, 1420–1424.

    Article  PubMed  Google Scholar 

  • Murphy, F.C., Nimmo-Smith, I., and Lawrence, A.D. (2003). Functional neuroanatomy of emotions: a meta-analysis. Cogn Affective Behaval Neurosci 3, 207–233.

    Article  Google Scholar 

  • Naselaris, T., Kay, K.N., Nishimoto, S., and Gallant, J.L. (2011). Encoding and decoding in fMRI. NeuroImage 56, 400–410.

    Article  PubMed  Google Scholar 

  • Palomero-Gallagher, N., and Amunts, K. (2022). A short review on emotion processing: a lateralized network of neuronal networks. Brain Struct Funct 227, 673–684.

    Article  PubMed  Google Scholar 

  • Pantazatos, S.P., Talati, A., Pavlidis, P., and Hirsch, J. (2012). Decoding unattended fearful faces with whole-brain correlations: an approach to identify condition-dependent large-scale functional connectivity. PLoS Comput Biol 8, e1002441.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Pessoa, L., and Adolphs, R. (2010). Emotion processing and the amygdala: from a ‘low road’ to ‘many roads’ of evaluating biological significance. Nat Rev Neurosci 11, 773–782.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Pessoa, L. (2018). Understanding emotion with brain networks. Curr Opin Behaval Sci 19, 19–25.

    Article  Google Scholar 

  • Saarimäki, H., Ejtehadian, L.F., Glerean, E., Jääskeläinen, I.P., Vuilleumier, P., Sams, M., and Nummenmaa, L. (2018). Distributed affective space represents multiple emotion categories across the human brain. Soc Cogn Affective Neurosci 13, 471–482.

    Article  Google Scholar 

  • Saarimäki, H., Glerean, E., Smirnov, D., Mynttinen, H., Jääskeläinen, I.P., Sams, M., and Nummenmaa, L. (2022). Classification of emotion categories based on functional connectivity patterns of the human brain. NeuroImage 247, 118800.

    Article  PubMed  Google Scholar 

  • Saarimäki, H., Gotsopoulos, A., Jääskeläinen, I.P., Lampinen, J., Vuilleumier, P., Hari, R., Sams, M., and Nummenmaa, L. (2016). Discrete neural signatures of basic emotions. Cereb Cortex 26, 2563–2573.

    Article  PubMed  Google Scholar 

  • Shakil, S., Lee, C.H., and Keilholz, S.D. (2016). Evaluation of sliding window correlation performance for characterizing dynamic functional connectivity and brain states. NeuroImage 133, 111–128.

    Article  PubMed  Google Scholar 

  • Shmueli, G. (2010). To explain or to predict? Statist Sci 25, 289–310, 222.

    Article  Google Scholar 

  • Song, T., Zheng, W., Song, P., and Cui, Z. (2020). EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affective Comput 11, 532–541.

    Article  Google Scholar 

  • Stam, C.J., and van Straaten, E.C.W. (2012). The organization of physiological brain networks. Clin Neurophysiol 123, 1067–1087.

    Article  CAS  PubMed  Google Scholar 

  • Szucs, D., and Ioannidis, J.P. (2020). Sample size evolution in neuroimaging research: An evaluation of highly-cited studies (1990–2012) and of latest practices (2017–2018) in high-impact journals. NeuroImage 221, 117164.

    Article  PubMed  Google Scholar 

  • Tottenham, N., Tanaka, J.W., Leon, A.C., McCarry, T., Nurse, M., Hare, T. A., Marcus, D.J., Westerlund, A., Casey, B.J., and Nelson, C. (2009). The NimStim set of facial expressions: judgments from untrained research participants. Psychiatry Res 168, 242–249.

    Article  PubMed  PubMed Central  Google Scholar 

  • Tu, Y., Cao, J., Bi, Y., and Hu, L. (2021). Magnetic resonance imaging for chronic pain: diagnosis, manipulation, and biomarkers. Sci China Life Sci 64, 879–896.

    Article  PubMed  Google Scholar 

  • Vytal, K., and Hamann, S. (2010). Neuroimaging support for discrete neural correlates of basic emotions: a voxel-based meta-analysis. J Cogn Neurosci 22, 2864–2885.

    Article  PubMed  Google Scholar 

  • Wang, S., Tudusciuc, O., Mamelak, A.N., Ross, I.B., Adolphs, R., and Rutishauser, U. (2014). Neurons in the human amygdala selective for perceived emotion. Proc Natl Acad Sci USA 111, E3110–E3119.

    CAS  PubMed  PubMed Central  Google Scholar 

  • Wang, X., Fang, Y., Cui, Z., Xu, Y., He, Y., Guo, Q., and Bi, Y. (2016). Representing object categories by connections: evidence from a mutivariate connectivity pattern classification approach. Hum Brain Mapp 37, 3685–3697.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wang, Y., Zhu, Z., Chen, B., and Fang, F. (2019). Perceptual learning and recognition confusion reveal the underlying relationships among the six basic emotions. Cognition Emotion 33, 754–767.

    Article  PubMed  Google Scholar 

  • Wicker, B., Keysers, C., Plailly, J., Royet, J.P., Gallese, V., and Rizzolatti, G. (2003). Both of us disgusted in my insula. Neuron 40, 655–664.

    Article  CAS  PubMed  Google Scholar 

  • Wong, W., Cabral, J., Rane, R., Ly, R., Kringelbach, M.L., and Feusner, J. D. (2021). Effects of visual attention modulation on dynamic functional connectivity during own-face viewing in body dysmorphic disorder. Neuropsychopharmacology 46, 2030–2038.

    Article  PubMed  PubMed Central  Google Scholar 

  • Wu, D., Li, X., and Jiang, T. (2020). Reconstruction of behavior-relevant individual brain activity: an individualized fMRI study. Sci China Life Sci 63, 410–418.

    Article  PubMed  Google Scholar 

  • Xu, P., Peng, S., Luo, Y.J., and Gong, G. (2021). Facial expression recognition: a meta-analytic review of theoretical models and neuroimaging evidence. Neurosci BioBehaval Rev 127, 820–836.

    Article  Google Scholar 

  • Yan, C.G., Wang, X.D., Zuo, X.N., and Zang, Y.F. (2016). DPABI: data processing & analysis for (resting-state) brain imaging. Neuroinformatics 14, 339–351.

    Article  PubMed  Google Scholar 

  • Zhang, G., Yu, M., Liu, Y.J., Zhao, G., Zhang, D., and Zheng, W. (2021). SparseDGCNN: recognizing emotion from multichannel EEG signals. IEEE Trans Affective Comput, doi: https://doi.org/10.1109/TAFFC.2021.3051332.

  • Zhang, H., Japee, S., Nolan, R., Chu, C., Liu, N., and Ungerleider, L.G. (2016). Face-selective regions differ in their ability to classify facial expressions. NeuroImage 130, 77–90.

    Article  PubMed  Google Scholar 

  • Zhang, Y., Padmanabhan, A., Gross, J.J., and Menon, V. (2019). Development of human emotion circuits investigated using a big-data analytic approach: stability, reliability, and robustness. J Neurosci 39, 7155–7172.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Zhuang, X., Yang, Z., Mishra, V., Sreenivasan, K., Bernick, C., and Cordes, D. (2020). Single-scale time-dependent window-sizes in sliding-window dynamic functional connectivity analysis: A validation study. NeuroImage 220, 117111.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China (31930053), the National Science and Technology Innovation 2030 Major Program (2022ZD0204802), Beijing Academy of Artificial Intelligence (BAAI), Project funded by China Postdoctoral Science Foundation (2022M710210), and the Fundamental Research Funds for the Central Universities (2021FZZX001-06). We also thank Xin-Yue Yang and Ruolin Yang for giving ideas during writing paper.

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Yingying Wang or Fang Fang.

Additional information

Compliance and ethics

The author(s) declare that they have no conflict of interest.

Electronic Supplementary Material

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, C., Wang, Y., Sun, X. et al. Decoding six basic emotions from brain functional connectivity patterns. Sci. China Life Sci. 66, 835–847 (2023). https://doi.org/10.1007/s11427-022-2206-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11427-022-2206-3

Keywords

Navigation