University of Leicester
revision.pdf (239.67 kB)
Download file

Explainable Multimodal Deep Dictionary Learning to Capture Developmental Differences from Three fMRI Paradigms

Download (239.67 kB)
journal contribution
posted on 2023-03-08, 10:00 authored by L Yang, C Qiao, Huiyu Zhou, VD Calhoun, JM Stephen, TW Wilson, Y Wang

Objective : Multimodal-based methods show great potential for neuroscience studies by integrating complementary information. There has been less multimodal work focussed on brain developmental changes. Methods : We propose an explainable multimodal deep dictionary learning method to uncover both the commonality and specificity of different modalities, which learns the shared dictionary and the modality-specific sparse representations based on the multimodal data and their encodings of a sparse deep autoencoder. Results : By regarding three fMRI paradigms collected during two tasks and resting state as modalities, we apply the proposed method on multimodal data to identify the brain developmental differences. The results show that the proposed model can not only achieve better performance in reconstruction, but also yield age-related differences in reoccurring patterns. Specifically, both children and young adults prefer to switch among states during two tasks while staying within a particular state during rest, but the difference is that children possess more diffuse functional connectivity patterns while young adults have more focused functional connectivity patterns. Conclusion and Significance : To uncover the commonality and specificity of three fMRI paradigms to developmental differences, multimodal data and their encodings are used to train the shared dictionary and the modality-specific sparse representations. Identifying brain network differences helps to understand how the neural circuits and brain networks form and develop with age. 


National Natural Science Foundation of China (Grant Number: 12090021 and 12271429)

National Key Research and Development Program of China (Grant Number: 2020AAA0106302)

Natural Science Basic Research Program of Shaanxi (Grant Number: 2022JM-005)

National Institutes of Health (Grant Number: R01 MH104680, R01 GM109068, R01 MH121101, R01 MH116782, R01 MH118013 and P20-GM144641)


Author affiliation

School of Computing and Mathematical Sciences, University of Leicester


  • AM (Accepted Manuscript)

Published in

IEEE Transactions on Biomedical Engineering


Institute of Electrical and Electronics Engineers



Copyright date


Available date




Usage metrics

    University of Leicester Publications