Refine
Year of publication
- 2024 (2) (remove)
Document Type
- Article (2)
Language
- English (2)
Has Fulltext
- yes (2)
Is part of the Bibliography
- no (2)
Keywords
- Memory (2) (remove)
Institute
Residual connections have been proposed as an architecture-based inductive bias to mitigate the problem of exploding and vanishing gradients and increased task performance in both feed-forward and recurrent networks (RNNs) when trained with the backpropagation algorithm. Yet, little is known about how residual connections in RNNs influence their dynamics and fading memory properties. Here, we introduce weakly coupled residual recurrent networks (WCRNNs) in which residual connections result in well-defined Lyapunov exponents and allow for studying properties of fading memory. We investigate how the residual connections of WCRNNs influence their performance, network dynamics, and memory properties on a set of benchmark tasks. We show that several distinct forms of residual connections yield effective inductive biases that result in increased network expressivity. In particular, those are residual connections that (i) result in network dynamics at the proximity of the edge of chaos, (ii) allow networks to capitalize on characteristic spectral properties of the data, and (iii) result in heterogeneous memory properties. In addition, we demonstrate how our results can be extended to non-linear residuals and introduce a weakly coupled residual initialization scheme that can be used for Elman RNNs.
Beyond well-established difficulties with working memory in individuals with attention deficit hyperactivity disorder (ADHD), evidence is emerging that other memory processes may also be affected. We investigated, first, which memory processes show differences in adults and adolescents with ADHD in comparison to control participants, focusing on working and short-term memory, initial learning, interference, delayed and recognition memory. Second, we investigated whether ADHD severity, co-occurring depressive symptoms, IQ and physical fitness are associated with the memory performance in the individuals with ADHD.
We assessed 205 participants with ADHD (mean age 25.8 years, SD 7.99) and 50 control participants (mean age 21.1 years, SD 5.07) on cognitive tasks including the digit span forward (DSF) and backward (DSB), the Rey Auditory Verbal Learning Test (RAVLT), and the vocabulary and matrix reasoning subtests of the Wechsler Abbreviated Scale of Intelligence. Participants with ADHD were additionally assessed on ADHD severity, depression symptoms and cardiorespiratory fitness. A series of regressions were run, with sensitivity analyses performed when variables were skewed.
ADHD-control comparisons were significant for DSF, DSB, delayed and recognition memory, with people with ADHD performing less well than the control participants. The result for recognition memory was no longer significant in sensitivity analysis. Memory performance was not associated with greater ADHD or depression symptoms severity. IQ was positively associated with all memory variables except DSF. Cardiorespiratory fitness was negatively associated with the majority of RAVLT variables.
Individuals with ADHD showed difficulties with working memory, short-term memory and delayed memory, as well as a potential difficulty with recognition memory, despite preserved initial learning.