Audiology & Neuroscience | July 2018 Hearing Review

Cognitive load and listening effort may be the key to future improvements in CI development

 Cochlear implants (CIs) have helped restore auditory perception in an estimated 400,000+ patients worldwide. Thus, CIs constitute the most successful neural prosthesis in use.

This article examines listening effort, auditory working memory, speech-in-noise comprehension, and the large network of interconnected brain areas now known as the “listening connectome.” As research continues to shed light on this area, it becomes increasingly apparent that traditional speech perception measures in quiet are insufficient for evaluating the effectiveness of many of the latest advances in CI technology.      

Modern multi-electrode cochlear implant systems encode sounds by translating acoustic information into amplitude-modulated electrical pulse trains delivered to the auditory nerve at a fixed stimulation rate. The restitution of spectral cues is realized by mapping different frequency bands extracted from input sounds by a filter bank onto a reduced number of individual channels (12 to 22 depending on manufacturer). Each channel corresponds to an electrode, surgically positioned inside the cochlea, taking advantage of its natural tonotopic organization (high frequencies are mapped onto electrodes at the basal end, low frequencies onto apical electrodes). Sound intensity is coded by mapping the energy level in each frequency-band onto the amount of charge delivered at each individual electrode, either by increasing the intensity or the duration of the electrical pulse. Stimulation is, in general, delivered in an interleaved sequential fashion, avoiding the stimulation of two neighboring electrodes at the same time, thereby limiting vector summation of electrical current between electrodes and significantly improving spectral channel independence and spectral resolution.1

By leveraging these general principles, patients with severe-to-profound hearing loss or congenital deafness can recover or acquire hearing abilities, and adults with post-lingual deafness or early-implanted children often reach 100% open-set speech perception in quiet with their CI.2,3 Such patient outcomes ensure that CIs constitute a cost-effective treatment for severe-to-profound hearing loss.4,5 Furthermore, based on successful outcomes, CI candidacy has been progressively widened to include elderly patients,patients with functional residual hearing, and patients with unilateral hearing loss.7,8

Despite these optimistic observations, the development of cochlear implant systems still faces important limitations—mostly related to the reductions imposed on natural sounds by the constraints of direct electrical neural modulation of surviving inner-ear ganglion cells. Primarily, electrical interactions inside the cochlea limit the number of independent spectral channels accessible to most CI users.In addition, the significantly reduced electrical dynamic range (10-30 dB) necessitates strong compression and the use of fixed stimulation rates, which currently limit the performance of CIs.

These limitations are associated with important individual variability in outcomes,10and reduced user benefits when it comes to perceiving sounds in acoustically adverse environments (eg, noisy or reverberant environments) or when processing music. Indeed, the very sparse rendering of sounds by CIs is sufficient to allow excellent speech recognition in quiet, but it is limited when it comes to perceiving speech in noise or music.

From Ear-Hearing to the “Listening Connectome:” Cognitive Hearing

Historically, the development of CI systems has primarily followed one primary goal: restoring speech perception abilities in patients deprived of hearing. However, as theorized by Arlinger and colleagues in the framework of modern “cognitive hearing science,”11 there is much more to speech perception than hearing sounds. Indeed, in 2018 (and as amply demonstrated in this special edition of The Hearing Review), speech understanding should be envisaged with respect to interactions with other cognitive functions.

Interestingly, in this context, the first studies that analyzed the brain responses associated with speech-in-noise perception—including the research of Obleser and Kotz, Scott and McGettigan, and Zekveld et al12-15— revealed that the difference between listening in quiet and listening in noise was a recruitment of a large number of neurologically interconnected areas when speech perception was made more difficult. This large fronto-temporal network shows important overlap with brain areas associated with increased listening effort16or auditory working memory.17

These studies highlight the functional proximity between listening effort, auditory working memory, and speech-in-noise comprehension, embodied in one large network of interconnected brain areas, now referred to as the listening connectome.18 This network represents brain regions engaged in cognitive functions, such as executive functions, selective attention, long-term memory, and working memory.

As noted above, this network is implicated in effortful listening, or the amount of functional connectivity and extent to which the network recruitment/engagement seems to be related to the complexity of the task performed with speech information. The idea was summarized by Rönnberg and colleagues in a unique model which takes into account sensory-cognitive interactions in the perception and comprehension of speech: the Ease of Language Understanding (ELU) model.19-21 The initial proposition assumed an interaction between the quality of the sensory input, information available in long-term memory and the supplementary help of working memory. These three components would determine the difficulty with which speech can be understood in challenging listening conditions. Successful listening ultimately relies on an ensemble of interconnected cognitive processing steps, constrained by the allocation of a finite amount of cognitive resource or attention, mediated by executive processes and working memory.

Executive functions (EF) are a set of high cognitive abilities used to control and regulate other functions and behaviors. EF may involve different abilities, such as the capacity to selectively focus on specific information in the environment (ie, selective attention), sustaining attention on a specific information source (ie, sustained attention), resistance to interfering information (ie, inhibition), and adapting behavior to changes in the environment (ie, mental flexibility).

Working memory (WM) corresponds to the active maintenance and manipulation of a small amount of information in memory to serve a specific task. It is usually divided into different subsystems corresponding to different types of information or modalities, such as visual-, auditory-, or verbal-working memory (VWM). The WM function is intimately tied and even merged, at least partially, with executive functioning and attention.

A second aspect of the notion of the listening connectome (ie, a map of neural networks throughout the brain) is related to brain plasticity and the physiological mechanisms pertaining to the development of a functional listening connectome based on the availability of sensory information (see Kral et al18for a review). Deafess and other significant sensory deprivation disrupt the development of connections among multiple cognitive constituents. This disruption may well be the cause of important difficulties faced by CI users regarding the development of auditory learning and memory abilities, impacting auditory sequential memory, explicit or implicit.22

As such, CI users (especially prelingually-deaf children or adults with long durations of auditory deprivation) will experience difficulties in developing aspects of sound processing representing higher levels of integration. Although hearing perception can be restored, the auditory learning, memory, and executive processes may remain impaired in certain users. Alleviating listening effort should allow a more efficient “reconnection” of the network and learning of higher auditory processing abilities, including auditory sequence learning.

More specifically, a relationship between working memory and executive functioning and speech intelligibility appear in cochlear implant users, such that difficulties with tasks typically used to assess executive functioning like the Trail Making Test (TMT), for example, appear to be predictive of speech-in-noise perception difficulties in the non-implanted population.23 Pre-lingual deaf CI users show deficits of verbal working memory and auditory working memory, which correlate with deprivation duration and could explain perhaps 20% of the variances observed in speech perception outcomes.24-30

Other aspects of cognitive hearing are impacted in CI users, such as auditory sequence learning31 or executive functioning.32-36 As a result, deaf children with CIs can experience difficulties relying on contextual information to ease speech perception37and non-literal language comprehension or complex relational concepts’ formation.38 These deficits remain even after long-term CI use compared to normal-hearing (NH), normal-developing children.39-40

Listening Effort in CI Users: A Review of the Scientific Literature

The notion of listening effort (LE) is associated with difficult listening conditions, such as understanding speech in noise in the presence of hearing loss due to aging. LE is also a foundational issue associated with cochlear implants, and LE is a core challenge for CI users with respect to their programmed stimulation mode, reduced spectral detail, and reduced dynamic range.

CIs deliver signals which require higher levels of LE, even for tasks associated with low levels of effort for normal-hearing listeners. Moreover, the difficulties experienced by CI users with speech-in-noise comprehension or emotional judgments on music fit directly into the framework of listening effort.

The following is a review of literature presenting evidence relating listening effort and cochlear implants.

Measuring listening effort in CI users. Listening effort is better evaluated by secondary tasks (concurrent tasks) or by objective measures such as pupil dilation16,41-48 and alpha-band oscillations in EEG,49 than by a direct measure of intelligibility or subjective measures of effort. Pals and colleagues50 used the dual-task paradigm to show the relationship between spectral resolution and listening effort in a vocoder CI model. They found that LE increases with decreasing spectral resolution. They noted that changes in LE are better reflected in objective measures of LE, such as response times (RTs) on a secondary task, rather than intelligibility scores or subjective effort measures. Their dual-RT task, as described in a subsequent paper,51 has the advantage of being easy to set up in laboratories or clinics without further objective testing material. This test relies on two RT measures performed together, one in the auditory and one in the visual modality.

Winn et al52 used pupil dilation to measure LE associated with spectral detail in normal-hearing subjects presented with simulated CI processing. They found that LE increased with degraded spectral resolution.

Until recently, few direct measures of listening effort in CI users have been performed. In 2013, Hughes et al53 used a dual-task paradigm to show that bilateral CI users required less listening effort than unilateral CI users when asked to repeat consonant-vowel-consonant (CVC) words presented in noise. In 2015, Steel et al54 studied binaural fusion in NH and CI users using different measures. They measured LE evaluated by pupil dilation and showed that, without binaural level cues, children had difficulty fusing input from their bilateral implants to perceive one sound, which increased LE.

LE can be evaluated using simulations of signal processing strategies in NH or HL patients using objective measures, such as pupillometry or behavioral measures like a memory test (eg, SWIR test or dual-task paradigms). Very few studies have directly evaluated LE in CI users with adapted methods (other than speech perception scores in different listening conditions).

Special populations: Pediatric issues. BrainHearingTM as applied to CIs necessitates offering clear and highly intelligible sound to alleviate sensory/perceptual cognitive effort specifically to leave cognitive resources available for the development of higher cognitive functions including VWM and EF.

Special populations: Adult issues. To avoid cognitive fatigue and allow for a deeper understanding of language, sound perception should remain effort-free or require only low effort. VWM may either serve as a predictor, metric, or as a rehabilitation tool accompanying CI. For the elderly, BrainHearingTM includes using a CI system offering effort-free listening to better counter the effects of aging and avoid or minimize cognitive decline.

Special populations: Bilateral CIs. If it can be conclusively shown that using bilateral CIs requires significantly less LE than unilateral CI fittings, it is likely more bilateral fittings would be recommended, and CI recipients would have an easier time attending to speech sounds. Further, for children who have not developed binaural listening ability prior to cochlear implantation, acquiring these same binaural listening abilities is more difficult post-implantation.54

Listening effort and rehabilitation. For speech perception in noise, cognitive training may provide benefit. Using non-speech maskers, Ingvalson et al55 showed that for young, normal-hearing, cognitively healthy listeners, memory training was associated with improved speech perception in noise as compared to untrained controls. However, the authors noted the study has major limitations, as the control group was non-contact (ie, they didn’t spend similar time in the lab like the trained listeners) and, even more crucially, the training paradigm contained a speech-perception-in-noise component. A second study performed with WM training and speech-in-noise perception by Wayne et al56 focused on the older adult population (age 59+ years), but found no evidence that cognitive training improved speech-in-noise comprehension or cognitive functioning.

For CI users, results seem more promising. Kronenberger and colleagues57 reported that working memory training may be beneficial to children with cochlear implants. Ingvalson et al58 used auditory cognitive games to improve language performance in these children (see Ingvalson & Wong59 for a discussion on auditory cognitive hearing). Some aspects of post-implantation rehabilitation and training could be viewed in the framework of cognitive hearing, if they rely, for example, on auditory working memory tasks or executive function processing and/or auditory sequence games, etc.

Conclusion

Despite the successes of CIs, it can be argued that no other advancement has resulted in a comparable increase in outcome since moving from single- to multi-channel devices. Furthermore, traditional speech-perception measures in quiet are not a sensitive enough measure to evaluate the full benefit of many of the latest advancements in CI technology, such as noise suppression, adaptive directionality, and dynamic range management. Going forward, by focusing on quantifying the impact of technologies on cognitive load and listening effort, it may be possible to realize a significant leap forward in outcomes for CI recipients.

Biography: Edward Overstreet, PhD, and Michel Hoen, PhD, are researchers in the Department of Clinical and Scientific Research at Oticon Medical in Vallauris, France.

Citation for this article: Overstreet E, Hoen M. Cochlear implants: Considerations regarding the relationship between cognitive load management and outcome. Hearing Review. 2018;25(7):33-35.

Correspondence can be addressed to Dr Overstreet at: [email protected]

References 

  1. Wilson BS, Finley CC, Lawson DT, Wolford RD, Eddington DK, Rabinowitz WM. Better speech recognition with cochlear implants. Nature. 1991; 352:236-238.

  2. Lazard DS, Vincent C, Venail F, et al. Pre-, per- and postoperative factors affecting performance of postlinguistically deaf adults using cochlear implants: A new conceptual model over time. PLoS One. 2012;7(11):e48739.

  3. Davidson LS, Geers AE, Blamey PJ, Tobey EA, Brenner CA. Factors contributing to speech perception scores in long-term pediatric cochlear implant users. Ear Hear. 2011;32[Suppl 1]:19S-26S.

  4. McKinnon BJ. Cost effectiveness of cochlear implants. Curr Opin Otolaryngol Head Neck Surg.2014;22(5):344-348.

  5. Saunders JE, Francis HW, Skarzynski PH. Measuring success: Cost-effectiveness and expanding access to cochlear implantation. Otol Neurotol. 2016;37(2):e135-40.

  6. Aimoni C, Ciorba A, Hatzopoulos S, et al. Cochlear implants in subjects over age 65: Quality of life and audiological outcomes. Med Sci Monit. 2016; 28(22):3035-3042.

  7. Boyd PJ. Potential benefits from cochlear implantation of children with unilateral hearing loss. Cochlear Implants Int. 2015;16(3):121-136.

  8. van Zon A, Peters JPM, Stegeman I, Smit AL, Grolman W. Cochlear implantation for patients with single-sided deafness or asymmetrical hearing loss: a systematic review of the evidence. Otol Neurotol.2015;36(2):209-219.

  9. Friesen LM, Shannon RV, Baskent D, Wang X. Speech recognition in noise as a function of the number of spectral channels: Comparison of acoustic hearing and cochlear implants. J Acoust Soc Am. 2001;110:1150.

  10. Firszt JB, Holden LK, Skinner MW, et al. Recognition of speech presented at soft to loud levels by adult cochlear implant recipients of three cochlear implant systems. Ear Hear. 2004;25(4):375-387.

  11. Arlinger S, Lunner T, Lyxell B, Pichora-Fuller MK. The emergence of cognitive hearing science. Scand J Psychol. 2009;50(5):371-384.

  12. Obleser J, Kotz SA. Expectancy constraints in degraded speech modulate the language comprehension network. Cereb Cortex.2010;20(3):633-640.

  13. Obleser J, Kotz SA. Multiple brain signatures of integration in the comprehension of degraded speech. Neuroimage.2011;55(2):713-723. doi: 10.1016/j.neuroimage.2010.12.020.

  14. Scott SK, McGettigan C. The neural processing of masked speech. Hear Res.2013;303:58-66.

  15. Zekveld AA, Heslenfeld DJ, Festen JM, Schoonhoven R. Top-down and bottom-up processes in speech comprehension. Neuroimage.2006; 32(4):1826-36.

  16. Zekveld AA, Heslenfeld DJ, Johnsrude IS, Versfeld NJ, Kramer SE. The eye as a window to the listening brain: Neural correlates of pupil size as a measure of cognitive listening load. Neuroimage. 2014;101:76-86.

  17. Kumar S, Joseph S, Gander PE, Barascud N, Halpern AR, Griffiths TD. A brain system for auditory working memory. J Neurosci.2016;20;36(16):4492-4505.

  18. Kral A, Kronenberger WG, Pisoni DB, O’Donoghue GM. Neurocognitive factors in sensory restoration of early deafness: a connectome model. Lancet Neurol. 2016; 15(6):610-621.

  19. Rönnberg J. Cognition in the hearing impaired and deaf as a bridge between signal and dialogue: A framework and a model. Int J Audiol.2003;42(S1):68–76.

  20. Rönnberg J, Rudner M, Foo C, Lunner T. Cognition counts: A working memory system for ease of language understanding (ELU). Int J Audiol.2008;47(S2):S99–S105.

  21. Rönnberg J, Lunner T, Zekveld A, et al. The Ease of Language Understanding (ELU) model: Theoretical, empirical, and clinical advances. Front Syst Neurosci. 2013;7(31):1-17.

  22. Pisoni DB, Kronenberger WG, Chandramouli SH, Conway CM. Learning and memory processes following cochlear implantation: The missing piece of the puzzle. Front Psychol. 2016;7(493):1-19.

  23. Ellis RJ, Munro KJ. Predictors of aided speech recognition, with and without frequency compression, in older adults. Int J Audiol. 2015;54(7)[Jul]:467-475.

  24. Dawson PW, Busby PA, McKay CM, Clark GM. Short-term auditory memory in children using cochlear implants and its relevance to receptive language. J Speech Lang Hear Res. 2002;45:789-801.

  25. Pisoni DB, Cleary. Measures of working memory span and verbal rehearsal speed in deaf children after cochlear implantation. Ear Hear. 2003;24(S1):106S-120S.

  26. Watson DR, Titterington J, Henry A, Toner JG. Auditory sensory memory and working memory processes in children with normal hearing and cochlear implants. Audiol Neurootol. 2007;12:65-76.

  27. Harris MS, Pisoni DB, Kronenberger WG, Gao S, Caffrey HM, Miyamoto RT. Developmental trajectories of forward and backward digit spans in deaf children with cochlear implants. Cochlear Implants Int. 2011;12(S1):S84-S88.

  28. Harris MS, Kronenberger WG, Gao S, Hoen HM, Miyamoto RT, Pisoni DB. Verbal short-term memory development and spoken language outcomes in deaf children with cochlear implants. Ear Hear. 2013;34(2]:179-192.

  29. Nittrouer S, Caldwell-Tarr A, Lowenstein JH. Working memory in children with cochlear implants: Problems are in storage, not processing. Int J Pediatr Otorhinolaryngol. 2013;77(11)[Nov]:1886-98.

  30. Bharadwaj SV, Maricle D, Green L, Allman T. Working memory, short-term memory and reading proficiency in school-age children with cochlear implants. Int J Pediatr Otorhinolaryngol. 2015;79(10):1647-1653.

  31. Conway CM, Pisoni DB, Anaya EM, Karpicke J, Henning SC. Implicit sequence learning in deaf children with cochlear implants. Dev Sci.2011;14(1):69-82.

  32. Beer J, Pisoni DB, Kronenberger W. Executive function in children with cochlear implants: The role of organizational-integrative processes. Volta Voices. 2009;16(3):18-21.

  33. Beer J, Kronenberger WG, Castellanos I, Colson BG, Henning SC, Pisoni DB. Executive functioning skills in preschool-age children with cochlear implants. J Speech Lang Hear Res. 2014;57:1521-1534.

  34. Kronenberger WG, Pisoni DB, Harris MS, Hoen HM, Xu H, Miyamoto RT. Profiles of verbal working memory growth predict speech and language development in chidren with cochlear implants. J Speech Lang Hear Res. 2013;56:805-825.

  35. Kronenberger WG, Beer J, Castellanos I, Pisoni DB, Miyamoto RT. Neurocognitive risk in children with cochlear implants. JAMA Otolaryngol Head Neck Surg.2014; 140(7):608-615.

  36. Park M, Song JJ, Oh SJ, Shin M-S, Lee JH, Oh SH. The relation between nonverbal IQ and postoperative CI outcomes in cochlear implant users: Preliminary result. Biomed Res Int. 2015:313274.

  37. Conway CM, Deocampo JA, Walk AM, Anaya EM, Pisoni DB. Deaf children with cochlear implants do not appear to use sentence context to help recognize spoken words. J Speech Lang Hear Res. 2014;57:2174-2190.

  38. Castellanos I, Kronenberger WG, Beer J, et al. Concept formation skills in long-term cochlear implant users. J Deaf Stud Deaf Educ.2015;20(1):27-40.

  39. Pisoni DB, Kronenberger WG, Roman AS, Geers AE. Measures of digit span and verbal rehearsal speed in deaf children after more than 10 years of cochlear implantation. Ear Hear.2011;32(1):60S-74S.

  40. AuBuchon AM, Pisoni DB, Kronenberger WG. Verbal processing speed and executive functioning in long-term cochlear implant users. J Speech Lang Hear Res.2015;58:151-162.

  41. Zekveld AA, Kramer SE, Festen JM. Pupil response as an indication of effortful listening: The influence of sentence intelligibility. Ear Hear. 2010;31(4):480-490.

  42. Zekveld AA, Kramer SE, Festen JM. Cognitive load during speech perception in noise: The influence of age, hearing loss, and cognition on the pupil response. Ear Hear.2011;32(4):498-510.

  43. Zekveld AA, Festen JM, Kramer SE. Task difficulty differentially affects two measures of processing load: The pupil response during sentence processing and delayed cued recall of the sentences. J Speech Lang Hear Res.2013;56(4):1156-1165.

  44. Koelewijn T, Zekveld AA, Festen JM, Kramer SE. Pupil dilation uncovers extra listening effort in the presence of a single-talker masker. Ear Hear. 2012;33(2):291-300.

  45. Koelewijn T, Shinn-Cunningham BG, Zekveld AA, Kramer SE. The pupil response is sensitive to divided attention during speech processing. Hear Res. 2014;312:114-20.

  46. Koelewijn T, de Kluiver H, Shinn-Cunningham BG, Zekveld AA, Kramer SE. The pupil response reveals increased listening effort when it is difficult to focus attention. Hear Res. 2015;323:81-90.

  47. Wagner A, Pals C, de Blecourt CM, Sarampalis A, Ba?kent D. Does signal degradation affect top-down processing of speech? Physiology, Psychoacoustics and Cognition in Normal and Impaired Hearing. 2016;894:297-306.

  48. Wang Y, Zekveld AA, Naylor G, et al. Parasympathetic nervous system dysfunction, as identified by pupil light reflex, and its possible connection to hearing impairment. PLoS One. 2016;11(4):e0153566.

  49. Wilsch A, Obleser J. What works in auditory working memory? A neural oscillations perspective. Brain Res. 2016;1640(Pt B):193-207.

  50. Pals C, Sarampalis A, Ba?kent D. Listening effort with cochlear implant simulations. J Speech Lang Hear Res. 2013;56:1075-1084.

  51. Pals C, Sarampalis A, van Rijn H, Ba?kent D. Validation of a simple response-time measure of listening effort. J Acoust Soc Am. 2015;138(3):EL187.

  52. Winn MB, Edwards JR, Litovsky RY. The impact of auditory spectral resolution on listening effort revealed by pupil dilation. Ear Hear.2015;36(4):e153-e165.

  53. Hughes KC, Galvin KL. Measuring listening effort expended by adolescents and young adults with unilateral or bilateral cochlear implants or normal hearing. Cochlear Implants Int. 2013;14(3):121-129.

  54. Steel MM, Papsin BC, Gordon KA. Binaural fusion and listening effort in children who use bilateral cochlear implants: A psychoacoustic and pupillometric study. PLoS One. 2015;10(10):e0141945.

  55. Ingvalson EM, Dhar S, Wong PC, Liu H. Working memory training to improve speech perception in noise across languages. J Acoust Soc Am. 2015;137(6)3477. 1

  56. Wayne RV, Hamilton C, Huyck JJ, Johnsrude IS. Working memory training and speech in noise comprehension in older adults. Front Aging Neurosci. 2016;8(49):1-15.

  57. Kronenberger WG, Pisoni DB, Henning SC, Colson BG, Hazzard LM. Working memory training for children with cochlear implants: A pilot study. J Speech Lang Hear Res. 2011;54:1182-1196.

  58. Ingvalson EM, Young NM, Wong PCM. Auditory-cognitive training improves language performance in prelingually deafened cochlear implant recipients. Int J Pediatr Otorhinolaryngol. 2014;78(10):1624-1631.

  59. Ingvalson EM, Wong PCM. Training to improve language outcomes in cochlear implant recipients. Front Psychol. 2013;4(263):1-9.

 

MORE ON “AUDIOLOGY & NEUROSCIENCE” from this special July 2018 edition of The Hearing Review:

Clinical Speech Audiometry in the Age of the AERP, by James Jerger, PhD.

Cortical Neuroplasticity in Hearing Loss: Why It Matters in Clinical Decision-Making for Children and Adults, by Anu Sharma, PhD, and Hannah Glick, AuD.

Effects of Amplification on Cortical Electrophysiological Function, by Sridhar Krishnamurti, PhD, and Larry Wise, AuD.

Cochlear implants: Considerations Regarding the Relationship between Cognitive Load Management and Outcome, by Edward Overstreet, PhD, and Michel Hoen, PhD.

Dementia Screening: A Role for Audiologists, by Douglas L. Beck, AuD, Barbara R. Weinstein, PhD, and Michael Harvey, PhD, ABPP.