Automatic word count estimation from daylong child-centered recordings in various language environments using language-independent syllabification of speech

Abstract

© 2019 The Authors Automatic word count estimation (WCE) from audio recordings can be used to quantify the amount of verbal communication in a recording environment. One key application of WCE is to measure language input heard by infants and toddlers in their natural environments, as captured by daylong recordings from microphones worn by the infants. Although WCE is nearly trivial for high-quality signals in high-resource languages, daylong recordings are substantially more challenging due to the unconstrained acoustic environments and the presence of near- and far-field speech. Moreover, many use cases of interest involve languages for which reliable ASR systems or even well-defined lexicons are not available. A good WCE system should also perform similarly for low- and high-resource languages in order to enable unbiased comparisons across different cultures and environments. Unfortunately, the current state-of-the-art solution, the LENA system, is based on proprietary software and has only been optimized for American English, limiting its applicability. In this paper, we build on existing work on WCE and present the steps we have taken towards a freely available system for WCE that can be adapted to different languages or dialects with a limited amount of orthographically transcribed speech data. Our system is based on language-independent syllabification of speech, followed by a language-dependent mapping from syllable counts (and a number of other acoustic features) to the corresponding word count estimates. We evaluate our system on samples from daylong infant recordings from six different corpora consisting of several languages and socioeconomic environments, all manually annotated with the same protocol to allow direct comparison. We compare a number of alternative techniques for the two key components in our system: speech activity detection and automatic syllabification of speech. As a result, we show that our system can reach relatively consistent WCE accuracy across multiple corpora and languages (with some limitations). In addition, the system outperforms LENA on three of the four corpora consisting of different varieties of English. We also demonstrate how an automatic neural network-based syllabifier, when trained on multiple languages, generalizes well to novel languages beyond the training data, outperforming two previously proposed unsupervised syllabifiers as a feature extractor for WCE.

Department

Description

Provenance

Citation

Published Version (Please cite this version)

10.1016/j.specom.2019.08.005

Publication Info

Räsänen, O, S Seshadri, J Karadayi, E Riebling, J Bunce, A Cristia, F Metze, M Casillas, et al. (2019). Automatic word count estimation from daylong child-centered recordings in various language environments using language-independent syllabification of speech. Speech Communication, 113. pp. 63–80. 10.1016/j.specom.2019.08.005 Retrieved from https://hdl.handle.net/10161/19710.

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.

Scholars@Duke

Bergelson

Elika Bergelson

Associate Research Professor of Psychology and Neuroscience

Dr. Bergelson's lab has moved to Harvard Psychology; she retains an unremunerated research appointment at Duke through mid-2024 for logistical reasons. She formerly accepted PhD applicants through the Developmental and Cog/CogNeuro areas of P&N and the CNAP program.

In my research, I try to understand the interplay of processes during language acquisition.
In particular, I am interested in how word learning relates to other aspects of learning language (e.g. speech sound acquisition, grammar/morphology learning), and social/cognitive development more broadly (e.g. joint attention processes) in the first few years of life.

I pursue these questions using three main approaches: in-lab measures of early comprehension and production (eye-tracking, looking-time, and in EEG studies in collaboration with the Woldorff lab), and at-home measures of infants' linguistic and social environment (as in the SEEDLingS project).

More recently the lab is branching out to look at a wider range of human populations and at infants who are blind or deaf/heard of hearing.


Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.