Automatic word count estimation from daylong child-centered recordings in various language environments using language-independent syllabification of speech
Abstract
© 2019 The Authors Automatic word count estimation (WCE) from audio recordings can
be used to quantify the amount of verbal communication in a recording environment.
One key application of WCE is to measure language input heard by infants and toddlers
in their natural environments, as captured by daylong recordings from microphones
worn by the infants. Although WCE is nearly trivial for high-quality signals in high-resource
languages, daylong recordings are substantially more challenging due to the unconstrained
acoustic environments and the presence of near- and far-field speech. Moreover, many
use cases of interest involve languages for which reliable ASR systems or even well-defined
lexicons are not available. A good WCE system should also perform similarly for low-
and high-resource languages in order to enable unbiased comparisons across different
cultures and environments. Unfortunately, the current state-of-the-art solution, the
LENA system, is based on proprietary software and has only been optimized for American
English, limiting its applicability. In this paper, we build on existing work on WCE
and present the steps we have taken towards a freely available system for WCE that
can be adapted to different languages or dialects with a limited amount of orthographically
transcribed speech data. Our system is based on language-independent syllabification
of speech, followed by a language-dependent mapping from syllable counts (and a number
of other acoustic features) to the corresponding word count estimates. We evaluate
our system on samples from daylong infant recordings from six different corpora consisting
of several languages and socioeconomic environments, all manually annotated with the
same protocol to allow direct comparison. We compare a number of alternative techniques
for the two key components in our system: speech activity detection and automatic
syllabification of speech. As a result, we show that our system can reach relatively
consistent WCE accuracy across multiple corpora and languages (with some limitations).
In addition, the system outperforms LENA on three of the four corpora consisting of
different varieties of English. We also demonstrate how an automatic neural network-based
syllabifier, when trained on multiple languages, generalizes well to novel languages
beyond the training data, outperforming two previously proposed unsupervised syllabifiers
as a feature extractor for WCE.
Type
Journal articleSubject
Science & TechnologyTechnology
Acoustics
Computer Science, Interdisciplinary Applications
Computer Science
Language acquisition
Word count estimation
Automatic syllabification
Daylong recordings
Noise robustness
SYSTEM
SEGMENTATION
RELIABILITY
LENA(TM)
Permalink
https://hdl.handle.net/10161/19710Published Version (Please cite this version)
10.1016/j.specom.2019.08.005Publication Info
Räsänen, O; Seshadri, S; Karadayi, J; Riebling, E; Bunce, J; Cristia, A; ... Soderstrom,
M (2019). Automatic word count estimation from daylong child-centered recordings in various
language environments using language-independent syllabification of speech. Speech Communication, 113. pp. 63-80. 10.1016/j.specom.2019.08.005. Retrieved from https://hdl.handle.net/10161/19710.This is constructed from limited available data and may be imprecise. To cite this
article, please review & use the official citation provided by the journal.
Collections
More Info
Show full item recordScholars@Duke
Elika Bergelson
Associate Research Professor of Psychology and Neuroscience
Dr. Bergelson accepts PhD applicants through the Developmental and Cog/CogNeuro areas
of P&N and the CNAP program.In my research, I try to understand the interplay of processes
during language acquisition. In particular, I am interested in how word learning relates
to other aspects of learning language (e.g. speech sound acquisition, grammar/morphology
learning), and social/cognitive development more broadly (e.g. joint attention processes)
in the first few

Articles written by Duke faculty are made available through the campus open access policy. For more information see: Duke Open Access Policy
Rights for Collection: Scholarly Articles
Works are deposited here by their authors, and represent their research and opinions, not that of Duke University. Some materials and descriptions may include offensive content. More info