Flexible, Real-Time Shaping of Zebra Finch Vocal Learning

Abstract

Male zebra finches learn to produce a single, highly stereotyped song and maintain this song over the course of their adult lives using auditory feedback. Such continuous production and evaluation of song is commonly conceptualized as a variant of actor-critic reinforcement learning, requiring the precise coordination of dozens of muscles with millisecond precision. Much of what we know of the underlying learning process comes from studies of adult male zebra finches adapting to white noise feedback, an auditory error signal, triggered by either high- or low-pitch variants of harmonic stack syllables — a single static perturbation of a single, simple syllable of a crystallized song. However, zebra finch song is spectrally and temporally rich, with numerous degrees of freedom, and the dynamic learning process must tackle this complexity. Thus, to more flexibly probe the full range of this complexity and characterize learning, new methods are needed for adaptively intervening in the learning process. To this end, I developed a pipeline to quantify and selectively manipulate song features within the high-dimensional song space in real time. Using a software platform for custom adaptive experimentation, improv, I acquired raw audio from adult male zebra finches as they practiced their songs, computed spectrograms, and encoded and classified them using a pretrained variational autoencoder, an unsupervised representation learning method, augmented with a supervised classification layer. The resulting latent representations can then be used to flexibly trigger feedback. Analysis can be performed in under 10 ms per 120 ms of song, and the delay between data acquisition and feedback to the bird is well within behaviorally and physiologically relevant timescales. As a result, this pipeline can be used to study adaptations of song in response to algorithmically guided perturbations, allowing us to test reinforcement learning hypotheses in a high-dimensional system. I then conducted in silico experiments using publicly available data to validate pipelines, and replicated and extended directed auditory feedback experiments to test hypotheses of vocal flexibility. Overall, I have developed a novel method for shaping vocal learning in real time and have demonstrated this method can be used to conduct novel experiments, providing a crucial tool for refined testing of learning algorithms.