DCFNet: Deep Neural Network with Decomposed Convolutional Filters
Date
2018-01-01
Authors
Journal Title
Journal ISSN
Volume Title
Repository Usage Stats
views
downloads
Abstract
©35th International Conference on Machine Learning, ICML 2018.All Rights Reserved. Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.
Type
Department
Description
Provenance
Citation
Permalink
Collections
Scholars@Duke
Robert Calderbank
Robert Calderbank is Director of the Information Initiative at Duke University, where he is Professor of Electrical Engineering, Computer Science and Mathematics. He joined Duke in 2010, completed a 3 year term as Dean of Natural Sciences in August 2013, and also served as Interim Director of the Duke Initiative in Innovation and Entrepreneurship in 2012. Before joining Duke he was Professor of Electrical Engineering and Mathematics at Princeton University where he also directed the Program in Applied and Computational Mathematics.
Before joining Princeton University Dr. Calderbank was Vice President for Research at AT&T. As Vice President for Research he managed AT&T intellectual property, and he was responsible for licensing revenue. AT&T Labs was the first of a new type of research lab where masses of data generated by network services became a giant sandbox in which fundamental discoveries in information science became a source of commercial advantage
At Duke, Dr. Calderbank works with researchers from the Duke Center for Autism and Brain Development, developing information technology that is able to capture a full spectrum of behavior in very young children. By supporting more consistent and cost-effective early diagnosis, the team is increasing the opportunity for early interventions that have proven very effective.
At the start of his career at Bell Labs, Dr. Calderbank developed voiceband modem technology that was widely licensed and incorporated in over a billion devices. Voiceband means the signals are audible so these modems burped and squeaked as they connected to the internet. One of these products was the AT&T COMSPHERE® modem which was the fastest modem in the world in 1994 – at 33.6kb/s!
Together with Peter Shor and colleagues at AT&T Labs Dr. Calderbank developed the group theoretic framework for quantum error correction. This framework changed the way physicists view quantum entanglement, and provided the foundation for fault tolerant quantum computation.
Dr. Calderbank has also developed technology that improves the speed and reliability of wireless communication by correlating signals across several transmit antennas. Invented in 1996, this space-time coding technology has been incorporated in a broad range of 3G, 4G and 5G wireless standards. He served on the Technical Advisory Board of Flarion Technologies a wireless infrastructure company founded by Rajiv Laroia and acquired by Qualcomm for $1B in 2008.
Dr. Calderbank is an IEEE Fellow and an AT&T Fellow, and he was elected to the National Academy of Engineering in 2005. He received the 2013 IEEE Hamming Medal for contributions to coding theory and communications and the 2015 Shannon Award.
Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.