DCFNet: Deep Neural Network with Decomposed Convolutional Filters

Loading...
Thumbnail Image

Date

2018-01-01

Journal Title

Journal ISSN

Volume Title

Repository Usage Stats

166
views
74
downloads

Abstract

©35th International Conference on Machine Learning, ICML 2018.All Rights Reserved. Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.

Department

Description

Provenance

Citation

Scholars@Duke

Calderbank

Robert Calderbank

Charles S. Sydnor Distinguished Professor of Computer Science

Robert Calderbank is Director of the Information Initiative at Duke University, where he is Professor of Electrical Engineering, Computer Science and Mathematics. He joined Duke in 2010, completed a 3 year term as Dean of Natural Sciences in August 2013, and also served as Interim Director of the Duke Initiative in Innovation and Entrepreneurship in 2012. Before joining Duke he was Professor of Electrical Engineering and Mathematics at Princeton University where he also directed the Program in Applied and Computational Mathematics.

Before joining Princeton University Dr. Calderbank was Vice President for Research at AT&T. As Vice President for Research he managed AT&T intellectual property, and he was responsible for licensing revenue. AT&T Labs was the first of a new type of research lab where masses of data generated by network services became a giant sandbox in which fundamental discoveries in information science became a source of commercial advantage

At Duke, Dr. Calderbank works with researchers from the Duke Center for Autism and Brain Development, developing information technology that is able to capture a full spectrum of behavior in very young children. By supporting more consistent and cost-effective early diagnosis, the team is increasing the opportunity for early interventions that have proven very effective.

At the start of his career at Bell Labs, Dr. Calderbank developed voiceband modem technology that was widely licensed and incorporated in over a billion devices. Voiceband means the signals are audible so these modems burped and squeaked as they connected to the internet. One of these products was the AT&T COMSPHERE® modem which was the fastest modem in the world in 1994 – at 33.6kb/s!   

Together with Peter Shor and colleagues at AT&T Labs Dr. Calderbank developed the group theoretic framework for quantum error correction. This framework changed the way physicists view quantum entanglement, and provided the foundation for fault tolerant quantum computation.

Dr. Calderbank has also developed technology that improves the speed and reliability of wireless communication by correlating signals across several transmit antennas. Invented in 1996, this space-time coding technology has been incorporated in a broad range of 3G, 4G and 5G wireless standards. He served on the Technical Advisory Board of Flarion Technologies a wireless infrastructure company founded by Rajiv Laroia and acquired by Qualcomm for $1B in 2008.

Dr. Calderbank is an IEEE Fellow and an AT&T Fellow, and he was elected to the National Academy of Engineering in 2005. He received the 2013 IEEE Hamming Medal for contributions to coding theory and communications and the 2015 Shannon Award.

Sapiro

Guillermo Sapiro

James B. Duke Distinguished Professor of Electrical and Computer Engineering

Guillermo Sapiro received his B.Sc. (summa cum laude), M.Sc., and Ph.D. from the Department of Electrical Engineering at the Technion, Israel Institute of Technology, in 1989, 1991, and 1993 respectively. After post-doctoral research at MIT, Dr. Sapiro became Member of Technical Staff at the research facilities of HP Labs in Palo Alto, California. He was with the Department of Electrical and Computer Engineering at the University of Minnesota, where he held the position of Distinguished McKnight University Professor and Vincentine Hermes-Luh Chair in Electrical and Computer Engineering. Currently he is the Edmund T. Pratt, Jr. School Professor with Duke University.

G. Sapiro works on theory and applications in computer vision, computer graphics, medical imaging, image analysis, and machine learning. He has authored and co-authored over 300 papers in these areas and has written a book published by Cambridge University Press, January 2001.

G. Sapiro was awarded the Gutwirth Scholarship for Special Excellence in Graduate Studies in 1991,  the Ollendorff Fellowship for Excellence in Vision and Image Understanding Work in 1992,  the Rothschild Fellowship for Post-Doctoral Studies in 1993, the Office of Naval Research Young Investigator Award in 1998,  the Presidential Early Career Awards for Scientist and Engineers (PECASE) in 1998, the National Science Foundation Career Award in 1999, and the National Security Science and Engineering Faculty Fellowship in 2010. He received the test of time award at ICCV 2011. He was elected to the American Academy of Arts and Sciences on 2018.

G. Sapiro is a Fellow of IEEE and SIAM.

G. Sapiro was the founding Editor-in-Chief of the SIAM Journal on Imaging Sciences.


Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.