Communications inspired linear discriminant analysis

Thumbnail Image



Journal Title

Journal ISSN

Volume Title

Repository Usage Stats



We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label. By harnessing a recent theoretical result on the gradient of mutual information, the above optimization problem can be solved directly using gradient descent, without requiring simplification of the objective function. Theoretical analysis and empirical comparison are made between the proposed method and two closely related methods, and comparisons are also made with a method in which Rényi entropy is used to define the mutual information (in this case the gradient may be computed simply, under a special parameter setting). Relative to these alternative approaches, the proposed method achieves promising results on real datasets. Copyright 2012 by the author(s)/owner(s).








Robert Calderbank

Charles S. Sydnor Distinguished Professor of Computer Science

Robert Calderbank is Director of the Information Initiative at Duke University, where he is Professor of Electrical Engineering, Computer Science and Mathematics. He joined Duke in 2010, completed a 3 year term as Dean of Natural Sciences in August 2013, and also served as Interim Director of the Duke Initiative in Innovation and Entrepreneurship in 2012. Before joining Duke he was Professor of Electrical Engineering and Mathematics at Princeton University where he also directed the Program in Applied and Computational Mathematics.

Before joining Princeton University Dr. Calderbank was Vice President for Research at AT&T. As Vice President for Research he managed AT&T intellectual property, and he was responsible for licensing revenue. AT&T Labs was the first of a new type of research lab where masses of data generated by network services became a giant sandbox in which fundamental discoveries in information science became a source of commercial advantage

At Duke, Dr. Calderbank works with researchers from the Duke Center for Autism and Brain Development, developing information technology that is able to capture a full spectrum of behavior in very young children. By supporting more consistent and cost-effective early diagnosis, the team is increasing the opportunity for early interventions that have proven very effective.

At the start of his career at Bell Labs, Dr. Calderbank developed voiceband modem technology that was widely licensed and incorporated in over a billion devices. Voiceband means the signals are audible so these modems burped and squeaked as they connected to the internet. One of these products was the AT&T COMSPHERE® modem which was the fastest modem in the world in 1994 – at 33.6kb/s!   

Together with Peter Shor and colleagues at AT&T Labs Dr. Calderbank developed the group theoretic framework for quantum error correction. This framework changed the way physicists view quantum entanglement, and provided the foundation for fault tolerant quantum computation.

Dr. Calderbank has also developed technology that improves the speed and reliability of wireless communication by correlating signals across several transmit antennas. Invented in 1996, this space-time coding technology has been incorporated in a broad range of 3G, 4G and 5G wireless standards. He served on the Technical Advisory Board of Flarion Technologies a wireless infrastructure company founded by Rajiv Laroia and acquired by Qualcomm for $1B in 2008.

Dr. Calderbank is an IEEE Fellow and an AT&T Fellow, and he was elected to the National Academy of Engineering in 2005. He received the 2013 IEEE Hamming Medal for contributions to coding theory and communications and the 2015 Shannon Award.


Lawrence Carin

Professor of Electrical and Computer Engineering

Lawrence Carin earned the BS, MS, and PhD degrees in electrical engineering at the University of Maryland, College Park, in 1985, 1986, and 1989, respectively. In 1989 he joined the Electrical Engineering Department at Brooklyn Polytechnic Institute (now part of NYU) as an Assistant Professor, and became an Associate Professor there in 1994. In September 1995 he joined the Electrical and Computer Engineering (ECE) Department at Duke University, where he is now a Professor. He was ECE Department Chair from 2011-2014, and Vice Provost and Vice President for Research from 2014-2020. He was the Provost at King Abdullah University of Science & Technology (KAUST) from 2020-2023, returning to Duke in 2023. From 2003-2014 he held the William H. Younger Distinguished Professorship, and since 2018 he has held the James L. Meriam Distinguished Professorship. Dr. Carin's research focuses on machine learning (ML) and artificial intelligence (AI). He publishes widely in the main ML/AI forums, and has addressed many applications of AI, including in  medicine and security. He was co-founder of the small business Signal Innovations Group, which was acquired by BAE Systems in 2014, and in 2017 he co-founded the company Infinia ML, which was acquired by Aspirion in 2023. He is an IEEE Fellow.

Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.