Communications inspired linear discriminant analysis
Abstract
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic
viewpoint. The linear projection matrix is designed by maximizing the mutual information
between the projected signal and the class label. By harnessing a recent theoretical
result on the gradient of mutual information, the above optimization problem can be
solved directly using gradient descent, without requiring simplification of the objective
function. Theoretical analysis and empirical comparison are made between the proposed
method and two closely related methods, and comparisons are also made with a method
in which Rényi entropy is used to define the mutual information (in this case the
gradient may be computed simply, under a special parameter setting). Relative to these
alternative approaches, the proposed method achieves promising results on real datasets.
Copyright 2012 by the author(s)/owner(s).
Type
Journal articlePermalink
https://hdl.handle.net/10161/8956Collections
More Info
Show full item recordScholars@Duke
Robert Calderbank
Charles S. Sydnor Distinguished Professor of Computer Science
Robert Calderbank is Director of the Information Initiative at Duke University, where
he is Professor of Electrical Engineering, Computer Science and Mathematics. He joined
Duke in 2010, completed a 3 year term as Dean of Natural Sciences in August 2013,
and also served as Interim Director of the Duke Initiative in Innovation and Entrepreneurship
in 2012. Before joining Duke he was Professor of Electrical Engineering and Mathematics
at Princeton University where he also directed the Program i
Lawrence Carin
Professor of Electrical and Computer Engineering
Lawrence Carin earned the BS, MS, and PhD degrees in electrical engineering at the
University of Maryland, College Park, in 1985, 1986, and 1989, respectively. In 1989
he joined the Electrical Engineering Department at Polytechnic University (Brooklyn)
as an Assistant Professor, and became an Associate Professor there in 1994. In September
1995 he joined the Electrical and Computer Engineering (ECE) Department at Duke University,
where he is now a Professor. He was ECE Department Chair from 2011
Alphabetical list of authors with Scholars@Duke profiles.

Articles written by Duke faculty are made available through the campus open access policy. For more information see: Duke Open Access Policy
Rights for Collection: Scholarly Articles
Works are deposited here by their authors, and represent their research and opinions, not that of Duke University. Some materials and descriptions may include offensive content. More info