Browsing by Author "Calderbank, R"

Now showing 1 - 4 of 4

Open Access
Communications inspired linear discriminant analysis
(Proceedings of the 29th International Conference on Machine Learning, ICML 2012, 2012-10-10) Chen, M; Carson, W; Rodrigues, M; Calderbank, R; Carin, L
We study the problem of supervised linear dimensionality reduction, taking an information-theoretic viewpoint. The linear projection matrix is designed by maximizing the mutual information between the projected signal and the class label. By harnessing a recent theoretical result on the gradient of mutual information, the above optimization problem can be solved directly using gradient descent, without requiring simplification of the objective function. Theoretical analysis and empirical comparison are made between the proposed method and two closely related methods, and comparisons are also made with a method in which Rényi entropy is used to define the mutual information (in this case the gradient may be computed simply, under a special parameter setting). Relative to these alternative approaches, the proposed method achieves promising results on real datasets. Copyright 2012 by the author(s)/owner(s).
Open Access
Communications-inspired projection design with application to compressive sensing
(SIAM Journal on Imaging Sciences, 2012-12-01) Carson, WR; Chen, M; Rodrigues, MRD; Calderbank, R; Carin, L
We consider the recovery of an underlying signal x ∈ ℂm based on projection measurements of the form y = Mx+w, where y ∈ ℂℓ and w is measurement noise; we are interested in the case ℓ ≪ m. It is assumed that the signal model p(x) is known and that w ~ CN(w; 0,Σw) for known Σ w. The objective is to design a projection matrix M ∈ ℂℓ×m to maximize key information-theoretic quantities with operational significance, including the mutual information between the signal and the projections I(x; y) or the Rényi entropy of the projections hα (y) (Shannon entropy is a special case). By capitalizing on explicit characterizations of the gradients of the information measures with respect to the projection matrix, where we also partially extend the well-known results of Palomar and Verdu ́ from the mutual information to the Rényi entropy domain, we reveal the key operations carried out by the optimal projection designs: mode exposure and mode alignment. Experiments are considered for the case of compressive sensing (CS) applied to imagery. In this context, we provide a demonstration of the performance improvement possible through the application of the novel projection designs in relation to conventional ones, as well as justification for a fast online projection design method with which state-of-the-art adaptive CS signal recovery is achieved. © 2012 Society for Industrial and Applied Mathematics.
Open Access
DCFNet: Deep Neural Network with Decomposed Convolutional Filters
(35th International Conference on Machine Learning, ICML 2018, 2018-01-01) Qiu, Q; Cheng, X; Calderbank, R; Sapiro, G
©35th International Conference on Machine Learning, ICML 2018.All Rights Reserved. Filters in a Convolutional Neural Network (CNN) contain model parameters learned from enormous amounts of data. In this paper, we suggest to decompose convolutional filters in CNN as a truncated expansion with pre-fixed bases, namely the Decomposed Convolutional Filters network (DCFNet), where the expansion coefficients remain learned from data. Such a structure not only reduces the number of trainable parameters and computation, but also imposes filter regularity by bases truncation. Through extensive experiments, we consistently observe that DCFNet maintains accuracy for image classification tasks with a significant reduction of model parameters, particularly with Fourier-Bessel (FB) bases, and even with random bases. Theoretically, we analyze the representation stability of DCFNet with respect to input variations, and prove representation stability under generic assumptions on the expansion coefficients. The analysis is consistent with the empirical observations.
Open Access
RotDCF: Decomposition of Convolutional Filters for Rotation-Equivariant Deep Networks.
(CoRR, 2018) Cheng, X; Qiu, Q; Calderbank, R; Sapiro, G
Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group geometry simultaneously, namely a rotation-equivariant CNN with decomposed convolutional filters (RotDCF). This decomposition facilitates computing the joint convolution, which is proved to be necessary for the group equivariance. It significantly reduces the model size and computational complexity while preserving performance, and truncation of the bases expansion serves implicitly to regularize the filters. On datasets involving in-plane and out-of-plane object rotations, RotDCF deep features demonstrate greater robustness and interpretability than regular CNNs. The stability of the equivariant representation to input variations is also proved theoretically under generic assumptions on the filters in the decomposed form. The RotDCF framework can be extended to groups other than rotations, providing a general approach which achieves both group equivariance and representation stability at a reduced model size.