Learning Features for Unsupervised Learning and Reinforcement Learning

Feature learning has been one of the main tasks in machine learning, and the recent advancement in deep learning only increases the importance of understanding the role of features, and designing efficient algorithms to learn these latent representations. Motivated by the successes from deep models, we investigate several important topics in unsupervised learning and reinforcement learning (RL). The first part of this thesis builds upon Bayesian statistics to address the problems of model learning and model selection in belief networks, respectively. The proposed methods possess the statistical guarantee, and are scalable for a broad class of large scale data. In the second part of this thesis, we develop and evaluate a theory of linear feature encoding, and demonstrate the connection between the linear value

function approximation and the deep RL. We then revisit the softmax Bellman operator, and prove its theoretical properties by showing its performance bound, and demonstrate its practical benefits in deep RL by stabilizing deep Q-networks.