Browsing by Subject "Hierarchical"

This thesis presents spatially and temporally dependent hierarchical Bayesian models for the analysis of social networks and associated textual data. Social network analysis has received significant recent attention and has been applied to fields as varied as analysis of Supreme Court votes, Congressional roll call data, and inferring links between authors of scientific papers. In many traditional social network analysis models, temporal and spatial dependencies are not considered due to computational difficulties, even though significant such dependencies often play a significant role in the underlying generative process of the observed social network data.

Thus motivated, this thesis presents four new models that consider spatial and/or temporal dependencies and (when available) the associated text. The first is a time-dependent (dynamic) relational topic model that models nodes by their relevant documents and uses probit regression construction to map topic overlap between nodes to a link. The second is a factor model with dynamic random effects that is used to analyze the voting patterns of the United States Supreme Court. hTe last two models present the primary contribution of this thesis two spatially and temporally dependent models that jointly analyze legislative roll call data and the their associated legislative text and introduce a new paradigm for social network factor analysis: being able to predict new columns (or rows) of matrices from the text. The first uses a nonparametric joint clustering approach to link the factor and topic models while the second uses a text regression construction. Finally, two other models on analysis of and tracking in video are also presented and discussed.

Rising temperatures with increased drought pose three challenges for management of future biodiversity. First, are the species expected to be vulnerable concentrated in specific regions and habitats? Second, are the impacts of drought and warming varying across regions? Third, could recent advances in remote sensing techniques help us in monitoring the impacts in real-time? This dissertation is an effort to address the above questions in the three chapters.

First, I used foliar chemistry as a proxy for drought vulnerability. I used soil and moisture gradients to quantify habitat variation that could be critical for alleviating drought. I used a large dataset of forest plots covering the eastern united states to understand how community weighted mean foliar nitrogen and phosphorus vary across climate and soil gradients. I exploited trends in these variables between species, traits, and habitats to evaluate sensitivity. Critical to our approach is the capacity to jointly model trait responses. Our data showed that nutrient demanding species strongly respond to environmental gradients. I identified a wide range of sites across low to high latitudes threatened by drought. The sensitivity of species to high temperatures is largely explained by soil variations. Drought vulnerability of nutrient and moisture demanding species could be amplified depending on local soil and moisture gradients. Although local soil moisture may dampen drought-induced stress for species with large leaves and high water use, nutrient demanding species remain vulnerable in wet regions during droughts. Phosphorus demanding species adapted to dry sites are drought resilient compared to communities in wet sites. This research is consistent with the studies that supports declining nutrient demanding species with increasing temperature and decreasing moisture. I also detected strong soil effects on shaping community weighted traits across a large geographical and environmental range. Our data showed that soil effects on controlling foliar traits strongly vary across different climates. The findings are critical for conservations and maintaining the biodiversity.

Next, I used space-borne remotely sensed vegetation indices to monitor the process of leaf development across climate gradients and ecoregions in the southeastern United States. A hierarchical state-space Bayesian model was developed to quantify how air temperature, drought severity, and canopy thermal stress contribute to changes in leaf opening from mountainous to coastal regions. I synthesized daily field climate data with daily remotely sensed vegetation indices and canopy surface temperature during spring green-up season. The study was focused on observation of leaf phenology at 59 sites in the southeast United States between 2001 to 2012. Our results suggest strong interaction effects between ecosystem properties and climate variables across ecoregions. The findings showed that despite the much faster spring green-up in the mountains, coastal forests express a larger sensitivity to inter-annual anomaly in temperature than mountain sites. In spite of the decreasing trend in sensitivity to warming with temperature in all regions, there is an ecosystem interaction: Deciduous-dominated forests are less sensitive to warming than are those with few deciduous trees, possibly due to the presence of developed leaves in evergreen species throughout the season. The findings revealed mountainous forests are more susceptible to intensifying drought and moisture deficit, while coastal areas are relatively resilient. I found that increasing canopy thermal stress, defined as canopy-air temperature difference, slows the leaf-development following a dry year, accelerates it after a wet year.

Finally, I demonstrate how space-borne canopy “thermal stress”, i.e. surface-air temperature difference, could be used as a surrogate for drought-induced stress to estimate forest transpiration. Using physics-based relationships that accommodates uncertainties, I showed how changes in canopy water flux may be reflected in surface energy balance and in remotely-sensed thermal stress. Validating with field measurements of canopy transpiration in the southeastern US, I quantified sensitivity of transpiration to thermal stress in a range of atmospheric and climate conditions. I found that a 1 mm change in daily transpiration may cause 3 to 4 °C of thermal stress, depending on site conditions. The cooling effect is large when solar radiation is high or wind speed is low. The effect has the highest control on water-use during warm and dry seasons, when monitoring drought is essential. I applied our model to available satellite and metrological data to detect patterns of drought. Using only air and surface temperatures, I predicted anomaly in water-use across the contiguous United States over the past 15 years, and then compared with anomaly in soil water content and conventional drought indices. Our simple model showed a reliable accuracy in compare to the state-of-the-art general circulation models. The technique can be used in varying time-scales to monitor surface water-use and drought in large scales.

This dissertation investigates the concept of transfer learning in a reinforcement learning (RL) context. Transfer learning is based on the idea that it is possible for an agent to use what it has learned in one task to improve the learning process in another task as compared to learning from scratch. This improvement can take multiple forms, such as reducing the number of samples required to reach a given level of performance or even increasing the best performance achieved. In particular, we examine properties and applications of successor features, which are a useful representation that allows efficient calculation of action-value functions for a given policy in different contexts.

Our first contribution is a method for incremental construction of a cache of policies for a family of tasks. When a family of tasks share transition dynamics but differ in reward function, successor features allow us to efficiently compute the action-value functions for known policies in new tasks. As the optimal policy for a new task might be the same as or similar to that for a previous task, it is not always necessary for an agent to learn a new policy for each new task it encounters, especially if it is allowed some amount of suboptimality. We present new bounds for the performance of optimal policies in a new task, as well as an approach to use these bounds to decide, when presented with a new task, whether to use cached policies or learn a new policy.

In our second contribution, we examine the problem of hierarchical reinforcement learning, which involves breaking a task down into smaller subtasks which are easier to solve, through the lens of transfer learning. Within a single task, a subtask may encapsulate a behavior which could be used multiple times for completing the task, but occur in different contexts, such as opening doors while navigating a building. When the reward function changes between tasks, a given subtask may be unaffected, i.e., the optimal behavior within that subtask may remain the same. If so, the behavior may be immediately reused to accelerate training of behaviors for other subtasks. In both of these cases, reusing the learned behavior can be viewed as a transfer learning problem. We introduce a method based on the MAXQ value function decomposition which uses two applications of successor features to facilitate both transfer within a task and transfer between tasks with different reward functions.

The final contribution of this dissertation introduces a method for transfer using a value-based approach in domains with continuous actions. When an environment's action space is continuous, finding the action which maximizes an action-value function approximator efficiently often requires defining a constrained approximator which results in suboptimal behavior. Recently the RBF-DQN approach was proposed to use deep radial-basis value functions to allow efficient maximization of an action-value approximator over the actions while not losing the universal approximator property of neural networks. We present a method which extends this approach to use successor features in order to allow for effective transfer learning between tasks which differ in reward function.