Browsing by Author "Cummings, Mary (Missy) L"

Autonomous systems are emerging across many industries. From unmanned aircraft to self-driving cars to closed-loop medical devices, these systems offer great benefits but also pose new risks. Regulators must grapple with how to manage these risks, challenged to keep pace with technological developments and exhibit appropriate precaution without stifling innovation. Seeking inspiration for a viable approach to the regulation of autonomous systems, this thesis draws from the practices of systems engineering, an interdisciplinary field of engineering aimed at managing the risks of complex projects. By comparing systems engineering practices to regulatory options, current regulations, and the inherent challenges of regulating emerging technologies, this thesis concludes that a systems engineering-based approach to regulating autonomous systems offers great potential for managing the risks of autonomous systems while also driving innovation.

Across multiple industries (e.g., railroads, airlines, on-demand air taxi services), there are growing investments in future automated transportation systems. Even with these investments, there are still significant human-systems engineering challenges that require deeper investigation and planning. Specifically, fleets that include new levels of automation may require new concepts of how to design and staff network operations centers. Network operations centers have existed for over a century in the railroad and airline industries, where dispatchers have played a central role in safely and efficiently managing networks of railroads and flights. With operators in such safety-critical and time-sensitive positions, workload is the key indicator of their performance in terms of accuracy and efficiency. Yet, there are few tools available for decision-makers in these industries to explore how increasing levels of automation in fleets and operations centers may ultimately affect dispatcher workload.

Thus, this thesis presents a model of dispatcher workload. While automation may be the most pressing change in transportation industries, 10 variables related to configurations of the fleet and the operations center and how those variables interact to influence dispatcher workload were defined. These ten variables come from fleet conditions, strategic design factors, tactical staffing factors, and operational factors. A discrete event simulation was developed to computationally model dispatcher workload with over 10^18 possible configurations of these variables. Additionally, using time-based metrics and integrating results from a prior human reliability assessment, the simulation predicts human error on tasks.

A multi-level validation strategy was developed to build internal, external, and general confidence in using the dispatcher workload model across different domains with data from freight railroad, commuter railroad, and airline operations. In the process of developing and validating the workload model, several other research contributions were made to the field. Eighty-five probability density functions of dispatcher task inter-arrival and service time distributions were generated in the three domains. A data collection tool, Dispatcher’s Rough Assessment of Workload-Over Usual Times (DRAW-OUT), was designed to gather empirical dispatcher-generated estimates of utilization, the proxy for workload, throughout their shifts.

Using the model, experiments were conducted to analyze the sensitivity of dispatcher workload and performance to changes in different parameters. The size of the fleet a dispatcher managed was found to be the most significant factor out of all the other internal parameters. On the other hand, shift schedule, environmental conditions, and operator strategy were the parameters found to have the smallest influence on dispatcher performance. The model was also used to investigate future scenarios that managers could not previously explore due to limitations of time and resources. Results show that the general model is applicable for use in simulating dispatcher workload in both freight and commuter railroad operations as well as airline operations, including short- and long-haul flights, in present-day and future cases.

General confidence was built in the workload model and the Simulator of Humans & Automation in Dispatch Operations (SHADO) was developed as an online platform to provide open access to the underlying discrete event simulation. SHADO is a novel tool that allows stakeholders, including operational managers, to rapidly prototype dispatch operations and investigate human performance in any transportation system. With several theoretical and practical contributions, this work establishes the foundation for future research in the growing field of advanced transportation network operations.

This dissertation proposes three control algorithms for active sensing with one or several autonomous robots.

The algorithms all rely on models of the information content of the sensor measurement with respect to the relative poses between sensors and subjects.

The approaches each predict how new information may impact the uncertainty in the subjects, controlling sensors to new locations or trajectories from where these uncertainties will me minimized.

The first algorithm deals with the Next-Best-View (NBV) problem for a single robot, where the goal is to control a mobile camera so that the next image of a set of possibly mobile targets will be as informative as possible.

The NBV controller is designed for a rig that hosts two cameras in a fronto-parallel arrangement, commonly known as stereo vision.

Assuming that the objects, landmarks, or targets being estimated are visible by both cameras in the rig and that these observations are corrupted by zero-mean Gaussian errors, the control algorithm moves the rig through pose space in order to reduce the expected Kalman-filtered uncertainty in the next location point-estimate.

This is done by differentiating the KF output error covariance matrix with respect to the sensor pose, which results in a nonlinear control problem.

The controller is decomposed so that first the robot computes the NBV in coordinates relative to the body-frame of the stereo rig, and then it moves in pose space to realize this view.

When an image is acquired, a switching signal changes the goal of pose control, giving rise to a stable hybrid system.

Experiments of on a real robot localizing targets in a laboratory setting are presented.

The second algorithm addresses the problem of estimating a finite set of hidden state vectors using a mobile robotic sensor network.

For every hidden state that needs to be estimated, a local Dynamic Program (DP) in the joint state-space of robot positions and state uncertainties determines robot paths and associated sequences of state observations that collectively minimize the estimation uncertainty.

It divides the collection of hidden states into clusters based on a prior belief of their geographic locations and, for each cluster, defines a second DP that determines how far along the local optimal trajectories the robot should travel before transitioning to estimating the next hidden state within the cluster.

Finally, a distributed assignment algorithm dynamically allocates controllers to the robot team from the set of optimal control policies at every cluster.

Assuming Gaussian priors on the hidden state vectors, the distributed state estimation method scales gracefully to large teams of mobile robots and hidden vectors and provide extensive simulations and real-world experiments using stereoscopic vision sensors to illustrate the approach.

The third chapter addresses the problem of controlling a network of mobile sensors so that a set of hidden states are estimated up to a user-specified accuracy. The sensors take measurements and fuse them online using an Information Consensus Filter (ICF). At the same time, the local estimates guide the sensors to their next best configuration. This leads to an LMI-constrained optimization problem that we solve by means of a new distributed random approximate projections method. The new method is robust to the state disagreement errors that exist among the robots as the ICF fuses the collected measurements. Assuming that the noise corrupting the measurements is zero-mean and Gaussian and that the robots are self localized in the environment, the integrated system converges to the next best positions from where new observations will be taken. This process is repeated with the robots taking a sequence of observations until the hidden states are estimated up to the desired user-specified accuracy. It presents simulations of sparse landmark localization, where the robotic team is achieves the desired estimation tolerances while exhibiting interesting emergent behavior.

Experiments of the first two algorithms are also presented.

Human supervisory control (HSC), in which operators indirectly control autonomous systems by sending and receiving commands, is a commonly-used scheme for various human-automation interaction scenarios. While many studies have investigated how factors, such as different levels of autonomy and interface designs, affect operator performance in HSC scenarios, no previous research has quantitatively evaluated the impact of such factors on operator strategies. Thus, this research focuses on developing a quantitative metric to compare strategy models to determine whether changing specific factors in HSC scenarios would affect operator strategies.

Previous studies have shown that operator strategies can be represented by operator behavior patterns in conducting tasks and achieving goals. Given that hidden Markov models (HMMs) can represent operator strategies, researchers can investigate impacts from technology or process changes on operator strategies by comparing HMM strategy models. However, no quantitative and systematic HMM strategy model comparison metric has been proposed. To resolve this problem, this research uses the divergence distance measure to develop a mesh comparison metric to comprehensively compare strategy models and obtain quantitative model difference measures.

As a part of the comparison metric, the data quantity requirement for model development is determined using a large external dataset from a typical HSC video game. Strategy models were trained based on different data quantities and then compared to benchmark models developed from the whole dataset. Comparison results show that a minimum of 30 data sequences can represent the whole population and be effectively used to model operator strategies. Also, as another part of the metric, an observation alignment approach is proposed to compare strategy models developed from different HSC scenarios with non-equivalent training data elements.

Utilizing this comparison metric, researchers can quantitatively measure differences between strategy models. However, it is not clear how the magnitude of such comparison measures maps to meaningful degrees of difference in HSC scenarios. To address this issue, an initial baseline of strategy difference comparisons was established by comparing strategy models developed from human-subject experiment sessions. Then, a continuum of comparisons was generated to provide references for the magnitude of impacts from different factors on operator strategies. Thus, researchers can apply changes in HSC scenarios and evaluate the impacts from such changes on operator strategies by measuring differences between strategy models and referring to comparison baselines.

In summary, the contributions of this dissertation include 1) proposing an operator strategy model comparison metric to quantitatively measure differences between operator strategies modeled from HSC scenarios and 2) establishing strategy model comparison references across multiple HSC scenarios with varying settings.

With increasing prevalence and capabilities of autonomous systems as part of complex heterogeneous manned-unmanned environments (HMUEs), an important consideration is the impact of the introduction of automation on the optimal assignment of human personnel. The US Navy has implemented optimal staffing techniques before in the 1990's and 2000's with a "minimal staffing" approach. The results were poor, leading to the degradation of Naval preparedness. Clearly, another approach to determining optimal staffing is necessary. To this end, the goal of this research is to develop human performance models for use in determining optimal manning of HMUEs. The human performance models are developed using an agent-based simulation of the aircraft carrier flight deck, a representative safety-critical HMUE. The Personnel Multi-Agent Safety and Control Simulation (PMASCS) simulates and analyzes the effects of introducing generalized maintenance crew skill sets and accelerated failure repair times on the overall performance and safety of the carrier flight deck. A behavioral model of four operator types (ordnance officers, chocks and chains, fueling officers, plane captains, and maintenance operators) is presented here along with an aircraft failure model. The main focus of this work is on the maintenance operators and aircraft failure modeling, since they have a direct impact on total launch time, a primary metric for carrier deck performance. With PMASCS I explore the effects of two variables on total launch time of 22 aircraft: 1) skill level of maintenance operators and 2) aircraft failure repair times while on the catapult (referred to as Phase 4 repair times). It is found that neither introducing a generic skill set to maintenance crews nor introducing a technology to accelerate Phase 4 aircraft repair times improves the average total launch time of 22 aircraft. An optimal manning level of 3 maintenance crews is found under all conditions, the point at which any additional maintenance crews does not reduce the total launch time. An additional discussion is included about how these results change if the operations are relieved of the bottleneck of installing the holdback bar at launch time.

Given their low operating costs and flight capabilities, Unmanned Aircraft Vehicles(UAVs), especially small size UAVs, have a wide range of applications, from civilian rescue missions to military surveillance. Easy control from a highly automated system has made these compact UAVs particularly efficient and effective devices by alleviating human operator workload. However, whether or not automation can lead to increased performance is not just a matter of system design but requires operators’ thorough understanding of the behavior of the system. Then, a question arises: which type of training and level of automation can help UAV operators perform the best?

To address this problem, an experiment was designed and conducted to compare the differences in performance between 3 groups of UAV operators. For this experiment, 2 different interfaces were first developed - Manual Control, which represents low LOA interface, and Supervisory Control, which represents high LOA interface - and people were recruited and randomly divided into 3 groups. Group 1 was trained using Manual Control, and Group 3 was trained using Supervisory Control while Group 2 was trained using both Manual and Supervisory Control. Participants then flew a drone in the Test Mission stage to compare performance.

The results of the experiment were rather surprising. Although group 3 outperformed group 1, as expected, the poor performance of group 2 was unexpected and gave us new perspectives on additional training. That is, additional training could lead not just to a mere surplus of extra skills but also a degradation of existing skills. An extended work using a more mathematical approach should allow for a more precise, quantitative description on the relation between extra training and performance.