Experimental Design and Partial Identification for Ordinal Outcomes

Abstract

In this dissertation, we develop experimental design and partial identification inference methods for causal experiments with ordinal outcomes. Chapter 2 of the dissertation investigates how social media users classify profiles as either bot or human. We design and implement a preregistered conjoint experiment in which respondents evaluate social media threads containing both human- and LLM-generated content, while randomizing race, gender, and partisanship cues on the LLM-operated profile. We find that users from dominant social groups classify out-group profiles as bots more often, whereas users from marginalized groups classify in-group profiles as bots more often. This study offers a framework for a ``Sociological Turing Test'' and highlights considerations in measuring online profile perception, including experimental power analysis and treatment stimuli construction.Chapter 3 develops theory for causal experiments with ordinal outcomes, where the typically-studied average treatment effect (ATE) is not well-defined. We derive and prove sharp upper and lower bounds on the probability that an individual’s potential outcome under treatment equals that under control. These bounds show the extent to which the observed data support a null effect conclusion. Situations in which the sharp lower bound is zero motivate our introduction of a new partial identification framework that incorporates minimal, interpretable beliefs on the probability of no effect, and we prove sharp bounds under this setting. The results demonstrate how injecting even small amounts of belief meaningfully alters the identification set of estimands and sharpens conclusions.Chapter 4 proposes a hypothesis testing procedure to test point null hypotheses regarding partially identified parameters. We state our decision rules under partial identification and show how to invert these tests to obtain confidence regions for parameters such as the probabilities of a beneficial treatment and strictly beneficial treatment. Through simulation studies and empirical applications, we demonstrate how these procedures work and how they help us draw meaningful conclusions even when point identification is not possible. These contributions advance causal inference for ordinal outcomes in partial identification settings, with an application to human-AI social interaction.