Selected Papers
Large Language Models
“Using Activation Steering to Trace and Mitigate Racial Bias in LLM-Based Essay Evaluation”
Abstract: Large Language Models (LLMs) exhibit documented racial biases in text evaluation tasks, including Automated Essay Scoring (AES). While prior work has identified these disparities, the internal decision-making processes that produce them remain poorly understood. I investigate these mechanisms through the lens of activation engineering, analyzing the decision representations learned by LLMs during pairwise essay comparisons. I identify a race-linked decision boundary that can be decomposed into a shared winner representation and a residual component specific to cross-racial comparisons. Using vector steering interventions on hidden-layer activations, I causally test the role of this residual component and find that reducing its influence substantially decreases racial bias while preserving scoring accuracy. The results demonstrate how representation-level decomposition can enable targeted debiasing in settings where fairness interventions must be balanced against predictive performance, and suggest a broader framework for debiasing pairwise comparison systems.
Link: Working Paper
“Effects of Open-Ended Response Quality on Stability of Automated Text Analysis in Survey Research" (with Andrew Gordon and David Rothschild)
Abstract: The growing use of automated text analysis has increased reliance on open-ended survey responses as measures of public opinion. Yet responses collected in online surveys vary widely in quality, raising concerns about the validity of substantive conclusions drawn from text-as-data and AI-assisted approaches. We introduce a Large Language Model (LLM)-based procedure for measuring the quality of open-ended survey responses and examine how variation in response quality shapes the outputs of automated text analysis. Using three different topic modeling approaches (vanilla LDA, BERTopic, and LLM-based topic discovery), we show that topic-level conclusions differ substantially when analyses are based on lower- versus higher-quality responses. In particular, topics derived from very low-quality responses overlap only partially with those derived from very high-quality responses, with only about 30-50\% of topics aligning across these groups. Leveraging a randomized experiment that influences response quality by assigning respondents to answer by voice or text, we show that even exogenous, moderate increases in quality produce statistically significant changes in inferred topic structure. These findings demonstrate that variation in open-ended response quality is not merely noise, but a consequential source of measurement error that is amplified by automated text analysis. As a result, routine and otherwise defensible research choices (such as survey vendor or response modality) can yield substantively different characterizations of public opinion.
Link: Working Paper
“Do LLMs Distort Public Opinion? Comparing Respondent Representation Across Topic Modelling Architectures” (with Andrew Gordon and David Rothschild)
Abstract: Large language models (LLMs) are increasingly used to summarize open-ended survey responses, yet little is known about how they represent public opinion relative to established topic modeling approaches. We compare LLM-based topic discovery with two widely used alternatives—Latent Dirichlet Allocation (LDA) and BERTopic—using open-ended responses from the 2024 American National Election Studies. To quantify representativeness, we introduce a document influence measure that estimates how much each individual response changes the discovered topic structure when removed from the corpus. We find that LLM-derived topics rely on a substantially different subset of responses than classical topic models, with little correspondence to BERTopic and only moderate agreement with LDA. Relative to LDA and BERTopic, LLM topic models disproportionately weight responses from politically interested respondents, an effect that is largely explained by their sensitivity to document length. These findings suggest that while LLM-generated topics are highly interpretable, they may systematically overrepresent certain respondents and perspectives, with important implications for the use of LLMs in survey research and public opinion analysis.
Political Behavior
“Student Loan Debt and Borrowers' Attitudes Toward Inequality”
Abstract: As of 2024, U.S. households hold over $1.6 trillion in outstanding student loan debt, with average repayment durations exceeding twenty years. This paper examines how student loan indebtedness relates to U.S. college graduates' perceptions of economic and racial inequality. Using a combination of publicly available and original surveys, I find that student loan borrowers with college degrees are more likely to attribute economic and racial inequality to structural, rather than individual factors, than do other college-educated respondents. These relationships persist after controlling for demographic characteristics and partisan identification. Leveraging borrowers' entry into the repayment period, I show that these patterns are less likely to be the result of selection into debt. Borrowers also exhibit significantly lower levels of system-justifying attitudes, indicating that student loan indebtedness may contribute to broader shifts in political attitudes among younger adults. These results suggest that student loan indebtedness is an important factor in the political socialization of college-educated U.S. adults.
“The Effect of Beliefs About American Opportunity on Immigrants’ Racial Attitudes”
Abstract: This paper studies the attitudes of first-generation immigrants in the US toward Black Americans. Using original and publicly available surveys, I find that first-generation immigrants of all racial and ethnic groups display more negative attitudes toward Black Americans than do their native-born co-ethnics. These effects persist after controlling for demographics and partisanship, and are not explained by immigrants' experiences of discrimination. Using an original measure of optimism about the US, I find that immigrants have substantially higher levels of US optimism than native-borns. I use both mediation analysis and a survey experiment to show that differences in US Optimism are responsible for nativity-based differences in racial attitudes. These findings suggest that immigrants' negative beliefs about Black Americans are a function of their own attitudes about American opportunity and social mobility.
Link: Working Paper
“Does Partisanship Affect Compliance with Government Recommendations?”
Abstract: This article studies the role of partisanship in American’s willingness to follow government recommendations. I combine survey and behavioral data to examine partisans’ vaccination rates during the Bush and Obama administrations. I find that presidential co-partisans are more likely to believe that vaccines are safe and more likely to vaccinate themselves and their children than presidential out-partisans. Depending on the vaccine, presidential co-partisans are 4-10 percentage points more likely to vaccinate than presidential out-partisans. Using causal mediation analysis, I find that this effect is the result of partisans’ differing levels of trust in government. This finding sheds light on the far-reaching role of partisanship in Americans’ interactions with the federal government.
Web Search Data as Public Opinion
“Gun Purchase Interest as Backlash to Black Lives Matter Protests" (with Elad Yom-Tov and David Rothschild)
Abstract: How do protests affect Americans' gun ownership decisions? Using a novel dataset of gun-related web searches in combination with geocoded protest data, we examine the effects of the 2020 Black Lives Matter protests on Americans' interest in firearm purchase. We find a clear relationship between geographic proximity to BLM protests and firearm purchase web searches, but a null relationship between these searches and proximity to re-opening protests. We then examine racial attitudes of would-be gun buyers using users' web search histories and find that users exposed to racially conservative narratives had significantly larger spikes in gun purchase interest during the 2020 BLM protests than did other comparable searchers. These results suggest that Black civil rights protests can serve as a catalyst for gun purchase interest.
Link: Social Forces (2025)
“Anti-Immigrant Rhetoric and ICE Reporting Interest: Evidence from a Large-Scale Study of Web Search Data” (with Shawndra Hill and David Rothschild)
Abstract: This paper studies whether media cues can motivate Americans to report suspected unauthorized immigrants to Immigration and Customs Enforcement (ICE). Using Google Trends data, a novel dataset of immigration-related Bing web searches, and automated content analysis of cable news transcripts, we examine the role of post-2016 media coverage on searches for information about how to report immigrants to ICE, as well as searches about immigrants related to crime and welfare dependency. We find significant and persistent increases in news segments on crime by immigrants and their use of public services after Trump's inauguration, accompanied by a sharp increase in searches for reporting immigrants. We find a strong and consistent association between daily reporting searches and immigration and crime coverage, as well as with media fear cues about immigrants. Using timestamped searches during broadcasts of Trump's and Obama's speeches, we isolate the specific effect of anti-immigrant media coverage on searches for how to report immigrants to ICE. The findings indicate that media's choices regarding the coverage of immigrants can have a strong impact on the public's willingness to engage in behavior that directly harms immigrants.
“Do Partisans Make Different Investment Decisions when their Party is in Power?” (with Shawndra Hill and David Rothschild)
Abstract: Partisans' stated beliefs about the economy vary dramatically depending on the party that holds the presidency. Do these responses represent genuine differences in beliefs about the economy, or do they reflect partisans' expressive reporting on surveys? To answer this question, we rely on a novel dataset of Bing searches related to housing, automobiles, and stock market purchases by partisans from February 2016 to July 2017. We find that in the aftermath of the 2016 election, Democrats, as members of the losing party, were modestly less likely to search for both house and car purchase terms. Republicans showed no change. This shift in investment behavior among Democrats suggests that partisans' survey responses are at least partially due to different beliefs about the economy, rather than just expressive reporting.