AI literacy warnings reduce LLM political persuasion by roughly one-half (meta-analytic -48.1%)¶

Claim · All claims
Evidence · 1 study · causal r3 · q3
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q3 i? Across two preregistered experiments, AI literacy warnings reduced persuasive effects of LLM conversations on political attitudes by 48.1% relative to control. → Reed Orchinik 2026

Evidence¶

Reed Orchinik 2026¶

Reed Orchinik, David Rand. (2026). A light-touch AI literacy intervention helps protect against AI political persuasion. https://arxiv.org/abs/2609.16432 doi:10.48550/arxiv.2609.16432

q3 · i? · causal · r3

Multilevel random-effects meta-analysis pooling 32 warning-by-cell estimates from two preregistered experiments (total N = 3,208), estimated by REML in metafor. The pooled percent reduction was "48.1% ([−59.5%,−36.8%],z=−8.32,p <0.001)"; the additive-scale robustness check gave b=−2.06, p= 0.002.

"the meta-analysis indicates that literacy warnings reduced persuasive effects by 48.1% ([−59.5%,−36.8%],z=−8.32,p <0.001)"

Discussion¶

Related Claims¶

• AI literacy interventions show a large, statistically significant overall positive effect on student AI literacy outcomes (g = 0.837) — the general claim this finding bears on, with the studies behind it
• In Study 1, the general warning significantly reduced persuasive effects on housing-policy attitudes (b=−1.81, p= 0.001) — a narrower finding that bears on this claim
• In Study 2, both general and specific warnings at least marginally reduce attitude change, with no significant difference between them — a narrower finding that bears on this claim
• Warning effect size shows no significant heterogeneity across topics and studies — related
• Test-takers rate LLM-generated feedback as persuasive and useful as human instructor feedback (Wilcoxon signed-rank tests show no significant difference) — related