Iterative generate-then-refine with multi-axis critiques raises the share of usable generated simulations from 3.5% after first generation to 69.3% after 10 improvement rounds¶
Claim · All claims
Evidence · 1 study · designr2·q2
Currency · newest study 2026 · checked against its sources 2026-10-20 (automated, 1 of 1 entry) · not reviewed by a person
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? Single-prompt Gen-UI generation satisfies pedagogical requirements only 3.5% of the time; iterative self-correction across visual, solution, telemetry and mechanical critiques raises the pass rate to 69.3% after 10 rounds. → Alisa Kovshov 2026
Evidence¶
Alisa Kovshov 2026¶
Alisa Kovshov, Anisha Choudhury, Anna Iurchenko, Amy Keeling, Avinatan Hassidim, Ayelet Shasha Evron, Ayça Çakmakli, Diana Akrong, Femi Olanubi, Gal Elidan, Ian Li, Ido Lerer, Katherine Chou, Lidan Hackmon, Michal Gordon, Nir Kerem, Niv Efron, Preeti Singh, Rena Levitt, Rotem Yulzary, Shlomi Ben Shimon, Sophie Allweis, Tracey Lee-Joe, Tzvika Stein, Yaniv Carmel, Yael Haramaty, Yishay Mor, Yossi Matias and Yuri Lev. (2026). Harnessing Generative UI for Education: Tailored Learning Interactives. Google Research. https://arxiv.org/abs/2609.20738 doi:10.48550/arxiv.2609.20738
q2 · i? · design · r2
Numerical result of the generate-then-refine UI generation phase, given validated simulation ideas and levelled goals, summarized in Figure 5. The article reports that "the percentage of usable simulations rises from a mere3.5%after the first generation to a69.3%after 10 generation improvement rounds."
"the percentage of usable simulations rises from a mere3.5%after the first generation to a69.3%after 10 generation improvement rounds."
Discussion¶
Related Claims¶
- In a usability study, 12 U.S. STEM teachers rated all generated learning interactives 7 or above for usability, averaging above 8.1, with 3 of 36 requests deemed too low quality to present — related
- Independent expert teachers accepted 86% of generated STEM learning interactives, rating most as good or excellent, with biology simulations weakest on aesthetics and UX — related
- Improvement across critique axes during iterative UI generation is not monotonic — related
- Review-revise iterations improve synthesized problem quality, with gains largely stabilizing by the third iteration — related
- Beam search-based self-refinement improves simulated solution alignment with predicted student behavior — related
- Refinement gains stabilize or decline beyond three iterations, and beam-size gains become negligible beyond two samples — related
- Student participation in validating and iteratively refining AI-generated simulations promotes physics understanding and problem-solving skills — related
- The system receives a good average usability rating (SUS 69.7, grade B) but with wide individual variation, which the authors take to suggest UI consistency issues — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
generate-then-refine-raises-usable-simulations-3-5-to-69-3 |
title |
Iterative generate-then-refine with multi-axis critiques raises the share of usable generated simulations from 3.5% after first generation to 69.3% after 10 improvement rounds |
description |
Iterative generate-then-refine with multi-axis critiques raises the share of usable generated simulations from 3.5% after first generation to 69.3% after 10 improvement rounds |
status |
review |
evidence_strength |
moderate |
newest_evidence |
2026 |
source_checked |
2026-10-20 |
generated |
by process:wiki-ingest on 2026-10-10 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — alisa-kovshov-2026 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.