The CAC proof of concept compiled 27 biology problems successfully, accumulating 188 Declarative Memory entries, but failed on the 28th problem¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? In a proof-of-concept run, the agent was iteratively compiled to correct answers on 27 biology problems, accumulating 188 Declarative Memory entries with a mean of 3.1 compilation iterations per problem (SD = 2.5), and failed on the 28th problem after 150 iterations. → Hyeongdon Moon 2026

Evidence¶

Hyeongdon Moon 2026¶

Hyeongdon Moon, Carolyn Rosé, and John Stamper. (2026). Cognitive Agent Compilation for Explicit Problem Solver Modeling. https://arxiv.org/abs/2605.07040 doi:10.48550/arxiv.2605.07040

q2 · i? · design · r2

Proof-of-concept implementation with Small Language Models on the OLI_Biology dataset from CMU DataShop. The run compiled 27 problems successfully, accumulating "188 Declarative Memory entries with a mean of 3.1 compilation iterations per problem (SD = 2.5)"; the 28th problem failed after 150 iterations and the experiment was terminated. No effect sizes are reported.

"We executed problem-solving attempts for a total of 27 biology problems. During training, the agent was iteratively compiled until it produced the correct answer for each problem, accumulating 188 Declarative Memory entries with a mean of 3.1 compilation iterations per problem (SD = 2.5). On the 28th problem, the agent failed to produce the correct answer after 150 iterations, and the experiment was terminated."

Discussion¶

Related Claims¶

• Similarity-based retrieval degrades as the Knowledge Base grows, consistent with the fan effect, stalling the agent — related
• The hybrid qualitative-COM solver solves arbitrary Camelot Jr. configuration challenges within the game's problem space — related