A small question budget of about 200 suffices for strong hypothesis quality, with room to scale¶

Claim · All claims
Evidence · 1 study · causal r1 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person · recheck by 2029: tool and model accuracy changes quickly

Subclaims¶

q2 i? With a suitable sampling and prompting strategy, the method reaches a good level of performance within 200 questions and retains potential to scale further, as none of the mixed-strategy curves has fully converged. → Peng Cui 2026

Evidence¶

Peng Cui 2026¶

Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mrinmaya Sachan. (2026). What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language. Preprint. https://arxiv.org/abs/2610.01627 doi:10.48550/arxiv.2610.01627

q2 · i? · causal · r1

Budget analysis tied to the Figure 4 sampling ablations, treating question numbers as a proxy for the annotation budget. The article reports good performance "within200 questions" and unconverged mixed-strategy curves.

"Overall, with a suitable sampling and prompting strategy, our method reaches a good level of performance within200 questions (a proxy for the annotation budget) and retains the potential to scale further when the budget allows, as none of the mixed-strategy curves in Figure 4 has fully converged."

Discussion¶

Related Claims¶

• Mixing group-wise and pair-wise contrastive sampling yields better hypotheses than either alone — related
• Baseline hypothesis generators saturate after 5–10 effective hypotheses while the proposed method keeps improving — related