Interactive and alternative review practices: applicant response rounds, sandpits, bibliometric scoring, decision aids, lotteries, and crowdfunding¶

Strategy · All strategies
Evidence · no claims cited

Description¶

The report catalogs practices that modify or supplement traditional peer review. Some funders allow more interaction between applicants and reviewers: "NWO allows applicants to respond in writing to comments after the first round of review and requires an oral presentation as a part of the funding process," and UKRI's Engineering and Physical Research Council sometimes brings proposers, research users, and other experts together in interactive "sandpits" where proposals can be developed with real-time feedback. The Hungarian Scientific Research Fund "uses an automated bibliometric scoring system that adjusts scores to account for disciplinary differences and career level." Researchers have proposed decision aids to help integrate sub scores into an overall proposal score; alternative methods like partial lotteries and crowdfunding are in early conceptual development or being piloted.

Design Implications¶

Context¶

Requirements¶

Interactive practices require funder capacity to manage multiple review rounds and applicant-reviewer interaction

Constraints¶

Evidence about the efficacy of these ideas is limited; the decision aid's impact on proposal decisions has not been evaluated; partial lotteries and crowdfunding are in early phases of conceptual development or piloting

Target Learners¶

Research funders, program officers, and grant applicants

Target Learning Goals¶

Improving the reliability, validity, or efficacy of grant proposal review processes

Examples¶

Key Sources¶

Chandler, Jesse, Emily Rosen, Kimberley Raue, Katlyn Lee Milless and Danielle Rockman. 2024. A Review of Funder Instructions and Grant Reviewer Practices for Assessing the Intellectual Merit and Other Impacts of Research. Alexandria, VA: U.S. National Science Foundation. https://www.nsf.gov/pubs/2024/nsf24020/nsf24020.pdf

Page metadata

Field	Value
`type`	`strategy`
`id`	`interactive-and-alternative-grant-review-practices`
`title`	`Interactive and alternative review practices: applicant response rounds, sandpits, bibliometric scoring, decision aids, lotteries, and crowdfunding`
`description`	`The report catalogs practices that modify or supplement traditional peer review.`
`status`	`review`
`generated`	by `process:wiki-ingest` on `2026-10-08`
`trust`	unverified — no one has confirmed this page's content
`sources`	1 — `chandler-jesse-emily-rosen-kimberley`

Frontmatter is the page's machine-readable record — see the Schema & Guide.