Hyperparameter setting with maximum token 53 and temperature 1 (Set C) yields the most accurate AGE grading performance¶

Claim · All claims
Evidence · 1 study · design r2 · q2
Currency · newest study 2024 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? Set C with (max-token, temperature) = (53, 1) becomes the most accurate configuration in the hyperparameter ablation study. → Mohammad Iqbal 2024

Evidence¶

Mohammad Iqbal 2024¶

Mohammad Iqbal, Tsamara Rana Nugraha and Hsing-Kuo Pao. (2024). Active Grade Estimator on Short Answer Assessment. https://doi.org/10.21203/rs.3.rs-5651820/v1

q2 · i? · design · r2

Ablation study of hyperparameter settings (temperature and max token) shown in Fig. 8(c). The article reports Set C "becomes the most accurate by letting the creativity be at maximum and a smaller token length (max-token, temperature) = (53, 1)".

"Further, Se t C becomes the most accurate by letting the creativity be at maximum and a smaller token length (max-token, temperature) = (53, 1)."

Discussion¶

Related Claims¶

• Higher LLM temperature settings yield the best English augmentation performance, with Llama temperature 2 the top generator overall
• Temperature-construct interactions: constructs with moderate theoretical coherence benefited from higher temperatures, while well-defined constructs required deterministic settings
• Lowering an LLM's temperature setting is one lever for improving output consistency
• LLM temperature settings significantly affect generated-text length and complexity, with Llama3 producing more complex texts than GPT-3.5
• GPT-4 surpasses other LLMs in the AGE query selection with an accuracy rate 15% better