Undergraduate students debugging analog circuits under exam pressure preferentially used images to capture the physical circuit and the exam assignment when conversing with LLMs¶

Claim · All claims
Evidence · 1 study · qualitative r2 · q2
Currency · newest study 2026 · not yet checked against its sources · not reviewed by a person

Subclaims¶

q2 i? A substantial share of chat logs used images of both the circuit and the exam question rather than purely verbal descriptions. → John Hu and Andrew J. Ash 2026

Evidence¶

John Hu and Andrew J. Ash 2026¶

John Hu and Andrew J. Ash. (2026). Chat Debugging: An Exploratory Study of Human-AI Collaboration to Debug Analog Circuits. https://arxiv.org/abs/2608.02955 doi:10.48550/arxiv.2608.02955

q2 · i? · qualitative · r2

Thematic analysis of voluntarily shared chat logs from a 30-minute timed hands-on lab final exam found an image-based usage pattern the authors had not expected: "2 out of 5 chat logs in Spring 2025 and 5 out of 12 chat logs in Fall 2025 followed this usage pattern." The authors attribute it to exam time pressure and the convenience of photos.

"2 out of 5 chat logs in Spring 2025 and 5 out of 12 chat logs in Fall 2025 followed this usage pattern."

Discussion¶

Related Claims¶

• More students embraced LLMs for debugging from Spring to Fall 2025 despite declining course enrollment — related
• LLM factual errors during collaborative circuit debugging occurred almost exclusively after image inputs, attributed to limited 3D spatial reasoning from breadboard or PCB photos — related
• LLMs expressed unjustified confidence in recommendations based on visual inputs, asserting incorrect pin-connection diagnoses with high stated certainty — related
• An LLM debugging assistant provides accurate, detailed descriptions of circuit components and their proper connections — related
• LLM-assisted hardware debugging requires consistent human feedback because the LLM's understanding of the circuit drifts — related
• Off-the-shelf LLMs without domain-specific fine-tuning suggested true root causes among their zero-shot debugging recommendations for buggy analog circuits — related
• An LLM debugging assistant correctly interprets brief, informal natural-language prompts describing circuits — related
• Hardware debugging with an LLM assistant takes multiple prompts and more patience than software debugging — related