Many learning algorithms, including neural network training and Q-learning, have recursive forms of the stochastic approximation type¶
Claim · All claims
Evidence · 1 study · theoreticalr3·q2
Currency · newest study 2003 · not yet checked against its sources · not reviewed by a person · overdue for a recheck since 2006: tool and model accuracy changes quickly
Subclaims¶
Reading the evidence codes
q is the design tier, i the impact, n the sample, and the kind with its r
the rigour judged within that kind: what each code means.
This claim: no standardized effect is recorded in observations/ for it yet, so no benchmark reading is possible.
q2 i? The actual recursive forms of learning algorithms in applications such as neural networks, Markov decision processes, learning automata, and repeated games are of the stochastic approximation type. → Harold J. Kushner and G. George Yin 2003
Evidence¶
Harold J. Kushner and G. George Yin 2003¶
Harold J. Kushner and G. George Yin. (2003). Stochastic Approximation and Recursive Algorithms and Applications, Second Edition. Springer-Verlag, New York. https://doi.org/10.1007/b97441
q2 · i? · theoretical · r3
Statement from the Preface and Introduction, following its discussion of renewed interest in learning algorithms motivated by neural network training and on-line learning of optimal strategies in high-dimensional Markov decision processes with unknown transition probabilities. It notes noise may be pseudorandom due to simulation methods.
"The actual recur- sive forms of the algorithms in many such applications are of the stochastic approximation type. Owing to the types of simulation methods used, the “noise” might be “pseudorandom” [184], rather than random."
Discussion¶
Related Claims¶
- Diminishing step sizes in stochastic approximation induce implicit averaging that eliminates noise effects in the long run — related
- For tracking time-varying systems, the step size is kept strictly away from zero — related
- Iterate averaging with step sizes decreasing slower than O(1/n) converges at an optimal rate — related
- OptimNN is insensitive to optimizer learning rate and network hyperparameters, unlike plain SGD on BKT — related
- Optimal play in coordination games requires estimating a partner's degree of recursion and one's certainty in that estimate — related
- Hyperparameter sweeps reproduce the lesson's claims: γ = 0 never solves the task while ε = 1 still converges — related
Page metadata
| Field | Value |
|---|---|
type |
claim |
id |
learning-algorithms-stochastic-approximation-recursive-forms |
title |
Many learning algorithms, including neural network training and Q-learning, have recursive forms of the stochastic approximation type |
description |
Many learning algorithms, including neural network training and Q-learning, have recursive forms of the stochastic approximation type |
status |
review |
evidence_strength |
weak |
newest_evidence |
2003 |
recheck_by |
2006 |
generated |
by process:wiki-ingest on 2026-10-14 |
trust |
unverified — no one has confirmed this page's content |
sources |
1 — harold-j-kushner-and-g-george-yin-2003 |
Frontmatter is the page's machine-readable record — see the Schema & Guide.