wave 7: LSTM follow-ups (5 stubs)

[0bserver07]

Wave 7 — LSTM follow-ups (2000–2002)

Five stubs implementing the LSTM follow-up era per SPEC issue #1. Octopus-merged from 5 local-only wave-7-local/<slug> branches.

Stub	Method / Paper	Headline
temporal-order-4bit	1997 LSTM, 8-class (H&S 1997 Exp 6b)	5/5 seeds 100% best validation; median ~8k seqs (paper: 571,100 with SGD — 70× faster with Adam); vanilla RNN at chance 12.3%; gradient check 3.5e-11
continual-embedded-reber	Forget gate vs no-forget (Gers 2000)	5/5 forget seeds solve (99.7%) vs 5/5 no-forget at chance (55%); cell-state norm 25 vs 295; mechanism mechanistically verified
anbn-anbncn	Peephole LSTM on CFL/CSL counters (Gers 2001)	a^n b^n trained n=1..10 → generalizes to n=1..65; a^n b^n c^n to n=1..29; cell 0 emerges as clean linear counter; gradcheck 5.66e-6
timing-counting-spikes	Peephole LSTM on MSD timing (Gers 2002)	Peephole seed 4: MSE 0.00073 / solve 0.998; vanilla 0.00240 / 0.900. Partial: across 7 seeds peep only ~5% lower; paper's strong "vanilla fails all 3 tasks" doesn't fully reproduce at short-MSD/laptop scale
blues-improvisation	2-layer stacked LSTM on synthetic blues (Eck 2002)	12/12 bar-onset chord match; step-chord 0.906; on-beat 0.792; 8 hand-synthesized 12-bar choruses (no external dataset); 12s wall

Audit verdict (separate Explore subagent)

APPROVE (after one cleanup commit). Originally REQUEST-CHANGES due to leftover blues-improvisation/problem.py; fixed in commit eae2229 before PR open.

• Numpy-only (hard pass): All 5 verified.
• Determinism (3 spot-checks): temporal-order-4bit, continual-embedded-reber, blues-improvisation — all bit-identical across reruns.
• Branch protocol: zero wave-7-local/* branches on origin.
• Algorithmic faithfulness (5/5): temporal-order-4bit uses original 1997 LSTM (no forget); continual-embedded-reber demonstrates forget-vs-no-forget contrast (Gers 2000); anbn-anbncn uses peephole LSTM (Gers 2002); timing-counting-spikes uses peephole + vanilla contrast; blues-improvisation uses 2-layer stacked LSTM with forget gate.
• Honest gap-reporting: timing-counting-spikes flagged as PARTIAL — paper's "vanilla fails entirely" doesn't reproduce at short-MSD scale on laptop budget. Documented in §Deviations and §Open questions with the v1.5 path (T ≥ 300, longer training).
• Cleanliness (after cleanup commit): zero TODO/FIXME, no hardcoded paths, no problem.py stubs left, no __pycache__ committed.
• Git authors: All 5 + cleanup commit by agent-0bserver07.
• GIF sizes: 108 KB to 1.0 MB (all under 2 MB).

Per-stub deviations (in each stub's §Deviations)

• temporal-order-4bit: T=50 (paper 100-110); 6 cells (paper 3 blocks of 2, 326 vs 308 weights); Adam vs SGD+momentum.
• continual-embedded-reber: 2000 chunks of 6 strings (paper streams differ); Adam vs SGD; forget-bias init +1; state clip |s|≤50 for numerical safety on no-forget variant.
• anbn-anbncn: trained n=1..10 (paper sweeps higher); peephole connections from c_{t-1} (input/forget) and c_t (output).
• timing-counting-spikes: PARTIAL — short-MSD (T=150 vs paper's longer); MSD only (GTS, PFG deferred to v1.5); peep-vs-vanilla contrast not fully reproduced.
• blues-improvisation: 2-layer stack vs paper's partition; synthetic 8-chorus corpus (no external MIDI); coarser pitch vocab; Adam vs paper's online BPTT/momentum.

Wave 0 → 7 progress

7 + 5 + 5 + 5 + 4 + 6 + 5 = 37/50 v1 stubs done (74%). 3 waves remaining = 13 stubs.

---

agent-0bserver07 (Claude Code) on behalf of Yad

[0bserver07]

Audit Report — PR #11 wave 7 (5 stubs)

Wave 7 verdict: APPROVE (after one cleanup commit).

Originally REQUEST-CHANGES due to leftover blues-improvisation/problem.py (same auditor catch as wave 6's noise-free-long-lag). Fixed in commit eae2229 on top of the wave merge before PR open.

Per-stub verdicts

Stub	Verdict	Reason
temporal-order-4bit	APPROVE	1997 LSTM verified; 5/5 seeds 100% best val; gradient check 3.5e-11
continual-embedded-reber	APPROVE	Forget-vs-no-forget contrast verified mechanistically (cell-state norm 25 vs 295)
anbn-anbncn	APPROVE	Peephole LSTM verified; cell 0 emerges as linear counter; gradcheck 5.66e-6
timing-counting-spikes	APPROVE-WITH-NOTES	Peephole + vanilla contrast verified; honest partial reproduction documented
blues-improvisation	APPROVE	2-layer stacked LSTM verified; bar-onset chord 12/12; cleanup commit removed orphan problem.py

Cross-cut findings

• Numpy-only (hard pass): All 5 verified.
• Determinism (3 spot-checks): bit-identical across reruns.
• Branch protocol: zero wave-7-local/* on origin.
• Git authors: all 5 + cleanup commit by agent-0bserver07.
• Cleanliness (after cleanup): no TODO/FIXME, no hardcoded paths, no orphan problem.py stubs.

Algorithmic faithfulness

All 5 stubs use the right LSTM variant:

• temporal-order-4bit: original 1997 LSTM (input + output gates only, no forget)
• continual-embedded-reber: side-by-side LSTMForget (Gers 2000) vs LSTMNoForget (1997)
• anbn-anbncn: peephole LSTM (3 peephole vectors p_i, p_f from c_{t-1}, p_o from c_t per Gers 2002)
• timing-counting-spikes: peephole LSTM + vanilla LSTM contrast
• blues-improvisation: 2-layer stacked LSTM (h1=20 chord layer, h2=24 melody layer, forget gate)

Honest gap-reporting

timing-counting-spikes flagged as PARTIAL: peep-vs-vanilla contrast doesn't fully reproduce at short-MSD scale on laptop budget. §Deviations explicitly states: paper claims vanilla "fails entirely," but at T=150 D∈[30,60] vanilla reaches solve_rate=0.9. Follow-up flagged: T ≥ 300 + longer training would likely close the gap. This is the SPEC's methodological caveat applied correctly.

Reproduce results

=== temporal-order-4bit seed 42 (100 steps) ===
lstm_final_acc 0.27148 — identical across runs

=== blues-improvisation seed 123 (5 epochs) ===
deterministic bar-onset 0.667; step-level chord 0.667 — consistent

=== continual-embedded-reber seed 0 ===
LSTMForget outer T/P 1.000 vs LSTMNoForget 0.500 — headline contrast preserved

---

agent-0bserver07 (Claude Code) on behalf of Yad — wave-7 audit subagent