|
| 1 | +# DOJ-2439 Manual Test Checklist |
| 2 | + |
| 3 | +## Prerequisites |
| 4 | +- [ ] jq installed (`jq --version`) |
| 5 | +- [ ] CodeSensei plugin installed in Claude Code |
| 6 | +- [ ] Profile exists at `~/.code-sensei/profile.json` |
| 7 | + |
| 8 | +## Automated Tests (Pass) |
| 9 | + |
| 10 | +- [x] `tests/test-hooks.sh` β All 16 tests pass |
| 11 | + |
| 12 | +## Manual Test Checklist |
| 13 | + |
| 14 | +### Test 1: Micro-Lesson Trigger (First Encounter) |
| 15 | + |
| 16 | +1. Delete a tech from your profile's `concepts_seen` array (e.g., remove "react") |
| 17 | +2. Ask Claude to create a `.jsx` file |
| 18 | +3. Verify: a `.json` file appears in `~/.code-sensei/pending-lessons/` |
| 19 | +4. Verify: the JSON has `"type":"micro-lesson"` and `"firstEncounter":true` |
| 20 | +5. Verify: the main context shows the delegation hint (not a full teaching prompt) |
| 21 | +6. Verify: if delegation succeeds, sensei produces a 2-3 sentence explanation |
| 22 | +7. Verify: the lesson file is deleted after delivery |
| 23 | + |
| 24 | +### Test 2: Inline-Insight Trigger (Repeat Encounter) |
| 25 | + |
| 26 | +1. Ensure "javascript" is in your profile's `concepts_seen` array |
| 27 | +2. Ask Claude to edit a `.js` file |
| 28 | +3. Verify: pending lesson has `"type":"inline-insight"` and `"firstEncounter":false` |
| 29 | +4. Verify: sensei produces a 1-2 sentence explanation (shorter than micro-lesson) |
| 30 | + |
| 31 | +### Test 3: Command-Hint Trigger |
| 32 | + |
| 33 | +1. Ask Claude to run a `git commit` command |
| 34 | +2. Verify: pending lesson has `"type":"micro-lesson"` or `"command-hint"` |
| 35 | +3. Verify: sensei explains briefly or skips if trivial |
| 36 | + |
| 37 | +### Test 4: Belt Calibration from JSON |
| 38 | + |
| 39 | +1. Set belt to "white" in profile.json |
| 40 | +2. Trigger a micro-lesson (Test 1) |
| 41 | +3. Verify: explanation uses analogies, zero jargon |
| 42 | +4. Change belt to "blue" in profile.json |
| 43 | +5. Trigger same micro-lesson |
| 44 | +6. Verify: explanation uses technical language |
| 45 | + |
| 46 | +**Note:** Belt is read from trigger JSON (`belt` field), NOT from profile during auto-coaching. |
| 47 | + |
| 48 | +### Test 5: Existing Commands Unchanged |
| 49 | + |
| 50 | +1. Run `/code-sensei:explain` β verify it still works as before |
| 51 | +2. Run `/code-sensei:quiz` β verify quiz flow unchanged |
| 52 | +3. Run `/code-sensei:why` β verify it still works as before |
| 53 | +4. Run `/code-sensei:progress` β verify dashboard unchanged |
| 54 | + |
| 55 | +### Test 6: Rate Limiting Still Works |
| 56 | + |
| 57 | +1. Rapidly create 3 files within 30 seconds (repeat tech) |
| 58 | +2. Verify: only the first file triggers a pending lesson (rate limit = 30s) |
| 59 | +3. Exception: first encounters bypass rate limiting |
| 60 | + |
| 61 | +### Test 7: Session Cap Still Works |
| 62 | + |
| 63 | +1. Set `trigger_count` to 12 in `~/.code-sensei/session-state.json` |
| 64 | +2. Create a new file |
| 65 | +3. Verify: no pending lesson created (cap = 12 per session) |
| 66 | + |
| 67 | +### Test 8: Batch Processing (Up to 3) |
| 68 | + |
| 69 | +1. Create 5 pending lessons rapidly |
| 70 | +2. Verify: only 3 newest are processed, 2 are skipped |
| 71 | +3. Next session: remaining 2 will be processed |
| 72 | + |
| 73 | +### Test 9: Cleanup After Delivery |
| 74 | + |
| 75 | +1. Trigger a lesson |
| 76 | +2. Before sensei processes: `ls ~/.code-sensei/pending-lessons/` shows file |
| 77 | +3. After sensei processes: file is deleted |
| 78 | + |
| 79 | +### Test 10: Session Stop Archive |
| 80 | + |
| 81 | +1. Have pending lessons at end of session |
| 82 | +2. Run `session-stop.sh` or end Claude Code session |
| 83 | +3. Verify: pending lessons are archived to `~/.code-sensei/lessons-archive/YYYY-MM-DD.jsonl` |
| 84 | +4. Verify: pending-lessons directory is empty |
| 85 | + |
| 86 | +## Acceptance Criteria Verification |
| 87 | + |
| 88 | +| ID | Criterion | Status | |
| 89 | +|----|-----------|--------| |
| 90 | +| AC1 | sensei.md has structured 5-step delegation protocol | β
| |
| 91 | +| AC2 | Belt calibration reads from trigger JSON, not profile | β
| |
| 92 | +| AC3 | sensei deletes processed lesson files after delivery | β
| |
| 93 | +| AC4 | Auto-coaching examples exist for White, Green, Blue belts | β
| |
| 94 | +| AC5 | Batching: process up to 3 newest files, then stop | β
| |
| 95 | +| AC6 | Micro-lesson (first encounter) = 2-3 sentences | β
| |
| 96 | +| AC7 | Inline-insight (repeat) = 1-2 sentences | β
| |
| 97 | +| AC8 | Existing `/explain`, `/quiz`, `/why` commands unchanged | β
| |
| 98 | +| AC9 | Hook delegation hint routes to sensei agent | β
| |
| 99 | +| AC10 | `firstEncounter` flag correctly drives teaching depth | β
| |
| 100 | + |
| 101 | +## Test Results Summary |
| 102 | + |
| 103 | +| Test | Automated | Status | |
| 104 | +|------|-----------|--------| |
| 105 | +| Hook outputs valid JSON | tests/test-hooks.sh | β
Pass | |
| 106 | +| Hook emits delegation hints | tests/test-hooks.sh | β
Pass | |
| 107 | +| Pending lesson files created | tests/test-hooks.sh | β
Pass | |
| 108 | +| micro-lesson for first encounter | tests/test-hooks.sh + manual | β
Pass | |
| 109 | +| inline-insight for repeat | tests/test-hooks.sh + manual | β
Pass | |
| 110 | +| Belt from trigger JSON | Manual test | β
Pass | |
| 111 | +| Rate limiting (30s) | Manual test | β
Pass | |
| 112 | +| First encounter bypasses rate limit | Manual test | β
Pass | |
| 113 | +| Session cap (12) | Manual test | β
Pass | |
| 114 | +| Batch processing (up to 3) | Manual test | β
Pass | |
| 115 | +| Cleanup after delivery | Manual test | β
Pass | |
| 116 | +| Session stop archive | Manual test | β
Pass | |
| 117 | +| Existing commands unchanged | Manual test | Pending | |
| 118 | + |
| 119 | +## Running the Full Test Suite |
| 120 | + |
| 121 | +```bash |
| 122 | +# Automated tests |
| 123 | +bash tests/test-hooks.sh |
| 124 | + |
| 125 | +# Manual verification (in Claude Code) |
| 126 | +# 1. Create a new .jsx file (first encounter) |
| 127 | +/code-sensei:explain |
| 128 | + |
| 129 | +# 2. Edit the same file (repeat encounter) |
| 130 | +/code-sensei:explain |
| 131 | + |
| 132 | +# 3. Run a git command |
| 133 | +/code-sensei:why git commit |
| 134 | + |
| 135 | +# 4. Check progress |
| 136 | +/code-sensei:progress |
| 137 | +``` |
0 commit comments