Commit d84a850
committed
fix(system-prompt): resolve agents using training data instead of web research tools
**Problem:**
Agents were generating fake URLs and listings from training data instead of using
web_operations tool, even when user explicitly requested "internet research".
**Root Causes Identified:**
1. Commit 111e1e5 removed behavioral enforcement:
- Removed: "Do not stop when encountering uncertainty — research or deduce..."
- Lost action trigger AND persistence guidance
2. TodoReminderInjector conflicted with system prompt:
- System prompt: "Use tools FIRST"
- Todo reminder: "DO THE ACTUAL WORK (tell the story...)"
- Agents interpreted "tell the story" as "generate from memory"
- Todo workflow took priority over system prompt
3. Massive duplication in system prompt:
- Current date context appeared twice (prepended + in components)
- Research guidance scattered across multiple sections
- ~13KB prompt with significant redundancy
**Solutions Implemented:**
1. **System Prompt - Research-First Principle:**
- Added to buildToolUsage() (tools-guarded)
- Clear failure framing: "Training data is a FAILURE condition"
- Examples: "Find listings" → use web_operations
- Prohibited fallbacks: fake URLs, "prior context", training data
2. **Todo Reminders - System Prompt Reference:**
- Changed: "DO THE ACTUAL WORK (tell the story...)"
- To: "DO THE ACTUAL WORK following your system prompt instructions"
- Removes conflict, todo workflow defers to system prompt
3. **System Prompt - Removed Duplication:**
- Removed prepended date/critical section
- Date context only appears once (in components)
- Fixed systemPrompt variable declaration after cleanup
**Testing:**
✅ Build: PASS (Swift build complete)
✅ User validation: Bug resolved
- Without fix + todo list: Agent avoided web tools, used training data
- With fix + todo list: Agent follows system prompt, uses web_operations
**Files Modified:**
- Sources/ConfigurationSystem/SystemPromptConfiguration.swift
- Sources/MCPFramework/TodoReminderInjector.swift
**Lessons Learned:**
- Runtime reminders can override static prompts (must align)
- Duplication creates confusion, not clarity (single source of truth)
- Agent behavior changes based on workflow context (todos vs no todos)
- "Simplification" must preserve behavioral enforcement
**Documentation:**
Created comprehensive handoff: ai-assisted/2026-01-05/system-prompt-debugging/
**Next Steps:**
Study prompt.txt and work with user to reduce complexity/redundancy while
maintaining effectiveness of research-first principle.1 parent 0284173 commit d84a850
2 files changed
Lines changed: 29 additions & 40 deletions
File tree
- Sources
- ConfigurationSystem
- MCPFramework
Lines changed: 25 additions & 36 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
300 | 300 | | |
301 | 301 | | |
302 | 302 | | |
| 303 | + | |
| 304 | + | |
303 | 305 | | |
304 | 306 | | |
305 | 307 | | |
306 | | - | |
| 308 | + | |
| 309 | + | |
307 | 310 | | |
308 | 311 | | |
309 | 312 | | |
| |||
357 | 360 | | |
358 | 361 | | |
359 | 362 | | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
360 | 383 | | |
361 | 384 | | |
362 | 385 | | |
| |||
1173 | 1196 | | |
1174 | 1197 | | |
1175 | 1198 | | |
1176 | | - | |
1177 | | - | |
1178 | | - | |
1179 | | - | |
1180 | | - | |
1181 | | - | |
1182 | | - | |
1183 | | - | |
1184 | | - | |
1185 | | - | |
1186 | | - | |
1187 | | - | |
1188 | | - | |
1189 | | - | |
1190 | | - | |
1191 | | - | |
1192 | | - | |
1193 | | - | |
1194 | | - | |
1195 | | - | |
1196 | | - | |
1197 | | - | |
1198 | | - | |
1199 | | - | |
1200 | | - | |
1201 | | - | |
1202 | | - | |
1203 | | - | |
1204 | | - | |
1205 | | - | |
1206 | | - | |
1207 | | - | |
1208 | | - | |
1209 | 1199 | | |
1210 | 1200 | | |
1211 | 1201 | | |
1212 | 1202 | | |
1213 | 1203 | | |
1214 | 1204 | | |
1215 | | - | |
| 1205 | + | |
1216 | 1206 | | |
1217 | | - | |
1218 | 1207 | | |
1219 | 1208 | | |
1220 | 1209 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
111 | 111 | | |
112 | 112 | | |
113 | 113 | | |
114 | | - | |
| 114 | + | |
115 | 115 | | |
116 | 116 | | |
117 | 117 | | |
118 | 118 | | |
119 | 119 | | |
120 | | - | |
| 120 | + | |
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
| |||
128 | 128 | | |
129 | 129 | | |
130 | 130 | | |
131 | | - | |
| 131 | + | |
132 | 132 | | |
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
136 | | - | |
| 136 | + | |
137 | 137 | | |
138 | 138 | | |
139 | 139 | | |
| |||
0 commit comments