Merge pull request #44 from rivet-dev/nathan/prd-in-vm-enforcement

Enforce in-VM execution for all CLI tool tests

Author: NathanFlurry

CLAUDE.md

@@ -17,9 +17,30 @@
 
 - NEVER mock external services in tests — use real implementations (Docker containers for databases/services, real HTTP servers for network tests, real binaries for CLI tool tests)
 - tests that validate sandbox behavior MUST run code through the secure-exec sandbox (NodeRuntime/proc.exec()), never directly on the host
-- CLI tool tests (Pi, Claude Code, OpenCode) must execute inside the sandbox: Pi runs as JS in the VM, Claude Code and OpenCode spawn their binaries via the sandbox's child_process.spawn bridge
+- NOTHING runs on the host except Docker containers for e2e-docker tests — all CLI tool code executes inside the sandbox VM
+- Pi, Claude Code, and OpenCode are ALL pure JavaScript/TypeScript — they ALL run in-VM via `import()` through `kernel.openShell()` or `kernel.spawn()`
+- Claude Code is a bundled ESM Node.js script (`@anthropic-ai/claude-code/cli.js`), not a native binary — its `.node` addons (tree-sitter, audio-capture) are optional and gracefully degrade
+- OpenCode is TypeScript (https://github.com/anomalyco/opencode) — the distributed `opencode` command is a Bun-compiled binary but the source is vanilla TS/JS; build the JS bundle from source and run it in-VM
+- if the sandbox can't run something, that is a secure-exec bug to fix, not a reason to spawn on the host
+- NEVER work around sandbox limitations with host-side execution — this is the #1 rule for CLI tool tests:
+  - do NOT use `child_process.spawn` or `child_process.spawnSync` from INSIDE sandbox code to run a tool binary on the host (e.g. `spawnSync('claude', [...args])` routing through the child_process bridge) — this is still host execution, the tool's JS runs on the host not in the V8 isolate
+  - do NOT use `node:child_process.spawn` from TEST code to run tools on the host
+  - do NOT create `HostBinaryDriver` classes that spawn binaries on the host
+  - do NOT use `script -qefc` or `python3 pty.spawn` to give host processes a PTY
+  - do NOT add `sandboxSkip` / probe-based skip logic that silently skips when the sandbox can't do something
+  - do NOT mark a story as passing if the tool runs on the host instead of in the V8 isolate
+  - the ONLY correct pattern is: `kernel.spawn('node', ['-e', 'import("tool-entry.js")'])` or equivalent — the tool's JavaScript executes inside the V8 sandbox isolate
+  - if `import()` hangs, if ESM loading fails, if the TUI crashes — those are secure-exec bugs to fix in packages/nodejs/src/, packages/core/src/, or native/v8-runtime/src/
 - e2e-docker fixtures connect to real Docker containers (Postgres, MySQL, Redis, SSH/SFTP) — skip gracefully via `skipUnlessDocker()` when Docker is unavailable
 - interactive/PTY tests must use `kernel.openShell()` with `@xterm/headless`, not host PTY via `script -qefc`
+- CLI tool tests (Pi, Claude Code, OpenCode) must support both mock and real LLM API tokens:
+  - check `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` env vars at test startup
+  - if a real token is present, use it instead of the mock LLM server — this validates true e2e behavior
+  - Pi supports both Anthropic and OpenAI tokens; OpenCode uses OpenAI; Claude Code uses Anthropic
+  - log which mode each test suite is using at startup: `"Using real ANTHROPIC_API_KEY"`, `"Using real OPENAI_API_KEY"`, or `"Using mock LLM server"`
+  - tests must pass with both mock and real tokens — mock is the fallback, real is preferred
+  - to run with real tokens locally: `source ~/misc/env.txt` before running tests
+  - real-token tests may use longer timeouts (up to 60s) since they hit external APIs
 
 ## Tooling
 

native/v8-runtime/Cargo.lock

@@ -10,7 +10,7 @@ name = "secure-exec-v8"
 path = "src/main.rs"
 
 [dependencies]
-v8 = "130"
+v8 = "134"
 crossbeam-channel = "0.5"
 signal-hook = "0.3"
 libc = "0.2"

native/v8-runtime/Cargo.toml

@@ -528,8 +528,7 @@ pub fn resolve_pending_promise(
         resolver.resolve(scope, undef.into());
     }
 
-    // Flush microtasks after resolution
-    scope.perform_microtask_checkpoint();
+    // Microtask checkpoint is the caller's responsibility (explicit policy).
 
     Ok(())
 }