Skip to content

Commit 287fa75

Browse files
NathanFlurryclaude
andcommitted
feat: WasmVM dynamic modules - POSIX-compliant WASM runtime (squashed ralph/wasmvm-dynamic-modules)
Full WasmVM runtime with POSIX-compliant process model, VFS, PTY, signals, pipes, FD table, and shell integration. Includes 98 user stories covering: - WasmVM Rust workspace migration and monorepo integration - Kernel process table, FD table, and VFS implementation - WASI host imports (host_process, host_user, host_net) - C toolchain with wasi-libc patches for process spawning - Interactive shell (brush-shell) with PTY support - Dynamic module loading and permission tiers - Network proxy and HTTP client via host_net - Codex CLI stub binaries (to be replaced with real fork) - C parity test suite for syscall coverage - Documentation and compatibility tracking Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 1f51b7f commit 287fa75

835 files changed

Lines changed: 516073 additions & 3569 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/ci.yml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,34 @@ jobs:
3737
- name: Build WASM binary
3838
run: cd wasmvm && make wasm
3939

40+
# --- C toolchain (wasi-sdk + patched sysroot + C test fixtures) ---
41+
- name: Cache wasi-sdk
42+
id: cache-wasi-sdk
43+
uses: actions/cache@v4
44+
with:
45+
path: wasmvm/c/vendor/wasi-sdk
46+
key: wasi-sdk-25-${{ runner.os }}-${{ runner.arch }}
47+
48+
- name: Download wasi-sdk
49+
if: steps.cache-wasi-sdk.outputs.cache-hit != 'true'
50+
run: make -C wasmvm/c wasi-sdk
51+
52+
- name: Cache patched wasi-libc sysroot
53+
id: cache-sysroot
54+
uses: actions/cache@v4
55+
with:
56+
path: |
57+
wasmvm/c/sysroot
58+
wasmvm/c/vendor/wasi-libc
59+
key: wasi-libc-sysroot-${{ runner.os }}-${{ hashFiles('wasmvm/patches/wasi-libc/*.patch', 'wasmvm/scripts/patch-wasi-libc.sh') }}
60+
61+
- name: Build patched wasi-libc sysroot
62+
if: steps.cache-sysroot.outputs.cache-hit != 'true'
63+
run: make -C wasmvm/c sysroot
64+
65+
- name: Build C test fixtures (WASM + native)
66+
run: make -C wasmvm/c programs native
67+
4068
# --- Node.js / TypeScript ---
4169
- name: Set up pnpm
4270
uses: pnpm/action-setup@v4

CLAUDE.md

Lines changed: 23 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -33,17 +33,34 @@
3333
- check GitHub Actions test/typecheck status per commit to identify when a failure first appeared
3434
- do not use `contract` in test filenames; use names like `suite`, `behavior`, `parity`, `integration`, or `policy` instead
3535

36+
## Tool Integration Policy
37+
38+
- NEVER implement a from-scratch reimplementation of a tool when the PRD specifies using an existing upstream project (e.g., codex, curl, git, make)
39+
- always fork, vendor, or depend on the real upstream source — do not build a "stub" or "demo" binary that fakes the tool's behavior
40+
- if the upstream cannot compile or link for the target, document the specific blockers and leave the story as failing — do not mark it passing with a placeholder
41+
- the PRD and story notes define which upstream project to use; follow them exactly unless explicitly told otherwise
42+
3643
## WASM Binary
3744

45+
- the goal for WasmVM is full POSIX compliance 1:1 — every command, syscall, and shell behavior should match a real Linux system exactly
3846
- WasmVM and Python are experimental surfaces in this repo
3947
- all docs for WasmVM, Python, or other experimental runtime features must live under the `Experimental` section of the docs navigation, not the main getting-started/reference sections
40-
- the WasmVM runtime requires a WASM binary at `wasmvm/target/wasm32-wasip1/release/multicall.wasm`
41-
- build it locally: `cd wasmvm && make wasm` (requires Rust nightly + wasm32-wasip1 target + rust-src component + wasm-opt/binaryen)
48+
- the WasmVM runtime requires standalone WASM binaries in `wasmvm/target/wasm32-wasip1/release/commands/`
49+
- build them locally: `cd wasmvm && make wasm` (requires Rust nightly + wasm32-wasip1 target + rust-src component + wasm-opt/binaryen)
4250
- the Rust toolchain is pinned in `wasmvm/rust-toolchain.toml` — rustup will auto-install it
43-
- CI builds the binary before tests; a CI-only guard test in `packages/runtime/wasmvm/test/driver.test.ts` fails if it's missing
44-
- tests gated behind `skipIf(!hasWasmBinary)` or `skipUnlessWasmBuilt()` will skip locally if the binary isn't built
51+
- CI builds the binaries before tests; a CI-only guard test in `packages/runtime/wasmvm/test/driver.test.ts` fails if they're missing
52+
- tests gated behind `skipIf(!hasWasmBinaries)` or `skipUnlessWasmBuilt()` will skip locally if binaries aren't built
4553
- see `wasmvm/CLAUDE.md` for full build details and architecture
4654

55+
## WasmVM Syscall Coverage
56+
57+
- every function in the `host_process` and `host_user` import modules (declared in `wasmvm/crates/wasi-ext/src/lib.rs`) must have at least one C parity test exercising it through libc
58+
- when adding a new host import, add a matching test case to `wasmvm/c/programs/syscall_coverage.c` and its parity test in `packages/runtime/wasmvm/test/c-parity.test.ts`
59+
- the canonical source of truth for import signatures is `wasmvm/crates/wasi-ext/src/lib.rs` — C patches and JS host implementations must match exactly
60+
- C patches in `wasmvm/patches/wasi-libc/` must be kept in sync with wasi-ext — ABI drift between C, Rust, and JS is a P0 bug
61+
- permission tier enforcement must cover ALL write/spawn/kill/pipe/dup operations — audit `packages/runtime/wasmvm/src/kernel-worker.ts` when adding new syscalls
62+
- `PATCHED_PROGRAMS` in `wasmvm/c/Makefile` must include all programs that use `host_process` or `host_user` imports (programs linking the patched sysroot)
63+
4764
## Terminology
4865

4966
- use `docs-internal/glossary.md` for canonical definitions of isolate, runtime, bridge, and driver
@@ -124,6 +141,8 @@ Follow the style in `packages/secure-exec/src/index.ts`.
124141
- `docs/system-drivers/browser.mdx` — update when createBrowserDriver options change
125142
- `docs/nodejs-compatibility.mdx` — update when bridge, polyfill, or stub implementations change; keep the Tested Packages section current when adding or removing project-matrix fixtures
126143
- `docs/cloudflare-workers-comparison.mdx` — update when secure-exec capabilities change; bump "Last updated" date
144+
- `docs/posix-compatibility.md` — update when kernel, WasmVM, Node bridge, or Python bridge behavior changes for any POSIX-relevant feature (signals, pipes, FDs, process model, TTY, VFS)
145+
- `docs/wasmvm/supported-commands.md` — update when adding, removing, or changing status of WasmVM commands; keep summary counts current
127146

128147
## Backlog Tracking
129148

docs-internal/review-notes.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -181,14 +181,14 @@ expect(stdout).not.toContain('root:x:0:0');
181181

182182
### Tests Gated Behind skipIf (May Not Run in CI)
183183

184-
4 WasmVM test suites require `multicall.wasm` binary (external Rust crate, not in repo):
185-
- `describe.skipIf(!hasWasmBinary)('real execution')` — echo, cat, false
186-
- `describe.skipIf(!hasWasmBinary)('stdin streaming')` — cat with writeStdin
187-
- `describe.skipIf(!hasWasmBinary)('proc_spawn routing')` — echo through kernel
184+
4 WasmVM test suites require standalone WASM binaries (built from Rust crates via `make wasm`):
185+
- `describe.skipIf(!hasWasmBinaries)('real execution')` — echo, cat, false
186+
- `describe.skipIf(!hasWasmBinaries)('stdin streaming')` — cat with writeStdin
187+
- `describe.skipIf(!hasWasmBinaries)('proc_spawn routing')` — echo through kernel
188188

189189
All E2E tests with real npm skip if npm registry unreachable.
190190

191-
**Risk:** If CI doesn't build the WASM binary or lacks network, these tests silently skip and the suite still passes green.
191+
**Risk:** If CI doesn't build WASM binaries or lacks network, these tests silently skip and the suite still passes green. (Mitigated: CI-only guard test asserts binaries exist.)
192192

193193
---
194194

@@ -335,7 +335,7 @@ What IS abstracted:
335335

336336
### P1 — Fix Before Production
337337
3. **Security boundary tests**: Add symlink escape, path traversal, host binary access, and resource limit enforcement tests.
338-
4. **WASM binary in CI**: Ensure CI builds multicall.wasm so gated tests actually run. Silent skips create false confidence.
338+
4. **WASM binaries in CI**: (RESOLVED) CI builds standalone WASM binaries via `make wasm`. Guard test fails if binaries are missing.
339339
5. **Permission wrapper tests**: The permission system exists but has zero test coverage. Add deny-scenario tests.
340340
6. **Replace negative security assertions**: "output doesn't contain X" tests should be replaced with positive assertions about error behavior.
341341

docs-internal/spec-hardening.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -204,12 +204,12 @@ Addresses bugs, test quality gaps, missing coverage, and documentation debt iden
204204

205205
### 14. WASM Binary CI Availability
206206

207-
**Problem:** WasmVM real execution tests are gated behind `skipIf(!hasWasmBinary)`. If CI doesn't build the Rust crate, all real execution tests silently skip. The test suite reports green despite not running critical tests.
207+
**Problem:** WasmVM real execution tests are gated behind `skipIf(!hasWasmBinaries)`. If CI doesn't build the Rust crates, all real execution tests silently skip. The test suite reports green despite not running critical tests.
208208

209-
**Acceptance criteria:**
210-
- CI pipeline builds `wasmvm/target/wasm32-wasip1/release/multicall.wasm` before test runs
211-
- OR: Add a CI-only test that asserts `hasWasmBinary === true` so CI fails if binary is missing
212-
- Document in CLAUDE.md how to build the WASM binary locally
209+
**Acceptance criteria:** (RESOLVED)
210+
- CI pipeline runs `make wasm` to build standalone binaries to `wasmvm/target/wasm32-wasip1/release/commands/`
211+
- CI-only test asserts `hasWasmBinaries === true` so CI fails if binaries are missing
212+
- CLAUDE.md documents how to build locally
213213

214214
### 15. Error String Matching → Structured Errors (WasmVM)
215215

0 commit comments

Comments
 (0)