Merge pull request #16 from rivet-dev/ralph/wasmvm-dynamic-modules

chore: wasmvm dynamic modules & libc support

Author: NathanFlurry

.github/workflows/ci.yml

@@ -37,6 +37,34 @@ jobs:
       - name: Build WASM binary
         run: cd wasmvm && make wasm
 
+      # --- C toolchain (wasi-sdk + patched sysroot + C test fixtures) ---
+      - name: Cache wasi-sdk
+        id: cache-wasi-sdk
+        uses: actions/cache@v4
+        with:
+          path: wasmvm/c/vendor/wasi-sdk
+          key: wasi-sdk-25-${{ runner.os }}-${{ runner.arch }}
+
+      - name: Download wasi-sdk
+        if: steps.cache-wasi-sdk.outputs.cache-hit != 'true'
+        run: make -C wasmvm/c wasi-sdk
+
+      - name: Cache patched wasi-libc sysroot
+        id: cache-sysroot
+        uses: actions/cache@v4
+        with:
+          path: |
+            wasmvm/c/sysroot
+            wasmvm/c/vendor/wasi-libc
+          key: wasi-libc-sysroot-${{ runner.os }}-${{ hashFiles('wasmvm/patches/wasi-libc/*.patch', 'wasmvm/scripts/patch-wasi-libc.sh') }}
+
+      - name: Build patched wasi-libc sysroot
+        if: steps.cache-sysroot.outputs.cache-hit != 'true'
+        run: make -C wasmvm/c sysroot
+
+      - name: Build C test fixtures (WASM + native)
+        run: make -C wasmvm/c programs native
+
       # --- Node.js / TypeScript ---
       - name: Set up pnpm
         uses: pnpm/action-setup@v4

CLAUDE.md

@@ -33,17 +33,43 @@
 - check GitHub Actions test/typecheck status per commit to identify when a failure first appeared
 - do not use `contract` in test filenames; use names like `suite`, `behavior`, `parity`, `integration`, or `policy` instead
 
+## Tool Integration Policy
+
+- NEVER implement a from-scratch reimplementation of a tool when the PRD specifies using an existing upstream project (e.g., codex, curl, git, make)
+- always fork, vendor, or depend on the real upstream source — do not build a "stub" or "demo" binary that fakes the tool's behavior
+- if the upstream cannot compile or link for the target, document the specific blockers and leave the story as failing — do not mark it passing with a placeholder
+- the PRD and story notes define which upstream project to use; follow them exactly unless explicitly told otherwise
+
+## C Library Vendoring Policy
+
+- NEVER commit third-party C library source code directly into this repo
+- **unmodified upstream libraries** (sqlite3, zlib, minizip, cJSON, etc.) must be downloaded at build time from their official release URLs — add a Makefile target in `wasmvm/c/Makefile` under `fetch-libs`
+- **modified libraries** (e.g., libcurl with WASI patches) must live in a fork under the `rivet-dev` GitHub org (e.g., `rivet-dev/secure-exec-curl`) — the Makefile downloads from the fork's archive URL
+- all downloaded library sources go in `wasmvm/c/libs/` which is gitignored — they are fetched by `make fetch-libs` and cached in `wasmvm/c/.cache/`
+- when adding a new C library dependency: (1) add its download URL and Makefile target to `fetch-libs`, (2) add `libs/<name>` to the appropriate `.gitignore`, (3) if WASI modifications are needed, create a `rivet-dev/secure-exec-<name>` fork first
+- existing forks: `rivet-dev/secure-exec-curl` (libcurl with `wasi_tls.c` and `wasi_stubs.c`)
+
 ## WASM Binary
 
+- the goal for WasmVM is full POSIX compliance 1:1 — every command, syscall, and shell behavior should match a real Linux system exactly
 - WasmVM and Python are experimental surfaces in this repo
 - all docs for WasmVM, Python, or other experimental runtime features must live under the `Experimental` section of the docs navigation, not the main getting-started/reference sections
-- the WasmVM runtime requires a WASM binary at `wasmvm/target/wasm32-wasip1/release/multicall.wasm`
-- build it locally: `cd wasmvm && make wasm` (requires Rust nightly + wasm32-wasip1 target + rust-src component + wasm-opt/binaryen)
+- the WasmVM runtime requires standalone WASM binaries in `wasmvm/target/wasm32-wasip1/release/commands/`
+- build them locally: `cd wasmvm && make wasm` (requires Rust nightly + wasm32-wasip1 target + rust-src component + wasm-opt/binaryen)
 - the Rust toolchain is pinned in `wasmvm/rust-toolchain.toml` — rustup will auto-install it
-- CI builds the binary before tests; a CI-only guard test in `packages/runtime/wasmvm/test/driver.test.ts` fails if it's missing
-- tests gated behind `skipIf(!hasWasmBinary)` or `skipUnlessWasmBuilt()` will skip locally if the binary isn't built
+- CI builds the binaries before tests; a CI-only guard test in `packages/runtime/wasmvm/test/driver.test.ts` fails if they're missing
+- tests gated behind `skipIf(!hasWasmBinaries)` or `skipUnlessWasmBuilt()` will skip locally if binaries aren't built
 - see `wasmvm/CLAUDE.md` for full build details and architecture
 
+## WasmVM Syscall Coverage
+
+- every function in the `host_process` and `host_user` import modules (declared in `wasmvm/crates/wasi-ext/src/lib.rs`) must have at least one C parity test exercising it through libc
+- when adding a new host import, add a matching test case to `wasmvm/c/programs/syscall_coverage.c` and its parity test in `packages/runtime/wasmvm/test/c-parity.test.ts`
+- the canonical source of truth for import signatures is `wasmvm/crates/wasi-ext/src/lib.rs` — C patches and JS host implementations must match exactly
+- C patches in `wasmvm/patches/wasi-libc/` must be kept in sync with wasi-ext — ABI drift between C, Rust, and JS is a P0 bug
+- permission tier enforcement must cover ALL write/spawn/kill/pipe/dup operations — audit `packages/runtime/wasmvm/src/kernel-worker.ts` when adding new syscalls
+- `PATCHED_PROGRAMS` in `wasmvm/c/Makefile` must include all programs that use `host_process` or `host_user` imports (programs linking the patched sysroot)
+
 ## Terminology
 
 - use `docs-internal/glossary.md` for canonical definitions of isolate, runtime, bridge, and driver
@@ -124,6 +150,8 @@ Follow the style in `packages/secure-exec/src/index.ts`.
   - `docs/system-drivers/browser.mdx` — update when createBrowserDriver options change
   - `docs/nodejs-compatibility.mdx` — update when bridge, polyfill, or stub implementations change; keep the Tested Packages section current when adding or removing project-matrix fixtures
   - `docs/cloudflare-workers-comparison.mdx` — update when secure-exec capabilities change; bump "Last updated" date
+  - `docs/posix-compatibility.md` — update when kernel, WasmVM, Node bridge, or Python bridge behavior changes for any POSIX-relevant feature (signals, pipes, FDs, process model, TTY, VFS)
+  - `docs/wasmvm/supported-commands.md` — update when adding, removing, or changing status of WasmVM commands; keep summary counts current
 
 ## Backlog Tracking