Skip to content

Commit f8025e1

Browse files
NathanFlurryclaude
andcommitted
docs: add Node.js compatibility roadmap and conformance test integrity policy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent e556dc0 commit f8025e1

2 files changed

Lines changed: 3531 additions & 0 deletions

File tree

CLAUDE.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,17 @@
2929
- **no suite-specific VFS special-casing** — the test runner must not branch on suite name to inject different filesystem state; if a test needs files to exist, either the kernel should provide them or the test should be excluded
3030
- **categorize exclusions honestly** — if a failure is fixable with a patch or build flag, it's `implementation-gap`, not `wasm-limitation`; reserve `wasm-limitation` for things genuinely impossible in wasm32-wasip1 (no 80-bit long double, no fork, no mmap)
3131

32+
### Node.js Conformance Test Integrity
33+
34+
- conformance tests live in `packages/secure-exec/tests/node-conformance/` — they are vendored upstream Node.js v22.14.0 test/parallel/ tests run through the sandbox
35+
- `docs-internal/nodejs-compat-roadmap.md` tracks every non-passing test with its fix category and resolution
36+
- when implementing bridge/polyfill features where both sides go through our code (e.g., loopback HTTP server + client), prevent overfitting with these three mitigations:
37+
- **wire-level snapshot tests**: capture raw protocol bytes and compare against known-good captures from real Node.js
38+
- **project-matrix cross-validation**: add a project-matrix fixture (`tests/projects/`) using a real npm package that exercises the feature — the matrix compares sandbox output to host Node.js
39+
- **real-server control tests**: for network features, maintain tests that hit real external endpoints (not loopback) to validate the client independently of the server
40+
- never inflate conformance numbers — if a test self-skips (exits 0 without testing anything), mark it `vacuous-skip` in expectations.json, not as a real pass
41+
- every entry in `expectations.json` must have a specific, verifiable reason — no vague "fails in sandbox" reasons
42+
3243
## Tooling
3344

3445
- use pnpm, vitest, and tsc for type checks

0 commit comments

Comments
 (0)