feat: US-067 - Fix Pi SDK import in NodeRuntime when Pi modules declare __filename

Author: NathanFlurry

.agent/contracts/node-runtime.md

@@ -240,6 +240,11 @@ The runtime MUST classify JavaScript modules using Node-compatible metadata rule
 - **WHEN** a package has `package.json` with `"type": "module"` and sandboxed code loads `./index.js`
 - **THEN** the runtime MUST evaluate the file as ESM semantics (including `import.meta` availability and ESM export behavior)
 
+#### Scenario: Require-transformed ESM keeps module-local `__filename` bindings
+- **WHEN** sandboxed code loads a package that is transformed for `require()` compatibility from ESM source using `import.meta.url`, and that source also declares its own `const __filename = fileURLToPath(import.meta.url)` or `const __dirname = dirname(__filename)`
+- **THEN** the runtime MUST compile and execute that module without colliding with the CommonJS wrapper parameters
+- **AND** any compatibility transform MUST preserve the module's own local bindings instead of rewriting source tokens to the wrapper globals
+
 #### Scenario: .js under type commonjs is treated as CJS
 - **WHEN** a package has `package.json` with `"type": "commonjs"` (or no ESM override) and sandboxed code loads `./index.js` via `require`
 - **THEN** the runtime MUST evaluate the file as CommonJS and return `module.exports`

CLAUDE.md

@@ -146,6 +146,7 @@
 - vendored `http2` nghttp2 error-path tests patch `internal/test/binding` `Http2Stream.prototype.respond`; keep that shim wired to the same bridge-facing `Http2Stream` / `internal/http2/util`.`NghttpError` constructors the runtime uses, or the tests stop exercising the real wrapper logic
 - bridge exports that userland constructs with `new` must be assigned as constructable function properties, not object-literal method shorthands; shorthand methods like `createReadStream() {}` are not constructable and vendored fs coverage calls `new fs.createReadStream(...)`
 - `/proc/sys/kernel/hostname` conformance hits both kernel-backed and standalone NodeRuntime paths; a procfs fix that only lands in the kernel layer still leaves `createTestNodeRuntime()` fs/FileHandle coverage red
+- require-transformed ESM must not rely on the CommonJS wrapper's `__filename` / `__dirname` parameter names; keep wrapper internals on private names, synthesize local CJS bindings only for plain CommonJS sources, and compute transformed `import.meta.url` from `pathToFileURL(__secureExecFilename).href`
 
 ## Virtual Kernel Architecture
 
@@ -172,7 +173,7 @@
 - instead, use proper tooling: `es-module-lexer` / `cjs-module-lexer` (the same WASM-based lexers Node.js uses), or run the transformation inside the V8 isolate where the JS engine handles parsing correctly
 - if a source transformation is needed at the bridge/host level, prefer a battle-tested library over hand-rolled regex
 - the V8 runtime already has dual-mode execution (`execute_script` for CJS, `execute_module` for ESM) — lean on V8's native module system rather than pre-transforming source on the host side
-- existing regex-based transforms (e.g., `convertEsmToCjs`, `transformDynamicImport`, `isESM`) are known technical debt and should be replaced
+- existing regex-based transforms (e.g., `convertEsmToCjs`, `transformDynamicImport`, `isESM`) are known technical debt and should be replaced; when `require()` compatibility needs `import.meta.url`, inject an internal file-URL helper instead of rewriting to the wrapper `__filename`
 
 
 ## Contracts (CRITICAL)

packages/core/isolate-runtime/src/inject/require-setup.ts

@@ -1,5 +1,6 @@
 // @ts-nocheck
 // This file is executed inside the isolate runtime.
+      const REQUIRE_TRANSFORM_MARKER = '/*__secure_exec_require_esm__*/';
       const __requireExposeCustomGlobal =
         typeof globalThis.__runtimeExposeCustomGlobal === "function"
           ? globalThis.__runtimeExposeCustomGlobal
@@ -3973,17 +3974,6 @@
 	          return parsed;
 	        }
 
-	        // Some CJS artifacts include import.meta.url probes that are valid in
-	        // ESM but a syntax error in Function()-compiled CJS wrappers.
-	        const normalizedSource =
-	          typeof source === 'string'
-	            ? source
-	                .replace(/import\.meta\.url/g, '__filename')
-	                .replace(/fileURLToPath\(__filename\)/g, '__filename')
-	                .replace(/url\.fileURLToPath\(__filename\)/g, '__filename')
-	                .replace(/fileURLToPath\.call\(void 0, __filename\)/g, '__filename')
-	            : source;
-
         // Create module object
         const module = {
           exports: {},
@@ -4001,15 +3991,22 @@
         try {
           // Wrap and execute the code
           let wrapper;
+          const isRequireTransformedEsm =
+            typeof source === 'string' &&
+            source.startsWith(REQUIRE_TRANSFORM_MARKER);
+          const wrapperPrologue = isRequireTransformedEsm
+            ? ''
+            : "var __filename = __secureExecFilename;\n" +
+              "var __dirname = __secureExecDirname;\n";
           try {
 	            wrapper = new Function(
 	              'exports',
 	              'require',
 	              'module',
-	              '__filename',
-	              '__dirname',
+	              '__secureExecFilename',
+	              '__secureExecDirname',
 	              '__dynamicImport',
-	              normalizedSource + '\n//# sourceURL=' + resolved
+	              wrapperPrologue + source + '\n//# sourceURL=' + resolved
 	            );
           } catch (error) {
             const details =

packages/core/src/generated/isolate-runtime.ts

@@ -1,6 +1,9 @@
 import { transform, transformSync } from "esbuild";
 import { init, initSync, parse } from "es-module-lexer";
 
+const REQUIRE_TRANSFORM_MARKER = "/*__secure_exec_require_esm__*/";
+const IMPORT_META_URL_HELPER = "__secureExecImportMetaUrl__";
+
 function isJavaScriptLikePath(filePath: string | undefined): boolean {
 	return filePath === undefined || /\.[cm]?[jt]sx?$/.test(filePath);
 }
@@ -12,21 +15,26 @@ function parseSourceSyntax(source: string, filePath?: string) {
 	return { hasModuleSyntax, hasDynamicImport, hasImportMeta };
 }
 
-function shouldTransformForRequire(source: string, filePath?: string): boolean {
-	if (!isJavaScriptLikePath(filePath)) {
-		return false;
+function getRequireTransformOptions(
+	filePath: string,
+	syntax: ReturnType<typeof parseSourceSyntax>,
+) {
+	const requiresEsmWrapper =
+		syntax.hasModuleSyntax || syntax.hasImportMeta;
+	const bannerLines = requiresEsmWrapper ? [REQUIRE_TRANSFORM_MARKER] : [];
+	if (syntax.hasImportMeta) {
+		bannerLines.push(
+			`const ${IMPORT_META_URL_HELPER} = require("node:url").pathToFileURL(__secureExecFilename).href;`,
+		);
 	}
 
-	const { hasModuleSyntax, hasDynamicImport, hasImportMeta } =
-		parseSourceSyntax(source, filePath);
-	return hasModuleSyntax || hasDynamicImport || hasImportMeta;
-}
-
-function getRequireTransformOptions(filePath: string) {
 	return {
-		define: {
-			"import.meta.url": "__filename",
-		},
+		banner: bannerLines.length > 0 ? bannerLines.join("\n") : undefined,
+		define: syntax.hasImportMeta
+			? {
+					"import.meta.url": IMPORT_META_URL_HELPER,
+				}
+			: undefined,
 		format: "cjs" as const,
 		loader: "js" as const,
 		platform: "node" as const,
@@ -62,12 +70,13 @@ export function transformSourceForRequireSync(
 	}
 
 	initSync();
-	if (!shouldTransformForRequire(source, filePath)) {
+	const syntax = parseSourceSyntax(source, filePath);
+	if (!(syntax.hasModuleSyntax || syntax.hasDynamicImport || syntax.hasImportMeta)) {
 		return source;
 	}
 
 	try {
-		return transformSync(source, getRequireTransformOptions(filePath)).code;
+		return transformSync(source, getRequireTransformOptions(filePath, syntax)).code;
 	} catch {
 		return source;
 	}
@@ -82,13 +91,14 @@ export async function transformSourceForRequire(
 	}
 
 	await init;
-	if (!shouldTransformForRequire(source, filePath)) {
+	const syntax = parseSourceSyntax(source, filePath);
+	if (!(syntax.hasModuleSyntax || syntax.hasDynamicImport || syntax.hasImportMeta)) {
 		return source;
 	}
 
 	try {
 		return (
-			await transform(source, getRequireTransformOptions(filePath))
+			await transform(source, getRequireTransformOptions(filePath, syntax))
 		).code;
 	} catch {
 		return source;

packages/nodejs/src/module-source.ts

@@ -1,4 +1,6 @@
 import { execFile } from "node:child_process";
+import { mkdtemp, rm, writeFile } from "node:fs/promises";
+import { tmpdir } from "node:os";
 import { dirname, resolve } from "node:path";
 import { fileURLToPath, pathToFileURL } from "node:url";
 import { promisify } from "node:util";
@@ -162,4 +164,94 @@ describe("standalone dist bootstrap", () => {
 			exports: { answer: 42 },
 		});
 	}, 30_000);
+
+	it("imports require-transformed ESM modules that declare __filename from import.meta.url", async () => {
+		const fixtureDir = await mkdtemp(
+			resolve(tmpdir(), "secure-exec-import-meta-url-"),
+		);
+		const fixturePath = resolve(fixtureDir, "entry.mjs");
+		await writeFile(
+			fixturePath,
+			[
+				'import { dirname } from "node:path";',
+				'import { fileURLToPath } from "node:url";',
+				'const __filename = fileURLToPath(import.meta.url);',
+				"const __dirname = dirname(__filename);",
+				"export const filePath = __filename;",
+				"export const dirPath = __dirname;",
+			].join("\n"),
+		);
+
+		try {
+			const stdout = await runStandaloneScript(`
+			import {
+				NodeRuntime,
+				allowAll,
+				createNodeDriver,
+				createNodeRuntimeDriverFactory,
+			} from ${JSON.stringify(DIST_INDEX_URL)};
+
+			const stdio = [];
+			const runtime = new NodeRuntime({
+				onStdio: (event) => {
+					if (event.channel === "stderr") {
+						stdio.push(event.message);
+					}
+				},
+				systemDriver: createNodeDriver({
+					moduleAccess: { cwd: ${JSON.stringify(WORKSPACE_ROOT)} },
+					permissions: allowAll,
+				}),
+				runtimeDriverFactory: createNodeRuntimeDriverFactory(),
+			});
+
+			const result = await runtime.exec(
+				\`(() => {
+					try {
+						const mod = require(${JSON.stringify(fixturePath)});
+						if (mod.filePath !== ${JSON.stringify(fixturePath)}) {
+							throw new Error("unexpected filePath export: " + mod.filePath);
+						}
+						console.log(JSON.stringify({
+							ok: true,
+							filePath: mod.filePath,
+							dirPath: mod.dirPath,
+						}));
+					} catch (error) {
+						console.error(String(error));
+						console.error(error && error.stack ? error.stack : "");
+						process.exitCode = 1;
+					}
+				})();\`,
+				{ cwd: ${JSON.stringify(WORKSPACE_ROOT)} },
+			);
+
+			await runtime.terminate();
+			await new Promise((resolve, reject) => {
+				process.stdout.write(JSON.stringify({
+					code: result.code,
+					stderr: stdio.join(""),
+				}), (error) => {
+					if (error) {
+						reject(error);
+						return;
+					}
+					resolve();
+				});
+			});
+			process.exit(0);
+		`);
+
+			const result = JSON.parse(stdout) as {
+				code: number;
+				stderr: string;
+			};
+
+			expect(result.code).toBe(0);
+			expect(result.stderr).not.toContain("Identifier '__filename' has already been declared");
+			expect(result.stderr).toBe("");
+		} finally {
+			await rm(fixtureDir, { recursive: true, force: true });
+		}
+	}, 30_000);
 });

packages/secure-exec/tests/runtime-driver/node/standalone-dist-smoke.test.ts

@@ -1125,8 +1125,24 @@
         "Typecheck passes"
       ],
       "priority": 53.1,
+      "passes": true,
+      "notes": "Fixed in the SecureExec runtime/module-loader layer. `transformSourceForRequire*()` now injects an internal file-URL helper for `import.meta.url` and marks require-transformed ESM, while the isolate CommonJS loader keeps wrapper filename/dirname on private parameter names and only synthesizes local `__filename`/`__dirname` bindings for plain CommonJS sources. Result: importing `@mariozechner/pi-coding-agent@0.60.0` through NodeRuntime no longer fails with `SyntaxError: Identifier '__filename' has already been declared`; the Pi import chain now advances to the next concrete blocker (`ENOENT` for `dist/package.json`)."
+    },
+    {
+      "id": "US-068",
+      "title": "Fix Pi package asset discovery after SDK import bootstrap",
+      "description": "As a developer, I need Pi's package metadata and bundled assets to resolve correctly inside SecureExec after the import bootstrap succeeds, so the SDK can continue booting beyond `dist/config.js`.",
+      "acceptanceCriteria": [
+        "Reproduce the new blocker after US-067 by importing `@mariozechner/pi-coding-agent@0.60.0` through NodeRuntime and observing the `ENOENT` for `dist/package.json`",
+        "SecureExec resolves Pi's package metadata/assets from the actual package root instead of trying to open `dist/package.json`",
+        "The fix applies in SecureExec's runtime/module/filesystem layer rather than by patching Pi locally",
+        "The Pi SDK real-provider harness advances beyond package-dir/bootstrap asset discovery and either completes successfully or exposes the next concrete blocker",
+        "Tests pass",
+        "Typecheck passes"
+      ],
+      "priority": 53.2,
       "passes": false,
-      "notes": "Blocked by SecureExec runtime/module-wrapper behavior. Exact repro as of 2026-03-26: `@mariozechner/pi-coding-agent@0.60.0` imports successfully in host Node, but inside NodeRuntime the import chain reaches `dist/config.js` and fails with `SyntaxError: Identifier '__filename' has already been declared`. Proposed unblock path: inspect the CommonJS compilation wrapper / loader path that injects `__filename`/`__dirname` and make it tolerate upstream packages that declare those identifiers in module scope."
+      "notes": "Next blocker exposed by US-067. Exact repro as of 2026-03-26: after the `__filename` collision fix, importing `@mariozechner/pi-coding-agent@0.60.0` through NodeRuntime reaches `dist/config.js` and then fails with `ENOENT: no such file or directory, open '<repo>/node_modules/.pnpm/.../@mariozechner/pi-coding-agent/dist/package.json'`. Proposed unblock path: inspect how the sandboxed FS / path resolution is handling Pi's package-root detection and why `getPackageDir()` stops at `dist/` instead of the package root."
     },
     {
       "id": "US-062",

scripts/ralph/prd.json

@@ -15,7 +15,7 @@
 - For overfitting-prone kernel/network stories, keep host-vs-sandbox cross-validation tests even when they currently expose a mismatch; assert the concrete exit code/stderr/raw-byte delta instead of downgrading back to another same-code-path loopback test.
 - Kernel-owned `SocketTable` instances must validate socket owner PIDs against the shared process table; only standalone/internal tables should omit that validation and stay opt-in.
 - `packages/wasmvm/src/driver.ts` resolves the worker entry to `packages/wasmvm/dist/kernel-worker.js` unless a sibling `src/kernel-worker.js` exists, so `packages/wasmvm/src/kernel-worker.ts` changes are not live until `pnpm --filter @secure-exec/wasmvm build` refreshes the worker bundle.
-- Replace active Node loader rewrites with parser-backed transforms: use `es-module-lexer` for `.js` module-syntax detection and `esbuild` CJS transforms with `supported: { "dynamic-import": false }` plus `define: { "import.meta.url": "__filename" }` so `require()` keeps module-relative `import()` behavior without touching strings/comments.
+- Replace active Node loader rewrites with parser-backed transforms: use `es-module-lexer` for `.js` module-syntax detection and `esbuild` CJS transforms with `supported: { "dynamic-import": false }`, inject a private file-URL helper from `pathToFileURL(__secureExecFilename).href`, and keep wrapper `__filename` / `__dirname` bindings private so require-transformed ESM can declare its own locals without collisions.
 - Standalone `NodeExecutionDriver` should always keep an internal `SocketTable` for sandbox loopback routing, but it must only attach `createNodeHostNetworkAdapter()` when `SystemDriver.network` is explicitly configured; otherwise omitted network capability silently regains host TCP access.
 - Bridge-level HTTP/fetch client work needs explicit in-flight tracking in `NodeExecutionDriver` state; otherwise `runtime.exec()` can finish before async bridge requests settle and payload-limit failures turn into false green exits.
 - Kernel-backed HTTP client routing should be limited to the loopback-aware default Node adapter path; preserve a documented legacy fallback for custom `NetworkAdapter` injections so adapter-focused tests and no-network stubs still exercise their own behavior.
@@ -722,3 +722,14 @@
   - For investigation stories that may uncover a runtime blocker, the opt-in harness should accept either true end-to-end success or the pinned blocker signature so the test stays useful before and after the runtime fix lands.
   - The current Pi SDK blocker is in SecureExec's module/runtime layer, not Pi's provider stack: host Node completes the real-token prompt, but sandbox import of `dist/config.js` dies on a duplicate `__filename` declaration before any provider request is made.
 ---
+## [2026-03-26 13:28 PDT] - US-067
+- Fixed the NodeRuntime require-transform path so ESM modules can declare their own `const __filename = fileURLToPath(import.meta.url)` without colliding with SecureExec's CommonJS wrapper globals.
+- Updated the host transform to inject a private file-URL helper for `import.meta.url`, taught the isolate loader to keep wrapper filename/dirname on private parameter names and only synthesize local `__filename` / `__dirname` bindings for plain CommonJS sources, and added a standalone-dist regression that requires a transformed ESM fixture outside Vitest's source transforms.
+- Confirmed the Pi SDK import now advances past the old bootstrap failure: importing `@mariozechner/pi-coding-agent@0.60.0` through `packages/secure-exec/dist/index.js` no longer throws `SyntaxError: Identifier '__filename' has already been declared` and instead exposes the next blocker, `ENOENT ... /dist/package.json`; recorded that follow-up as `US-068`.
+- Verified with `pnpm vitest run packages/secure-exec/tests/runtime-driver/node/standalone-dist-smoke.test.ts`, `pnpm --filter @secure-exec/nodejs check-types`, `pnpm --filter @secure-exec/core check-types`, and `pnpm --filter secure-exec check-types`.
+- Files changed: `AGENTS.md`, `.agent/contracts/node-runtime.md`, `packages/core/isolate-runtime/src/inject/require-setup.ts`, `packages/core/src/generated/isolate-runtime.ts`, `packages/nodejs/src/module-source.ts`, `packages/secure-exec/tests/runtime-driver/node/standalone-dist-smoke.test.ts`, `scripts/ralph/prd.json`, `scripts/ralph/progress.txt`
+- **Learnings for future iterations:**
+  - Require-transformed ESM needs different wrapper treatment than plain CommonJS: keep runtime filename/dirname on private parameter names and only synthesize `__filename` / `__dirname` locals for true CommonJS sources.
+  - `import.meta.url` compatibility cannot be implemented by rewriting to the wrapper `__filename`; it must stay a file URL string, which in this runtime means deriving it from `pathToFileURL(__secureExecFilename).href`.
+  - Once the duplicate-binding bootstrap bug is removed, Pi v0.60.0 immediately hits package-root discovery (`dist/package.json` ENOENT), so the next investigation should focus on sandboxed filesystem/path resolution rather than provider traffic.
+---