Skip to content

Commit 77a6032

Browse files
NathanFlurryclaude
andcommitted
feat: add S3 and SQLite virtual file system driver examples
- examples/virtual-file-system-s3: S3-backed VirtualFileSystem using @aws-sdk/client-s3, with Docker Compose for MinIO - examples/virtual-file-system-sqlite: SQLite-backed VirtualFileSystem using sql.js (WASM), no native deps - Both tested end-to-end: sandbox writes/reads files, host verifies via storage API - docs/features/virtual-filesystem.mdx: linked examples with use case descriptions Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 6b2d903 commit 77a6032

15 files changed

Lines changed: 1240 additions & 9 deletions

File tree

docs/features/virtual-filesystem.mdx

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -167,10 +167,30 @@ const result = await runtime.exec(`
167167
console.log(config.greeting);
168168
`);
169169

170-
console.log(result.stdout); // "hello\n"
170+
// Output captured via onStdio callback — see Output Capture docs
171171
runtime.dispose();
172172
```
173173

174+
## More examples
175+
176+
Full, tested implementations for common storage backends:
177+
178+
<CardGroup cols={2}>
179+
<Card title="S3 / MinIO" icon="cloud" href="https://github.com/rivet-dev/secure-exec/tree/main/examples/virtual-file-system-s3">
180+
Store sandbox files as S3 objects. Works with any S3-compatible store (AWS, MinIO, R2). Includes Docker Compose for local testing with MinIO.
181+
</Card>
182+
<Card title="SQLite" icon="database" href="https://github.com/rivet-dev/secure-exec/tree/main/examples/virtual-file-system-sqlite">
183+
Store sandbox files as rows in a SQLite database (via sql.js / WASM). Snapshot and restore entire filesystems. No external services needed.
184+
</Card>
185+
</CardGroup>
186+
187+
Other use cases you can build with `VirtualFileSystem`:
188+
189+
- **Git-backed storage** — store files in a bare Git repo, with each sandbox session as a commit
190+
- **Encrypted filesystem** — wrap another VFS and encrypt/decrypt content transparently
191+
- **Union / overlay filesystem** — layer a writable VFS on top of a read-only base (e.g. template files + user modifications)
192+
- **Remote filesystem** — proxy file operations to a remote server via HTTP or gRPC
193+
174194
## Tips
175195

176196
- **Throw Node-style errors.** Sandboxed code expects errors like `ENOENT`, `EACCES`, and `EISDIR`. Match the error message prefix so `fs` error handling works naturally.
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
# S3 Filesystem Driver Example
2+
3+
A custom filesystem driver backed by S3 (or any S3-compatible store like [MinIO](https://min.io)). Sandboxed code uses `fs.readFileSync`, `fs.writeFileSync`, and other Node.js filesystem APIs as normal — all I/O is transparently routed to S3 objects.
4+
5+
[Docs](https://secureexec.dev/docs) | [GitHub](https://github.com/rivet-dev/secure-exec)
6+
7+
## Start the test server
8+
9+
Run MinIO locally with Docker:
10+
11+
```bash
12+
docker compose up -d
13+
```
14+
15+
Or without Docker Compose:
16+
17+
```bash
18+
docker run -d -p 9000:9000 -p 9001:9001 \
19+
-e MINIO_ROOT_USER=minioadmin \
20+
-e MINIO_ROOT_PASSWORD=minioadmin \
21+
minio/minio server /data --console-address ":9001"
22+
```
23+
24+
MinIO console is available at http://localhost:9001 (login: minioadmin / minioadmin).
25+
26+
## Run the example
27+
28+
```bash
29+
pnpm install
30+
pnpm test
31+
```
32+
33+
This creates a `NodeRuntime` with the S3-backed filesystem, runs sandboxed code that writes and reads files, then verifies the data was persisted to MinIO via the S3 API.
34+
35+
## How it works
36+
37+
- `S3FileSystem` implements the `VirtualFileSystem` interface from `secure-exec`
38+
- Files are stored as S3 objects keyed by their path (leading `/` stripped)
39+
- Directories are inferred from key prefixes; `mkdir` creates empty marker objects
40+
- `readDir` uses `ListObjectsV2` with delimiter-based prefix listing
41+
- Symlinks and hard links throw `ENOSYS` (not practical for object storage)
42+
- `chmod`, `chown`, `utimes` are no-ops (S3 doesn't have POSIX permissions)
43+
44+
## Use cases
45+
46+
- **Persistent sandboxes**: User-generated code writes files that survive across sessions
47+
- **Multi-tenant isolation**: Each sandbox gets its own S3 prefix or bucket
48+
- **Cloud-native deployments**: Store sandbox artifacts in the same object store as your application
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
services:
2+
minio:
3+
image: minio/minio
4+
ports:
5+
- "9000:9000"
6+
- "9001:9001"
7+
environment:
8+
MINIO_ROOT_USER: minioadmin
9+
MINIO_ROOT_PASSWORD: minioadmin
10+
command: server /data --console-address ":9001"
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
{
2+
"name": "@secure-exec/example-virtual-file-system-s3",
3+
"private": true,
4+
"type": "module",
5+
"scripts": {
6+
"check-types": "tsc --noEmit -p tsconfig.json",
7+
"test": "tsx src/index.ts"
8+
},
9+
"dependencies": {
10+
"@aws-sdk/client-s3": "^3.700.0",
11+
"secure-exec": "workspace:*"
12+
},
13+
"devDependencies": {
14+
"@types/node": "^22.10.2",
15+
"typescript": "^5.7.2"
16+
}
17+
}
Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
import {
2+
NodeRuntime,
3+
allowAllFs,
4+
createNodeDriver,
5+
createNodeRuntimeDriverFactory,
6+
} from "secure-exec";
7+
import {
8+
S3Client,
9+
CreateBucketCommand,
10+
DeleteBucketCommand,
11+
ListObjectsV2Command,
12+
DeleteObjectCommand,
13+
} from "@aws-sdk/client-s3";
14+
import { S3FileSystem } from "./s3-filesystem.js";
15+
16+
const BUCKET = "secure-exec-vfs-test";
17+
18+
// Connect to MinIO (S3-compatible) running on localhost
19+
const client = new S3Client({
20+
endpoint: "http://localhost:9000",
21+
region: "us-east-1",
22+
credentials: {
23+
accessKeyId: "minioadmin",
24+
secretAccessKey: "minioadmin",
25+
},
26+
forcePathStyle: true,
27+
});
28+
29+
// Create test bucket
30+
try {
31+
await client.send(new CreateBucketCommand({ Bucket: BUCKET }));
32+
} catch (err: unknown) {
33+
const e = err as { name?: string };
34+
if (
35+
e.name !== "BucketAlreadyOwnedByYou" &&
36+
e.name !== "BucketAlreadyExists"
37+
) {
38+
throw err;
39+
}
40+
}
41+
42+
const filesystem = new S3FileSystem({ client, bucket: BUCKET });
43+
44+
const runtime = new NodeRuntime({
45+
systemDriver: createNodeDriver({
46+
filesystem,
47+
permissions: { ...allowAllFs },
48+
}),
49+
runtimeDriverFactory: createNodeRuntimeDriverFactory(),
50+
});
51+
52+
try {
53+
// Sandbox writes files — stored in MinIO via S3 API
54+
const writeResult = await runtime.exec(`
55+
const fs = require("node:fs");
56+
fs.mkdirSync("/workspace", { recursive: true });
57+
fs.writeFileSync("/workspace/hello.txt", "hello from sandbox via S3");
58+
fs.writeFileSync("/workspace/data.json", JSON.stringify({ count: 42 }));
59+
console.log("files written");
60+
`);
61+
62+
if (writeResult.code !== 0) {
63+
throw new Error(`Write step failed: ${writeResult.errorMessage}`);
64+
}
65+
66+
// Sandbox reads files back
67+
let readOutput = "";
68+
const readResult = await runtime.exec(
69+
`
70+
const fs = require("node:fs");
71+
const text = fs.readFileSync("/workspace/hello.txt", "utf8");
72+
const data = JSON.parse(fs.readFileSync("/workspace/data.json", "utf8"));
73+
const entries = fs.readdirSync("/workspace");
74+
console.log(JSON.stringify({ text, count: data.count, entries }));
75+
`,
76+
{
77+
onStdio: (event) => {
78+
if (event.channel === "stdout") readOutput += event.message;
79+
},
80+
},
81+
);
82+
83+
if (readResult.code !== 0) {
84+
throw new Error(`Read step failed: ${readResult.errorMessage}`);
85+
}
86+
87+
// Verify from host side via S3 API
88+
const hostContent = await filesystem.readTextFile("/workspace/hello.txt");
89+
const parsed = JSON.parse(readOutput.trim());
90+
91+
const ok =
92+
hostContent === "hello from sandbox via S3" &&
93+
parsed.text === "hello from sandbox via S3" &&
94+
parsed.count === 42 &&
95+
parsed.entries.includes("hello.txt") &&
96+
parsed.entries.includes("data.json");
97+
98+
console.log(
99+
JSON.stringify({
100+
ok,
101+
hostContent,
102+
sandboxResult: parsed,
103+
summary:
104+
"S3-backed VFS: sandbox wrote files to MinIO, read them back, host verified via S3 API",
105+
}),
106+
);
107+
} finally {
108+
runtime.dispose();
109+
110+
// Clean up: delete all objects and the bucket
111+
const list = await client.send(
112+
new ListObjectsV2Command({ Bucket: BUCKET }),
113+
);
114+
for (const obj of list.Contents ?? []) {
115+
await client.send(
116+
new DeleteObjectCommand({ Bucket: BUCKET, Key: obj.Key! }),
117+
);
118+
}
119+
await client.send(new DeleteBucketCommand({ Bucket: BUCKET }));
120+
}

0 commit comments

Comments
 (0)