Skip to content

Commit 62ab792

Browse files
NathanFlurryNathanFlurry
committed
feat(website): redesign benchmark charts with tabbed UI and standardized layout
1 parent 6bc5ebc commit 62ab792

11 files changed

Lines changed: 676 additions & 61 deletions

File tree

‎CLAUDE.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## Brand
2+
3+
- accent color: #CC0000 (red), light variant: #FF3333
4+
15
## Tooling
26

37
- use pnpm, vitest, and tsc for type checks

‎docs/cost-evaluation.mdx‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
title: "Cost Evaluation"
3+
description: "Cost-per-second comparison of Secure Exec on self-hosted hardware vs. sandbox providers"
4+
---
5+
6+
{/* Figures generated by scripts/calculate-costs.js — rerun when updating pricing */}
7+
8+
## Methodology
9+
10+
We compare the **cost per execution-second**: the cost of running one isolated execution for one second.
11+
12+
- **Sandbox providers**: minimum allocatable memory × provider's per-GiB-second rate. We use the cheapest provider (Cloudflare Containers) as the baseline.
13+
- **Secure Exec**: EC2/Hetzner instance cost ÷ number of concurrent executions that fit in RAM, with 30% empty capacity overhead to account for bin-packing inefficiency.
14+
15+
Secure Exec baseline memory per execution: **~3.4 MB** (converged at-scale average, p95). See [benchmarks](/docs/benchmarks) for measurement details.
16+
17+
## Results
18+
19+
All pricing as of March 2026. Almost all software runs on ARM today — we recommend comparing against ARM as a baseline.
20+
21+
| Provider | Arch | $/exec-second | vs. Sandbox |
22+
| --------------------------------- | ------ | -------------- | ----------- |
23+
| Cloudflare Containers (baseline) | — | $6.25 × 10⁻⁷ | 1× |
24+
| **AWS t4g.micro (ARM/Graviton)** | arm64 | $1.11 × 10⁻⁸ | **56× cheaper** |
25+
| AWS t3.micro (x86/Intel) | x86_64 | $1.38 × 10⁻⁸ | 45× cheaper |
26+
| **Hetzner CAX11 (ARM/Ampere)** | arm64 | $1.64 × 10⁻⁹ | **380× cheaper** |
27+
| Hetzner CX22 (x86/Intel) | x86_64 | $2.69 × 10⁻⁹ | 232× cheaper |
28+
29+
### Pricing Sources
30+
31+
- Cloudflare Containers: [$0.0000025/GiB·s, 256 MB minimum](https://developers.cloudflare.com/containers/platform/pricing/)
32+
- AWS EC2 t4g.micro: [$0.0084/hr, 1 GiB RAM](https://aws.amazon.com/ec2/pricing/on-demand/)
33+
- AWS EC2 t3.micro: [$0.0104/hr, 1 GiB RAM](https://aws.amazon.com/ec2/pricing/on-demand/)
34+
- Hetzner CAX11: [€3.29/mo, 4 GiB RAM](https://www.hetzner.com/cloud/)
35+
- Hetzner CX22: [€5.39/mo, 4 GiB RAM](https://www.hetzner.com/cloud/)
36+
37+
EUR→USD conversion: 1.09.
38+
39+
## Scenario Breakdown
40+
41+
### Lightweight Code Snippets
42+
43+
This is where Secure Exec shines the most.
44+
45+
- **~3.4 MB baseline** instead of 256 MB — you fit ~75× more executions per GB of RAM
46+
- Executions are **bin-packed** onto instances to maximize capacity utilization
47+
- Cold starts are **~16 ms** (p95) instead of **~950 ms** — you're not paying for time waiting for a sandbox to boot
48+
- On Hetzner ARM, lightweight snippets are **380× cheaper** than the cheapest sandbox provider
49+
50+
### Heavyweight Execution (e.g. Dev Servers)
51+
52+
Memory overhead matters less when dev servers use 512 MB+ regardless.
53+
54+
- The advantage shifts to **hardware choice**: you pick your cloud, instance type, and region
55+
- Sandbox providers vendor-lock you to their hardware and pricing
56+
- Self-hosted options like **Hetzner** or **AWS Spot instances** can dramatically reduce costs
57+
- No egress fees on most self-hosted providers
58+
59+
### Summary
60+
61+
| Workload | Primary cost advantage |
62+
| --------------------- | ----------------------------------------- |
63+
| Light snippets | 56–380× cheaper (memory + cold start) |
64+
| Heavy (dev servers) | Hardware choice, no vendor lock-in |
65+
| High concurrency | Bin-packing, no per-sandbox minimums |
66+
67+
See also: [Sandbox vs. Secure Exec](/docs/sandbox-vs-secure-exec) for a feature-level comparison.

‎docs/docs.json‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,9 @@
7171
"api-reference",
7272
"architecture",
7373
"security-model",
74+
75+
"cost-evaluation",
76+
"sandbox-vs-secure-exec",
7477
"nodejs-compatibility",
7578
"python-compatibility",
7679
"cloudflare-workers-comparison",

‎docs/sandbox-vs-secure-exec.mdx‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
title: "Sandbox vs. Secure Exec"
3+
description: "When to use a container sandbox vs. Secure Exec for code execution"
4+
---
5+
6+
## How They Work
7+
8+
**Container sandboxes** (e2b, Daytona, Modal, Cloudflare Containers) run each execution in an isolated VM or container with its own OS, filesystem, and network stack. You connect over a network API.
9+
10+
**Secure Exec** runs each execution in a V8 isolate inside your Node.js process. There is no separate container, no network hop, and no vendor dependency. Isolation is enforced at the V8 engine level — the same technology that powers Cloudflare Workers.
11+
12+
## Comparison
13+
14+
| Dimension | Secure Exec | Container Sandbox |
15+
| ---------------------- | ------------------------------ | ------------------------------ |
16+
| Isolation | V8 isolate | VM / container |
17+
| Performance | Native V8 | Native container |
18+
| Cold start | ~16 ms (p95) | ~950 ms (best provider) |
19+
| Memory baseline | ~3.4 MB | ~256 MB minimum |
20+
| Permissions | Granular, deny-by-default | Coarse-grained |
21+
| Infrastructure | `npm install` | Vendor account + API keys |
22+
| Hardware | Your choice | Vendor-locked |
23+
| Egress | None (in-process) | Per-GB fees |
24+
| Language support | Node.js, Python | Any (full OS) |
25+
| Filesystem | Virtual, scoped | Full OS filesystem |
26+
| Network access | Configurable per-execution | Full (or firewall rules) |
27+
28+
## When to Use Each
29+
30+
### Use Secure Exec when:
31+
32+
- You need **fast, lightweight** code execution (AI tool calls, user scripts, plugins)
33+
- You want **no vendor dependency** and full control over infrastructure
34+
- You need **granular permissions** (file, network, module-level)
35+
- Cost matters at scale — especially for high-concurrency lightweight workloads
36+
37+
### Use a container sandbox when:
38+
39+
- You need a **full OS environment** (install system packages, run arbitrary binaries)
40+
- You need languages beyond Node.js/Python with full OS-level access
41+
- You need **persistent, long-lived** environments (multi-hour dev sessions)
42+
- Workloads are heavyweight enough that the 256 MB minimum doesn't matter
43+
44+
## Cost
45+
46+
See [Cost Evaluation](/docs/cost-evaluation) for detailed per-second cost comparison across providers.

0 commit comments

Comments
 (0)