op-conductor,op-node: SSZ-binary commit-unsafe-payload endpoint

[BrianBland]

Summary

• New conductor HTTP endpoint: POST /commit-unsafe-payload accepts a raw SSZ-encoded ExecutionPayloadEnvelope body. Bypasses JSON-RPC's per-byte tokenizer cost.
• New op-node opt-in flag: --conductor.binary-commit (env OP_NODE_CONDUCTOR_BINARY_COMMIT). Uses the same conductor URL — binary endpoint shares the HTTP server with the JSON-RPC route.
• External CL clients (base's Rust CL replacement, etc.) can hit the endpoint with any HTTP client + SSZ encoder.

Wire protocol

POST /commit-unsafe-payload
Content-Type: application/octet-stream
Body: SSZ-encoded ExecutionPayloadEnvelope (no length prefix)

200 OK on success; 4xx/5xx with plain-text body otherwise. Body cap 16 MiB (fits 10 MB SSZ blocks comfortably).

Speedup

Loopback bench (M4 Max, 2 MB SSZ):

Path	E2E	Throughput
JSON-RPC (status quo)	38.4 ms	109 MB/s
Binary SSZ	1.4 ms	1.5 GB/s (~27x)

Production extrapolation (per DD #13980177 calibration — loopback throughput matches mainnet on a per-byte basis):

Scenario	Today (JSON-RPC)	Predicted (binary)	Speedup
Mainnet typical (~500 KB SSZ)	10.1 ms	~1.7 ms	~6x
Mainnet peak (~1.3 MB SSZ)	27.1 ms	~2.5 ms	~11x
10 MB SSZ (future goal)	~95 ms (blocked by 5 MB cap)	~7 ms	~12x

Floor is the HTTP machinery itself (~1.37 ms for 2 MB; verified by a discard-only handler).

Why not just hex-encode SSZ in JSON-RPC?

I tried that variant — same speed as structured JSON-RPC. The bottleneck is encoding/json lexing through every byte of the body looking for backslash escapes, regardless of whether it's one big string token or thousands of small tokens. Only escaping JSON entirely helps.

Why not gRPC / Unix socket / custom framed TCP?

• gRPC/HTTP2: ~same speed for one-shot POST, adds protobuf+codegen for no benefit since SSZ is already the wire format.
• Unix socket: ~5–10% over loopback TCP, only useful if colocated.
• Custom framed TCP: ~30–40% faster but loses HTTP middleware (JWT, TLS, observability) and language-agnostic clients (would block the Rust CL story).

Plain HTTP is already at the floor and stays Rust-friendly.

Op-node integration

ConductorClient.CommitUnsafePayload:
  if binaryClient != nil:
    binaryClient.CommitUnsafePayload(...)   # SSZ-marshal then HTTP POST
  else:
    apiClient.CommitUnsafePayload(...)       # JSON-RPC (existing)

Off by default. Existing JSON-RPC method untouched. Rollout is conductor-side first (additive — both routes served), then op-node opt-in per environment.

Stacked on

PR #35 (raft-mdb backend + commit-path metrics) — base branch is brianbland/conductor-raft-mdb. The metrics from #35 are what enable verifying the speedup in production.

Test plan

• Existing tests pass (consensus, conductor, rpc, ws, op-node/node)
• Devnet: 3-node conductor cluster with op-node using the new flag
• Compare commit_unsafe_payload latency before/after on devnet (the new metrics from [NOMERGE] op-conductor: add configurable raft-mdb backend #35 give the breakdown)
• Roll conductor side first (additive), then op-node per-env

🤖 Generated with Claude Code