Skip to content

Resolve and deliver tong sidecar secrets through host provider CLIs#10

Merged
CrypticSwarm merged 1 commit into
masterfrom
tongs-phase-6-secret-providers
Jun 17, 2026
Merged

Resolve and deliver tong sidecar secrets through host provider CLIs#10
CrypticSwarm merged 1 commit into
masterfrom
tongs-phase-6-secret-providers

Conversation

@CrypticSwarm

Copy link
Copy Markdown
Owner

Summary

Sidecar (tong) definitions reference secrets as ${secret:<provider>:<ref>}.
This adds host-side resolution of those references and a leak-resistant path for
delivering the resolved values into a sidecar.

Secrets are resolved on the host by shelling out to a provider CLI declared once
in the user layer (~/.swarmforge/secret-providers.yaml), e.g.
op: ["op", "read", "{ref}"]. This is the docker-credential-helper pattern, so
Swarmforge knows nothing about any individual secret manager and 1Password, pass,
AWS, Doppler, etc. work without bespoke support. Interactive unlocks (op signin,
biometrics) work because the launcher runs in the user's terminal.

Resolved secrets reach a sidecar through an in-memory tmpfs file plus a
<NAME>_FILE env pointer, never a docker -e env var — anything holding the
docker socket could otherwise read an env value back via docker inspect.

These are host-side helpers exercised only when a definition carries a secret
reference; a launch for a repo with no sidecar definitions is unchanged.

Changes

  • Add pure helpers in scripts/tongs.py:
    • load_secret_providers parses the user-layer provider table, returning {}
      for a missing file and raising on a malformed one so a typo surfaces at load
      time; secret_provider_command substitutes the literal {ref} token in
      every argv element.
    • partition_secret_env, secret_delivery_plan, and plan_tong_secrets route
      secret-bearing env to tmpfs files with <NAME>_FILE pointers while leaving
      plain env as -e. The resolved value only ever appears in the file plan,
      never in the -e env. An env name that would escape the tmpfs dir as a path
      component is refused, and a file pointer that collides with a declared env
      var keeps the declared value with a warning.
  • Add make_secret_resolver in scripts/run_anvil.py: the impure resolver
    closure that shells out to the provider CLI and returns the secret, raising a
    clear error — naming the provider/reference, never the secret — on an unknown
    provider, an unrunnable CLI, or a non-zero exit. A single trailing newline is
    stripped; other whitespace is preserved.
  • Extend scripts/test_tongs.py and scripts/test_run_anvil.py with coverage
    for provider-table loading and validation, {ref} substitution, the exec
    resolver (success, newline trimming, unknown provider, non-zero exit,
    unrunnable CLI), the secret-never-in-env invariant end to end, the
    traversal-name rejection, and the pointer-collision warning. The existing
    passthrough-invariant tests still pass.

Tong definitions reference secrets as ${secret:<provider>:<ref>}. Resolve
them on the host by shelling out to a provider CLI declared once in the
user layer (~/.swarmforge/secret-providers.yaml), the
docker-credential-helper pattern, so Swarmforge knows nothing about any
individual secret manager. Loading the provider table and building the
argv are pure (in tongs.py); the subprocess that runs the CLI lives in the
launcher's resolver closure, which surfaces a clear error -- never the
secret -- on an unknown provider, an unrunnable CLI, or a non-zero exit.
Only the CLI's stdout is captured; its stderr passes through to the user's
terminal, so interactive unlocks (op signin, biometrics) stay visible.

Deliver resolved secrets to a tong through an in-memory tmpfs file rather
than a docker -e env var: anything holding the docker socket could read an
env value back via docker inspect. Each secret-bearing env var becomes a
tmpfs file plus a <NAME>_FILE pointer the tong reads; plain env keeps
flowing through -e. The resolved value only ever appears in the file plan,
never in the env the launcher passes as -e, and the launcher never writes
a secret to host disk. An env name that would escape the tmpfs dir as a
path component is refused, and a declared env var that collides with a
secret's <NAME>_FILE pointer fails the launch (and validation) unless it
already names the generated path, so a tong is never started unable to
find its secret.

These are host-side helpers exercised only when a tong carries a secret
reference; the empty-discovery launch path is unchanged.
@CrypticSwarm CrypticSwarm merged commit ee7bcf2 into master Jun 17, 2026
1 check passed
@CrypticSwarm CrypticSwarm deleted the tongs-phase-6-secret-providers branch June 17, 2026 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant