Daytona persistent resumes do not refresh SCM auth

[ColeMurray]

Summary

Daytona sandboxes use persistent resume/recover rather than Modal-style snapshot restore. On resume, we call Daytona startSandbox / recoverSandbox against the existing sandbox, so the sandbox keeps its previous filesystem, environment variables, git remotes, and runtime bootstrap state.

That means SCM auth is not refreshed on Daytona resume/recover.

Why this matters

Create-time SCM auth can expire while a Daytona sandbox is stopped, idle, or running for longer than the token TTL. Because resume/recover does not rebuild the environment or rewrite credentialed git remotes, later git operations can fail with stale credentials.

Modal snapshot restore is different: restore mints a fresh token and the restored entrypoint rewrites origin before fetching. This issue is specific to Daytona persistent resume/recover behavior.

Current behavior

• Fresh Daytona sandbox creation receives SCM auth setup.
• Daytona resume/recover only restarts or recovers the existing sandbox.
• No new clone token is resolved during resume.
• Existing env vars and credentialed remotes are preserved.

Expected behavior

On Daytona resume/recover, we should ensure SCM auth state is current before the agent performs git operations.

Possible fixes:

• Force a fresh Daytona sandbox when the previous create-time SCM auth may be expired.
• Add a resume-time auth refresh step that updates the sandbox's git auth state before git operations run.
• Track auth freshness/expiry in control-plane state and choose resume vs fresh spawn accordingly.
• Avoid relying on persisted env tokens/remotes for resumed Daytona sandboxes.

Impact

Users with Daytona persistent sandboxes can hit git auth failures after token expiry, especially after idle/resume or sandboxes running longer than the token TTL.

[ColeMurray]

Relevant code references from current main (90473af):

1. Daytona is explicitly modeled as persistent resume, not snapshot restore:

background-agents/packages/control-plane/src/sandbox/providers/daytona-provider.ts

Lines 52 to 57 in 90473af

	readonly capabilities: SandboxProviderCapabilities = {
	supportsSnapshots: false,
	supportsRestore: false,
	supportsWarm: false,
	supportsPersistentResume: true,
	supportsExplicitStop: true,

readonly capabilities: SandboxProviderCapabilities = {
  supportsSnapshots: false,
  supportsRestore: false,
  supportsWarm: false,
  supportsPersistentResume: true,
  supportsExplicitStop: true,
};

2. Fresh Daytona creation resolves SCM auth and injects it into the sandbox env:

background-agents/packages/control-plane/src/sandbox/providers/daytona-provider.ts

Lines 194 to 245 in 90473af

	private async buildEnvVars(config: CreateSandboxConfig): Promise<Record<string, string>> {
	const cloneToken = await this.getCloneToken();
	// Start with user env vars (repo secrets), then overlay system vars
	const envVars: Record<string, string> = { ...(config.userEnvVars ?? {}) };
	const sessionConfig: Record<string, string> = {
	session_id: config.sessionId,
	repo_owner: config.repoOwner,
	repo_name: config.repoName,
	provider: config.provider,
	model: config.model,
	};
	if (config.branch) {
	sessionConfig.branch = config.branch;
	}
	Object.assign(envVars, {
	PYTHONUNBUFFERED: "1",
	SANDBOX_ID: config.sandboxId,
	CONTROL_PLANE_URL: config.controlPlaneUrl,
	SANDBOX_AUTH_TOKEN: config.sandboxAuthToken,
	REPO_OWNER: config.repoOwner,
	REPO_NAME: config.repoName,
	SESSION_CONFIG: JSON.stringify(sessionConfig),
	});
	if (config.codeServerEnabled) {
	envVars.CODE_SERVER_PASSWORD = await this.deriveCodeServerPassword(config.sandboxId);
	}
	if (config.agentSlackNotifyEnabled) {
	envVars.AGENT_SLACK_NOTIFY_ENABLED = "true";
	}
	if (this.providerConfig.scmProvider === "gitlab") {
	envVars.VCS_HOST = "gitlab.com";
	envVars.VCS_CLONE_USERNAME = "oauth2";
	} else {
	envVars.VCS_HOST = "github.com";
	envVars.VCS_CLONE_USERNAME = "x-access-token";
	}
	if (cloneToken) {
	envVars.VCS_CLONE_TOKEN = cloneToken;
	if (this.providerConfig.scmProvider === "github") {
	envVars.GITHUB_APP_TOKEN = cloneToken;
	envVars.GITHUB_TOKEN = cloneToken;
	}
	}
	return envVars;

private async buildEnvVars(config: CreateSandboxConfig): Promise<Record<string, string>> {
  const cloneToken = await this.getCloneToken();
  // ...
  if (cloneToken) {
    envVars.VCS_CLONE_TOKEN = cloneToken;
    if (this.providerConfig.scmProvider === "github") {
      envVars.GITHUB_APP_TOKEN = cloneToken;
      envVars.GITHUB_TOKEN = cloneToken;
    }
  }

  return envVars;
}

3. The Daytona resume path only starts or recovers the existing sandbox; it does not call buildEnvVars() or resolve a new clone token:

background-agents/packages/control-plane/src/sandbox/providers/daytona-provider.ts

Lines 112 to 135 in 90473af

	async resumeSandbox(config: ResumeConfig): Promise<ResumeResult> {
	try {
	let sandbox;
	try {
	sandbox = await this.client.getSandbox(config.providerObjectId);
	} catch (error) {
	if (error instanceof DaytonaNotFoundError) {
	return {
	success: false,
	error: "Sandbox no longer exists in Daytona",
	shouldSpawnFresh: true,
	};
	}
	throw error;
	}
	const state = sandbox.state;
	if ((state === "error" || state === "build_failed") && sandbox.recoverable) {
	await this.client.recoverSandbox(config.providerObjectId);
	} else if (state !== "started") {
	// Covers stopped, archived, and non-recoverable error states —
	// Daytona's start endpoint handles the state transition internally.
	await this.client.startSandbox(config.providerObjectId);
	}

async resumeSandbox(config: ResumeConfig): Promise<ResumeResult> {
  // ...
  const state = sandbox.state;
  if ((state === "error" || state === "build_failed") && sandbox.recoverable) {
    await this.client.recoverSandbox(config.providerObjectId);
  } else if (state !== "started") {
    await this.client.startSandbox(config.providerObjectId);
  }
  // ...
}

4. The lifecycle manager only passes resume metadata into the provider; there is no repo/auth payload equivalent to fresh create:

background-agents/packages/control-plane/src/sandbox/lifecycle/manager.ts

Lines 731 to 738 in 90473af

	const result = await this.provider.resumeSandbox({
	providerObjectId,
	sessionId: session.session_name || session.id,
	sandboxId: sandbox.modal_sandbox_id,
	timeoutSeconds,
	codeServerEnabled: session.code_server_enabled === 1,
	sandboxSettings: this.parseSandboxSettings(session),
	});

const result = await this.provider.resumeSandbox({
  providerObjectId,
  sessionId: session.session_name || session.id,
  sandboxId: sandbox.modal_sandbox_id,
  timeoutSeconds,
  codeServerEnabled: session.code_server_enabled === 1,
  sandboxSettings: this.parseSandboxSettings(session),
});

5. The Daytona REST client start/recover calls do not include an env update body:

background-agents/packages/control-plane/src/sandbox/daytona-rest-client.ts

Lines 140 to 149 in 90473af

	async startSandbox(id: string): Promise<void> {
	await this.request<void>("POST", `/sandbox/${id}/start`, TIMEOUT_START_MS);
	}
	async stopSandbox(id: string): Promise<void> {
	await this.request<void>("POST", `/sandbox/${id}/stop`, TIMEOUT_STOP_MS);
	}
	async recoverSandbox(id: string): Promise<void> {
	await this.request<void>("POST", `/sandbox/${id}/recover`, TIMEOUT_RECOVER_MS);

async startSandbox(id: string): Promise<void> {
  await this.request<void>("POST", `/sandbox/${id}/start`, TIMEOUT_START_MS);
}

async recoverSandbox(id: string): Promise<void> {
  await this.request<void>("POST", `/sandbox/${id}/recover`, TIMEOUT_RECOVER_MS);
}

For contrast, Modal restore resolves a fresh token per restore request and passes it into the restore path:

background-agents/packages/modal-infra/src/web_api.py

Lines 467 to 481 in 90473af

	manager = SandboxManager()
	clone_token = _resolve_clone_token()
	code_server_enabled = bool(request.get("code_server_enabled", False))
	agent_slack_notify_enabled = bool(request.get("agent_slack_notify_enabled", False))
	sandbox_settings = request.get("sandbox_settings") or None
	# Restore sandbox from snapshot
	handle = await manager.restore_from_snapshot(
	snapshot_image_id=snapshot_image_id,
	session_config=session_config,
	sandbox_id=sandbox_id,
	control_plane_url=control_plane_url,
	sandbox_auth_token=sandbox_auth_token,
	clone_token=clone_token,

manager = SandboxManager()
clone_token = _resolve_clone_token()

handle = await manager.restore_from_snapshot(
    snapshot_image_id=snapshot_image_id,
    session_config=session_config,
    sandbox_id=sandbox_id,
    control_plane_url=control_plane_url,
    sandbox_auth_token=sandbox_auth_token,
    clone_token=clone_token,

[koala73]

Production-readiness review found this issue should remain a Daytona launch blocker until the live proof covers more than resume auth.

Additional code pointers from the review

• packages/daytona-infra/src/toolchain.py:66 installs OpenCode, code-server, agent-browser, and helpers, but does not install ttyd.
• packages/modal-infra/src/images/base.py:153 installs ttyd for the Modal runtime.
• packages/sandbox-runtime/src/sandbox_runtime/entrypoint.py:729 expects ttyd when terminal support is enabled.
• packages/control-plane/src/sandbox/providers/daytona-provider.ts:200 does not set TERMINAL_ENABLED.
• packages/control-plane/src/sandbox/providers/daytona-provider.ts:258 returns code-server tunnel data but no ttydUrl.

Why this matters

Even after SCM auth refresh is fixed, Daytona should not be considered production-ready unless the provider explicitly documents disabled terminal support or reaches parity with Modal terminal/tunnel behavior. Otherwise the provider can look compatible while terminal support silently cannot work.

Suggested acceptance criteria to fold into this issue or a follow-up

• Persistent Daytona resume refreshes SCM credentials without requiring sandbox rebuild.
• Live proof covers clone, setup/start, first prompt, push branch/PR, snapshot or persistent resume, and SCM credential refresh after resume.
• Terminal support is either explicitly disabled and hidden for Daytona, or ttyd is installed, tunneled, signed, returned as ttydUrl, and covered by live smoke.
• The Daytona production evidence bundle references the exact staging run and commit tested.

Related production-readiness issues filed from the same review: #768, #769, #770, #771, #772, #773, #774, #775.