Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions Unity/Assets/_SecondSpawn/Scripts/AI/AgentContextDto.cs
Original file line number Diff line number Diff line change
Expand Up @@ -1404,5 +1404,45 @@ public sealed class VoiceSessionDto
public string provider;
public bool requires_ephemeral_token;
public string reason;
public string actor_id;
public string body_id;
public string conversation_session_id;
public VoiceProfileDto voice_profile;
public VoiceSessionMaterialDto session;
public VoiceSessionDebugDto debug;
}

[Serializable]
public sealed class VoiceSessionRequestDto
{
public string actor_id;
public string conversation_session_id;
public string line_id;
public string playback_mode = "tts";
public int ttl_seconds = 90;
public string[] lip_sync_tiers;
}

[Serializable]
public sealed class VoiceSessionMaterialDto
{
public string session_id;
public long expires_at_ms;
public int ttl_seconds;
public string audience;
public string transport;
public string endpoint;
public string ephemeral_token;
public string[] lip_sync_tiers;
public bool presentation_only;
public string authority_note;
}

[Serializable]
public sealed class VoiceSessionDebugDto
{
public string source;
public string provider_status;
public string fallback_mode;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ namespace SecondSpawn.AI
public sealed class PrototypeNPCChatClient : MonoBehaviour
{
[SerializeField] private bool _enablePrototypeHotkeys;
[SerializeField] private string _npcId = "prototype-guide";
[SerializeField] private string _npcId = "npc-synthetic-sentinel-0101";
[SerializeField, TextArea] private string _prototypeMessage =
"What should this body remember while I am offline?";
[SerializeField] private Key _talkKey = Key.O;
Expand Down Expand Up @@ -63,7 +63,12 @@ private IEnumerator SendPrototypeChat()

private IEnumerator CheckVoiceSession()
{
yield return _gateway.GetVoiceSession(response =>
yield return _gateway.GetVoiceSession(new VoiceSessionRequestDto
{
actor_id = _npcId,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Fallback invalid NPC id before requesting voice session

The voice-session request now sends actor_id = _npcId directly, but existing serialized scene data still uses legacy values like prototype-guide (for example Unity/Assets/_SecondSpawn/Scenes/ZoneTest_Hub.unity). The new Nakama RPC rejects non-permanent NPC ids, so enabling prototype hotkeys in that scene and pressing the voice key now fails with an RPC error instead of returning the structured fallback response.

Useful? React with 👍 / 👎.

playback_mode = "voice_preview",
lip_sync_tiers = new[] { "text_timed", "audio_amplitude_hook" }
}, response =>
{
Debug.Log($"[PrototypeNPCChatClient] Voice provider={response.provider}, available={response.voice_available}, reason={response.reason}");
}, Debug.LogWarning);
Expand Down
14 changes: 6 additions & 8 deletions Unity/Assets/_SecondSpawn/Scripts/AI/SecondSpawnGatewayClient.cs
Original file line number Diff line number Diff line change
Expand Up @@ -430,14 +430,12 @@ public IEnumerator Chat(NpcChatRequestDto request, Action<NpcChatResponseDto> on

public IEnumerator GetVoiceSession(Action<VoiceSessionDto> onSuccess, Action<string> onError = null)
{
yield return null;
onSuccess?.Invoke(new VoiceSessionDto
{
voice_available = false,
provider = "not_configured",
requires_ephemeral_token = true,
reason = "Voice sessions require a future Nakama RPC that mints an api.dos.ai ephemeral token."
});
yield return GetVoiceSession(new VoiceSessionRequestDto(), onSuccess, onError);
}

public IEnumerator GetVoiceSession(VoiceSessionRequestDto request, Action<VoiceSessionDto> onSuccess, Action<string> onError = null)
{
yield return SendNakamaRpc("secondspawn_voice_session_request", request ?? new VoiceSessionRequestDto(), onSuccess, onError);
}

private IEnumerator SendNakamaRpc<TResponse>(
Expand Down
20 changes: 20 additions & 0 deletions backend/nakama/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,23 @@ circuit-open state and tries DOS.AI again. If the model still fails, the NPC
does not invent a canned reply. It returns a degraded `stop` decision with the
model failure reason so Unity can show `AI BACKOFF` or `AI FALLBACK` honestly.

Optional scoped voice session env:

```text
DOS_AI_VOICE_SESSIONS_ENABLED=false
DOS_AI_VOICE_SESSION_URL=https://api.dos.ai/v1/voice/sessions
DOS_AI_VOICE_SESSION_TTL_SECONDS=90
```

`secondspawn_voice_session_request` is disabled by default. When enabled,
Nakama uses `DOS_AI_API_KEY` server-side to ask `api.dos.ai` for short-lived
voice playback material, then returns only a scoped session descriptor and an
ephemeral token to Unity. The response is presentation-only and cannot mutate
memory, relationships, quests, TIME, SECOND, inventory, combat, or body
lifecycle state. When voice is disabled, unconfigured, timed out, or rejected,
the RPC returns a structured text-only fallback so focused dialogue remains
usable.

### Metrics and Structured Logs

Nakama exposes its normal Prometheus-style server metrics through deployment
Expand All @@ -155,6 +172,9 @@ can filter prototype game events without parsing free-form text:
duplicate claims.
- `secondspawn.body_time_mutation` records accepted BodyTime changes and
duplicate BodyTime events.
- `secondspawn.voice_session` records voice session availability, provider,
fallback reason, target actor, and voice profile id without logging provider
keys or ephemeral tokens.

Structured logs must stay public-safe: do not log provider API keys, RPC
secrets, raw prompts, raw payloads, or private provider responses. Use
Expand Down
Loading
Loading