Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 18 additions & 14 deletions .claude/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Cultivation Master mechanics without a new approved design update.

### Engine

- Unity 6.5 beta (currently `6000.5.0b9`) + URP. JOY explicitly chose beta over Unity 6.0 LTS for newest features; accept risk of breaking changes between beta builds and that some 3rd-party assets (Opsive UCC, Behavior Designer, Convai) may not be tested against this version yet. Re-evaluate if beta blocks progress.
- Unity 6.5 beta (currently `6000.5.0b9`) + URP. JOY explicitly chose beta over Unity 6.0 LTS for newest features; accept risk of breaking changes between beta builds and that some 3rd-party assets (Opsive UCC, Behavior Designer, Ida Faber character packs) may not be tested against this version yet. Re-evaluate if beta blocks progress.
- Force Text serialization (default Unity 6 - DO NOT change)

### Networking
Expand All @@ -92,7 +92,7 @@ Cultivation Master mechanics without a new approved design update.

- Opsive Ultimate Character Controller (ARPG character + combat)
- Behavior Designer (NPC combat behavior tree, NOT dialogue AI)
- Convai (NPC dialogue - phase 1 only, migrate to custom LLM phase 2)
- Ida Faber character packs (semi-real character bodies, Unity Humanoid retargeting, and ARKit 52 blendshape candidates)
- TextMeshPro (UI)
- Unity ML-Agents (deferred, future research)

Expand All @@ -108,20 +108,23 @@ Cultivation Master mechanics without a new approved design update.
- **`api.dos.ai` model service** (shared AI service; provider keys, model routing, prompt safety, voice token minting, AI-specific endpoints only)
- **Redis** (session, rate limit, transient cache)

### LLM
### LLM And NPC Voice

**Phase 1 (MVP):**

- Convai SDK in Unity for NPC dialogue
- Limit: no full custom LLM control, accept Convai cost
- Custom NPC dialogue through Nakama or the dedicated server to `api.dos.ai`.
- Text-first focused dialogue and ambient NPC speech.
- Unity client never embeds an LLM or voice vendor SDK for authoritative dialogue.
- Voice, TTS, and facial animation are optional presentation layers behind scoped server-minted sessions, not dialogue dependencies.

**Phase 2 (post-MVP):**
**Phase 2 (post-MVP / advanced presentation):**

- Migrate LLM calls to `api.dos.ai`, models:
- Route model calls inside `api.dos.ai`, models:
- Haiku 4.5 for NPC chat (fast, cheap)
- Sonnet 4.6 for boss / quest-critical dialog
- RAG memory: Supabase pgvector or Qdrant
- Voice: OpenAI Realtime API via ephemeral token (NOT API key in client) OR ElevenLabs
- Voice: custom `api.dos.ai` voice session lane using OpenAI Realtime, ElevenLabs, or self-hosted TTS via ephemeral token only (NOT API key in client)
- Facial animation: custom Ida Faber / ARKit-style blendshape profiles after verifying the imported Unity `SkinnedMeshRenderer` blendshape names
- Client AI: Unity Sentis for small perception (optional, phase 3)

### OpenClaw-Connected NPCs (CONCEPT)
Expand Down Expand Up @@ -187,7 +190,7 @@ Cultivation Master mechanics without a new approved design update.
- **Game server:** Linux headless Unity build on Hetzner VPS, Dockerized
- **Nakama backend:** self-hosted OSS first; Heroic Cloud only if operations become worth paying for
- **AI model service:** `api.dos.ai`
- **LLM API:** Convai phase 1, then Anthropic + OpenAI phase 2
- **LLM API:** `api.dos.ai` model service, with Anthropic / OpenAI / future providers hidden behind server-side routing
- **Monitoring:** Sentry (error) + Grafana (metrics)

### Testing
Expand Down Expand Up @@ -250,7 +253,7 @@ Cultivation Master mechanics without a new approved design update.
owner.
6. Only one agent may mutate Unity scenes, prefabs, package imports, or project settings at a time. Read-only inspection by the next agent is OK after the previous agent reports dirty files and current console state.
7. Every agent switch must leave a handoff using `docs/setup/agent-handoff.md`.
8. Unity package imports happen one package per commit, in this order unless JOY changes it: Opsive Ultimate Character Controller, Behavior Designer, Convai.
8. Unity package imports happen one package per commit unless JOY changes it. Current priority order: Opsive Ultimate Character Controller, Behavior Designer, Ida Faber character packs.
9. Before claiming Unity work is complete, check the Unity console and active scene through MCP or the Editor.
10. Significant commits still require independent reviewer pass per Hard Rule #7.

Expand Down Expand Up @@ -303,7 +306,8 @@ Cultivation Master mechanics without a new approved design update.
- Photon Fusion Dedicated Server overview
- Opsive Ultimate Character Controller getting started
- Behavior Designer manual
- Convai Unity SDK docs
- Ida Faber Unity retargeting docs
- Ida Faber blendshapes and facial animation docs
- Unity Multiplayer Play Mode tutorial
- Coplay unity-mcp + Claude Code setup guide
- `api.dos.ai` model-service contract patterns from JOY's existing DOS.AI stack
Expand Down Expand Up @@ -383,7 +387,7 @@ Scope:
- 1 small Town or Relay Yard with at least one useful facility or decor/care
interaction
- 1 dungeon instance
- 1 boss with LLM dialogue (Convai)
- 1 boss with custom LLM dialogue through `api.dos.ai`
- 1 questline (3-5 quests)
- Reinhabitation MVP (die -> spend test SECOND -> inhabit another eligible body
with selected reset)
Expand All @@ -407,7 +411,7 @@ OUT of scope for vertical slice:

1. **NEVER copy MetaDOS gameplay code.** Extract patterns only. Reference path: `D:\Projects\MetaDOS` (read-only).
2. **NEVER let LLM mutate authoritative game state.** Server validates all intent.
3. **NEVER put API keys (Anthropic, OpenAI, Convai, ElevenLabs) in Unity client.** All LLM calls go through Nakama or the dedicated server to `api.dos.ai`.
3. **NEVER put API keys (Anthropic, OpenAI, ElevenLabs, or any voice/model provider) in Unity client.** All LLM calls go through Nakama or the dedicated server to `api.dos.ai`.
4. **NEVER use Host Mode for production.** Server Mode dedicated only.
5. **NEVER add or replace backend / auth / social stack without an ADR and JOY approval.** Nakama OSS is the accepted game backend baseline per ADR 0010. Heroic Cloud, Hiro, Satori, OpenAuth, PlayFab, AccelByte, or a Supabase-first rollback require a new ADR.
6. **NEVER change Unity Asset Serialization away from Force Text.** Breaks LFS + diff.
Expand All @@ -422,7 +426,7 @@ OUT of scope for vertical slice:
- Advanced body or soul progression replacing the deferred concept
- Hunter NFT integration approach: Option 1 (preset hero) vs Hybrid 1+3 (modular pieces)
- Phase 2 LLM model split (when to use Haiku vs Sonnet)
- Voice NPC vendor (OpenAI Realtime vs ElevenLabs vs self-host)
- Voice NPC vendor behind `api.dos.ai` (OpenAI Realtime vs ElevenLabs vs self-host)
- Backend deployment path: self-hosted Nakama OSS vs Heroic Cloud later. Hiro / Satori require license and pricing review before adoption.
- Dedicated server hosting (Hetzner specs, region)
- Photon Fusion 2 license tier when scaling beyond Cloud free 20 CCU
32 changes: 18 additions & 14 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Cultivation Master mechanics without a new approved design update.

### Engine

- Unity 6.5 beta (currently `6000.5.0b9`) + URP. JOY explicitly chose beta over Unity 6.0 LTS for newest features; accept risk of breaking changes between beta builds and that some 3rd-party assets (Opsive UCC, Behavior Designer, Convai) may not be tested against this version yet. Re-evaluate if beta blocks progress.
- Unity 6.5 beta (currently `6000.5.0b9`) + URP. JOY explicitly chose beta over Unity 6.0 LTS for newest features; accept risk of breaking changes between beta builds and that some 3rd-party assets (Opsive UCC, Behavior Designer, Ida Faber character packs) may not be tested against this version yet. Re-evaluate if beta blocks progress.
- Force Text serialization (default Unity 6 - DO NOT change)

### Networking
Expand All @@ -92,7 +92,7 @@ Cultivation Master mechanics without a new approved design update.

- Opsive Ultimate Character Controller (ARPG character + combat)
- Behavior Designer (NPC combat behavior tree, NOT dialogue AI)
- Convai (NPC dialogue - phase 1 only, migrate to custom LLM phase 2)
- Ida Faber character packs (semi-real character bodies, Unity Humanoid retargeting, and ARKit 52 blendshape candidates)
- TextMeshPro (UI)
- Unity ML-Agents (deferred, future research)

Expand All @@ -108,20 +108,23 @@ Cultivation Master mechanics without a new approved design update.
- **`api.dos.ai` model service** (shared AI service; provider keys, model routing, prompt safety, voice token minting, AI-specific endpoints only)
- **Redis** (session, rate limit, transient cache)

### LLM
### LLM And NPC Voice

**Phase 1 (MVP):**

- Convai SDK in Unity for NPC dialogue
- Limit: no full custom LLM control, accept Convai cost
- Custom NPC dialogue through Nakama or the dedicated server to `api.dos.ai`.
- Text-first focused dialogue and ambient NPC speech.
- Unity client never embeds an LLM or voice vendor SDK for authoritative dialogue.
- Voice, TTS, and facial animation are optional presentation layers behind scoped server-minted sessions, not dialogue dependencies.

**Phase 2 (post-MVP):**
**Phase 2 (post-MVP / advanced presentation):**

- Migrate LLM calls to `api.dos.ai`, models:
- Route model calls inside `api.dos.ai`, models:
- Haiku 4.5 for NPC chat (fast, cheap)
- Sonnet 4.6 for boss / quest-critical dialog
- RAG memory: Supabase pgvector or Qdrant
- Voice: OpenAI Realtime API via ephemeral token (NOT API key in client) OR ElevenLabs
- Voice: custom `api.dos.ai` voice session lane using OpenAI Realtime, ElevenLabs, or self-hosted TTS via ephemeral token only (NOT API key in client)
- Facial animation: custom Ida Faber / ARKit-style blendshape profiles after verifying the imported Unity `SkinnedMeshRenderer` blendshape names
- Client AI: Unity Sentis for small perception (optional, phase 3)

### OpenClaw-Connected NPCs (CONCEPT)
Expand Down Expand Up @@ -187,7 +190,7 @@ Cultivation Master mechanics without a new approved design update.
- **Game server:** Linux headless Unity build on Hetzner VPS, Dockerized
- **Nakama backend:** self-hosted OSS first; Heroic Cloud only if operations become worth paying for
- **AI model service:** `api.dos.ai`
- **LLM API:** Convai phase 1, then Anthropic + OpenAI phase 2
- **LLM API:** `api.dos.ai` model service, with Anthropic / OpenAI / future providers hidden behind server-side routing
- **Monitoring:** Sentry (error) + Grafana (metrics)

### Testing
Expand Down Expand Up @@ -250,7 +253,7 @@ Cultivation Master mechanics without a new approved design update.
owner.
6. Only one agent may mutate Unity scenes, prefabs, package imports, or project settings at a time. Read-only inspection by the next agent is OK after the previous agent reports dirty files and current console state.
7. Every agent switch must leave a handoff using `docs/setup/agent-handoff.md`.
8. Unity package imports happen one package per commit, in this order unless JOY changes it: Opsive Ultimate Character Controller, Behavior Designer, Convai.
8. Unity package imports happen one package per commit unless JOY changes it. Current priority order: Opsive Ultimate Character Controller, Behavior Designer, Ida Faber character packs.
9. Before claiming Unity work is complete, check the Unity console and active scene through MCP or the Editor.
10. Significant commits still require independent reviewer pass per Hard Rule #7.

Expand Down Expand Up @@ -303,7 +306,8 @@ Cultivation Master mechanics without a new approved design update.
- Photon Fusion Dedicated Server overview
- Opsive Ultimate Character Controller getting started
- Behavior Designer manual
- Convai Unity SDK docs
- Ida Faber Unity retargeting docs
- Ida Faber blendshapes and facial animation docs
- Unity Multiplayer Play Mode tutorial
- Coplay unity-mcp + Claude Code setup guide
- `api.dos.ai` model-service contract patterns from JOY's existing DOS.AI stack
Expand Down Expand Up @@ -383,7 +387,7 @@ Scope:
- 1 small Town or Relay Yard with at least one useful facility or decor/care
interaction
- 1 dungeon instance
- 1 boss with LLM dialogue (Convai)
- 1 boss with custom LLM dialogue through `api.dos.ai`
- 1 questline (3-5 quests)
- Reinhabitation MVP (die -> spend test SECOND -> inhabit another eligible body
with selected reset)
Expand All @@ -407,7 +411,7 @@ OUT of scope for vertical slice:

1. **NEVER copy MetaDOS gameplay code.** Extract patterns only. Reference path: `D:\Projects\MetaDOS` (read-only).
2. **NEVER let LLM mutate authoritative game state.** Server validates all intent.
3. **NEVER put API keys (Anthropic, OpenAI, Convai, ElevenLabs) in Unity client.** All LLM calls go through Nakama or the dedicated server to `api.dos.ai`.
3. **NEVER put API keys (Anthropic, OpenAI, ElevenLabs, or any voice/model provider) in Unity client.** All LLM calls go through Nakama or the dedicated server to `api.dos.ai`.
4. **NEVER use Host Mode for production.** Server Mode dedicated only.
5. **NEVER add or replace backend / auth / social stack without an ADR and JOY approval.** Nakama OSS is the accepted game backend baseline per ADR 0010. Heroic Cloud, Hiro, Satori, OpenAuth, PlayFab, AccelByte, or a Supabase-first rollback require a new ADR.
6. **NEVER change Unity Asset Serialization away from Force Text.** Breaks LFS + diff.
Expand All @@ -422,7 +426,7 @@ OUT of scope for vertical slice:
- Advanced body or soul progression replacing the deferred concept
- Hunter NFT integration approach: Option 1 (preset hero) vs Hybrid 1+3 (modular pieces)
- Phase 2 LLM model split (when to use Haiku vs Sonnet)
- Voice NPC vendor (OpenAI Realtime vs ElevenLabs vs self-host)
- Voice NPC vendor behind `api.dos.ai` (OpenAI Realtime vs ElevenLabs vs self-host)
- Backend deployment path: self-hosted Nakama OSS vs Heroic Cloud later. Hiro / Satori require license and pricing review before adoption.
- Dedicated server hosting (Hetzner specs, region)
- Photon Fusion 2 license tier when scaling beyond Cloud free 20 CCU
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ namespace SecondSpawn.Settings
/// - Per-environment toggles (dev / staging / prod)
///
/// What does NOT live here (Hard Rule #3 in CLAUDE.md / AGENTS.md):
/// - Anthropic / OpenAI / Convai API keys
/// - Anthropic / OpenAI / TTS provider API keys
/// - Supabase service role key
/// - thirdweb secret key
/// - DOS Chain signing keys
Expand Down
12 changes: 7 additions & 5 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,8 @@ High-level architecture overview. For detailed component design see `docs/design
v v
+---------------+ +----------------------------+
| Nakama OSS | | api.dos.ai model service |
| - Game APIs | | - Convai (phase 1) |
| - Social | | - Anthropic + OpenAI (P2) |
| - Game APIs | | - Anthropic + OpenAI |
| - Social | | - Voice session minting |
| - Storage | | - RAG memory retrieval |
| - Postgres | | - AI rate limit + safety |
+---------------+ +-------------+--------------+
Expand Down Expand Up @@ -376,8 +376,10 @@ future TTS. It is disabled by default, mints only short-lived `api.dos.ai`
playback material when configured, and returns a text fallback reason when
voice is unavailable. The session response is presentation-only: it cannot
write memory, relationship, quest, TIME, SECOND, inventory, combat, or body
lifecycle state. Convai remains an isolated phase 1 spike lane for one boss or
hub NPC until cost, latency, reliability, and stable voice identity are proven.
lifecycle state. The facial-animation path is custom: Unity should map text,
audio amplitude, or future viseme data into character-specific profiles such as
Ida Faber / ARKit-style blendshape profiles only after the imported mesh names
are verified.

#### NPC Society Event Path

Expand Down Expand Up @@ -571,7 +573,7 @@ quest state directly.
### `api.dos.ai` Model Service

- All LLM calls go through here
- Multi-provider routing (Convai phase 1, Anthropic + OpenAI phase 2)
- Multi-provider routing for Anthropic, OpenAI, TTS, and future providers
- Experimental role-play providers such as Alibaba Qwen-Character are
evaluated only as replaceable adapters behind `api.dos.ai`; see
[design/52-llm-role-play-provider-evaluation.md](design/52-llm-role-play-provider-evaluation.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/adr/0003-llm-safety-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Game has LLM-driven NPCs and an LLM-driven AI agent that controls the player's c
## Decision

1. **All LLM calls go through Nakama or the dedicated server to `api.dos.ai`, never directly from Unity client.**
2. **API keys (Anthropic, OpenAI, Convai, ElevenLabs) live only in server env.** Never in client, never reachable from client.
2. **API keys (Anthropic, OpenAI, ElevenLabs, or any model/voice provider) live only in server env.** Never in client, never reachable from client.
3. **LLM output is parsed into structured intent (JSON schema enforced).** Free-form text is only for display.
4. **Server validates every intent before applying state changes.** LLM cannot grant items, gold, XP, or mutate progression directly.
5. **Rate limit per player + per NPC.** Token budget cap, request count cap.
Expand Down
4 changes: 2 additions & 2 deletions docs/adr/0005-unity-6-5-beta.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Two options were on the table:

- **Option A: Roll back to Unity 6.0 LTS.** Stable, supported, all
3rd-party assets (Opsive Ultimate Character Controller, Behavior
Designer, Convai, Photon Fusion 2) are tested against this version.
Designer, Ida Faber character packs, Photon Fusion 2) are tested against this version.
Predictable for a 3-6 month vertical slice timeline. Ecosystem
packages (URP 17.x, Input System) are stable on 6.0.
- **Option B: Stay on Unity 6.5 beta.** Newer features,
Expand Down Expand Up @@ -57,7 +57,7 @@ Per JOY's reasoning at decision time:

This is a mutable decision. Re-evaluate if any of the following:

- A 3rd-party asset (Opsive UCC, Behavior Designer, Convai, Photon
- A 3rd-party asset (Opsive UCC, Behavior Designer, Ida Faber character packs, Photon
Fusion 2) breaks on a beta build update
- A specific Unity 6.5 feature is the reason to stay (we should name it
here when that becomes true; today it is "newest features in general")
Expand Down
4 changes: 2 additions & 2 deletions docs/design/00-game-concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ Complete one Gate run or Town objective chain, update relationships and memories
| **Engine** | Unity 6.5 beta (currently `6000.5.0b9`) + URP. JOY chose beta for newest features. |
| **Networking** | Photon Fusion 2 (Server Mode dedicated for production; Host Mode + Photon Cloud free 20 CCU for dev) |
| **Persistence** | Nakama OSS + Postgres (profile, inventory, quest, NFT lock state, level/stats) |
| **LLM** | Convai phase 1 (NPC dialogue) -> `api.dos.ai` / api.dos.ai model service phase 2 (Haiku 4.5 for NPC chat, Sonnet 4.6 for boss / quest-critical NPCs). Server-side intent validation only. |
| **LLM** | Custom NPC dialogue through `api.dos.ai` / api.dos.ai model service (Haiku 4.5 for NPC chat, Sonnet 4.6 for boss / quest-critical NPCs). Server-side intent validation only. |
| **NFT** | DOS Chain via thirdweb-api MCP. Wallet auth, escrow contracts, skin / weapon / pet inventory if kept in scope. |
| **Art** | Anime-Ready Semi-Real with clean / controlled PBR as the target direction; prototype assets are scaffolding until look-dev proves the recipe |
| **Key technical risks** | LLM intent validation at scale; AI agent server tick load; NFT-Unity inventory sync latency |
Expand Down Expand Up @@ -197,7 +197,7 @@ See [docs/ARCHITECTURE.md](../ARCHITECTURE.md) for system diagram + critical inv

### Scope Risks
- Solo dev + AI agent (Claude Code) + 3-6 month vertical slice on novel architecture is tight
- 3rd-party assets (Opsive UCC, Behavior Designer, Convai) may not be tested against Unity 6.5 beta. Opsive is not mandatory for the first movement prototype until it proves value against the minimal controller baseline.
- 3rd-party assets (Opsive UCC, Behavior Designer, Ida Faber character packs) may not be tested against Unity 6.5 beta. Opsive is not mandatory for the first movement prototype until it proves value against the minimal controller baseline.

### Open Questions (need JOY input later)
- SECOND economy: reinhabitation cost, source, sink. (Open Decision Point in CLAUDE.md.)
Expand Down
Loading
Loading