|
| 1 | +# Configurable Models Implementation Plan |
| 2 | + |
| 3 | +## Overview |
| 4 | +Add user-configurable model selection to ComputerUseAgent, allowing users to choose between different Claude models via settings. |
| 5 | + |
| 6 | +## Supported Models |
| 7 | +- `claude-3-5-sonnet-20241022` (3.5 Sonnet) - Default |
| 8 | +- `claude-3-7-sonnet-20250219` (3.7 Sonnet) |
| 9 | +- `claude-sonnet-4-20250514` (4 Sonnet) |
| 10 | +- `claude-opus-4-20250514` (4 Opus) |
| 11 | + |
| 12 | +## Implementation Steps |
| 13 | + |
| 14 | +### 1. Update UserSettings Interface |
| 15 | +- Add `model?: string` field to `UserSettings` interface in `src/types/interfaces.ts` |
| 16 | +- Set default model to current model (`claude-3-5-sonnet-20241022`) |
| 17 | + |
| 18 | +### 2. Update Settings Configuration |
| 19 | +- Modify `DEFAULT_SETTINGS` in `src/config/settings.ts` to include default model |
| 20 | +- Add helper function `getSelectedModel()` to retrieve user's model preference |
| 21 | +- Add model validation function to ensure only supported models are allowed |
| 22 | + |
| 23 | +### 3. Update Constants Configuration |
| 24 | +- Modify `API_CONFIG` in `src/config/constants.ts` to use configurable model |
| 25 | +- Replace hardcoded `MODEL` with dynamic model selection |
| 26 | +- **Smart Planner Model Selection**: Configure `REASONING_MODEL` based on main model: |
| 27 | + - If main model is 3.5 Sonnet → Use 3.7 Sonnet for planning BUT exclude thinking budget |
| 28 | + - If main model is 3.7 Sonnet → Use 3.7 Sonnet for planning with thinking capabilities |
| 29 | + - If main model is 4 Sonnet/Opus → Use same model for planning with full thinking capabilities |
| 30 | +- Keep `INTENT_MODEL` (Haiku) separate as it serves specific purposes |
| 31 | + |
| 32 | +### 4. Update Settings Command |
| 33 | +- Add `--set-model` flag to settings command in `src/commands/settings.ts` |
| 34 | +- Add `--list-models` flag to show available models |
| 35 | +- Include model validation when setting new model |
| 36 | +- Update help text to include model configuration options |
| 37 | + |
| 38 | +### 5. Update Session Classes |
| 39 | +- Modify classes that use `API_CONFIG.MODEL` to dynamically get model from settings |
| 40 | +- Ensure all Claude API calls use the configured model |
| 41 | +- **Implement Thinking Budget Control**: Modify planner to conditionally include thinking based on model |
| 42 | +- **Update cost tracking calls**: Add model name parameter to updateTokenUsage() calls |
| 43 | +- Key files to update: |
| 44 | + - `src/modules/hybrid/hybrid_session.ts` - Update token tracking with model names |
| 45 | + - `src/modules/planner/planner.ts` - Add logic to exclude thinking budget for 3.5 Sonnet, track planning model costs |
| 46 | + - Any other files that directly reference `API_CONFIG.MODEL` |
| 47 | + |
| 48 | +### 6. Update Cost Tracking Infrastructure |
| 49 | +- **SessionLogger class**: Add model-specific token tracking |
| 50 | +- **Database schema**: Keep existing cost field for backward compatibility |
| 51 | +- **Cost calculation**: Create model pricing lookup function |
| 52 | +- **API call sites**: Pass model name to updateTokenUsage() throughout codebase |
| 53 | + |
| 54 | +## Model Mapping |
| 55 | +Create a mapping between user-friendly names and actual model identifiers: |
| 56 | +```typescript |
| 57 | +const MODEL_MAP = { |
| 58 | + "3.5-sonnet": "claude-3-5-sonnet-20241022", |
| 59 | + "3.7-sonnet": "claude-3-7-sonnet-20250219", |
| 60 | + "4-sonnet": "claude-sonnet-4-20250514", |
| 61 | + "4-opus": "claude-opus-4-20250514" |
| 62 | +} |
| 63 | +``` |
| 64 | + |
| 65 | +## Usage Examples |
| 66 | +```bash |
| 67 | +# Set model to 4 Sonnet |
| 68 | +deno run -A src/main.ts settings --set-model "4-sonnet" |
| 69 | + |
| 70 | +# List available models |
| 71 | +deno run -A src/main.ts settings --list-models |
| 72 | + |
| 73 | +# View current settings including model |
| 74 | +deno run -A src/main.ts settings --list |
| 75 | +``` |
| 76 | + |
| 77 | +## Cost Tracking and Reporting |
| 78 | + |
| 79 | +### Model Pricing (as of May 2025) |
| 80 | +Based on current Anthropic pricing: |
| 81 | + |
| 82 | +| Model | Input Price (per 1M tokens) | Output Price (per 1M tokens) | With Caching Input (90% off) | With Caching Output (90% off) | |
| 83 | +|-------|----------------------------|------------------------------|------------------------------|-------------------------------| |
| 84 | +| 3.5 Sonnet | $3.00 | $15.00 | $0.30 | $1.50 | |
| 85 | +| 3.7 Sonnet | $3.00 | $15.00 | $0.30 | $1.50 | |
| 86 | +| **4 Sonnet** | **$3.00** | **$15.00** | **$0.30** | **$1.50** | |
| 87 | +| **4 Opus** | **$15.00** | **$75.00** | **$1.50** | **$7.50** | |
| 88 | + |
| 89 | +### Current Cost Implementation Analysis |
| 90 | +**Existing Cost Tracking (SessionLogger in src/utils/session.ts):** |
| 91 | +1. **Hard-coded pricing**: Uses fixed $3/$15 per million tokens (3.5/3.7 Sonnet pricing) |
| 92 | +2. **Single model tracking**: Only tracks one set of tokens per session |
| 93 | +3. **Simple aggregation**: Combines all token usage regardless of which model was used |
| 94 | +4. **Database storage**: Saves total cost per session to SQLite database |
| 95 | + |
| 96 | +**Current Limitations:** |
| 97 | +1. **No model-specific costs**: All API calls treated as same model pricing |
| 98 | +2. **Planning model invisible**: REASONING_MODEL usage not tracked separately |
| 99 | +3. **Fixed pricing**: Doesn't account for different model costs (4 Opus is 5x more expensive) |
| 100 | +4. **No per-model breakdown**: Can't see which model consumed which tokens |
| 101 | + |
| 102 | +### Enhanced Cost Tracking Implementation |
| 103 | +**Update SessionLogger class (src/utils/session.ts):** |
| 104 | +1. **Dynamic pricing**: Replace hard-coded costs with model-specific pricing lookup |
| 105 | +2. **Multi-model tracking**: Track tokens per model type (main vs planning) |
| 106 | +3. **Cost breakdown**: Maintain separate cost calculations for each model |
| 107 | +4. **Backward compatibility**: Keep existing getTotalCost() for database storage |
| 108 | + |
| 109 | +**Changes needed in SessionLogger:** |
| 110 | +```typescript |
| 111 | +class SessionLogger { |
| 112 | + private modelTokenUsage = new Map<string, {input: number, output: number}>() |
| 113 | + |
| 114 | + updateTokenUsage(inputTokens: number, outputTokens: number, modelName: string): void |
| 115 | + getModelCosts(): Map<string, number> |
| 116 | + getTotalCost(): number // Updated to sum all model costs |
| 117 | +} |
| 118 | +``` |
| 119 | + |
| 120 | +**Update cost calculation methods:** |
| 121 | +- Replace hard-coded $3/$15 with dynamic pricing based on model |
| 122 | +- Add model parameter to updateTokenUsage() calls |
| 123 | +- Modify getTotalCost() to sum costs across all models used |
| 124 | + |
| 125 | +## Validation Rules |
| 126 | +- Only allow predefined model names |
| 127 | +- Provide clear error messages for invalid models |
| 128 | +- Fall back to default model if configured model becomes invalid |
| 129 | +- Warn users about cost implications when selecting expensive models |
| 130 | + |
| 131 | +## Backward Compatibility |
| 132 | +- Existing users without model setting will use default (3.5 Sonnet) |
| 133 | +- No breaking changes to existing functionality |
| 134 | +- Settings file will be automatically updated with default model on first access |
| 135 | +- Cost tracking will start from implementation date forward |
0 commit comments