Skip to content

Commit 940898c

Browse files
TabishBTabishB
authored
Add optional anonymous usage statistics (Fission-AI#468)
* chore: add proposal for PostHog analytics integration Introduces the proposal artifact for adding opt-in telemetry to OpenSpec using PostHog. Covers command tracking, feature adoption metrics, and privacy-respecting consent management. * feat: add optional anonymous usage statistics Introduces privacy-first usage analytics to help understand how OpenSpec is being used. Key privacy protections: - Only tracks command names and version (no arguments, paths, or content) - Opt-out via OPENSPEC_TELEMETRY=0 or DO_NOT_TRACK=1 - Auto-disabled in CI environments - No IP address collection (explicitly disabled) - Anonymous ID is a random UUID with no PII Uses PostHog with a reverse proxy to avoid ad blockers. First-run shows a one-line notice informing users about the collection. * docs: make telemetry section collapsible and concise
1 parent d49a88c commit 940898c

16 files changed

Lines changed: 1164 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -410,6 +410,15 @@ You can always go back:
410410

411411
</details>
412412

413+
<details>
414+
<summary><strong>Telemetry</strong> – OpenSpec collects anonymous usage stats (opt-out: <code>OPENSPEC_TELEMETRY=0</code>)</summary>
415+
416+
We collect only command names and version to understand usage patterns. No arguments, paths, content, or PII. Automatically disabled in CI.
417+
418+
**Opt-out:** `export OPENSPEC_TELEMETRY=0` or `export DO_NOT_TRACK=1`
419+
420+
</details>
421+
413422
## Contributing
414423

415424
- Install dependencies: `pnpm install`

‎openspec/changes/archive/2026-01-09-add-posthog-analytics/.openspec.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
schema: spec-driven
2+
created: 2026-01-10

‎openspec/changes/archive/2026-01-09-add-posthog-analytics/design.md‎

Lines changed: 175 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,175 @@
1+
## Context
2+
3+
OpenSpec needs usage analytics to understand adoption and inform product decisions. PostHog provides a privacy-conscious analytics platform suitable for open source projects.
4+
5+
## Goals / Non-Goals
6+
7+
**Goals:**
8+
- Track daily/weekly/monthly active usage
9+
- Understand command usage patterns
10+
- Keep implementation minimal and privacy-respecting
11+
- Enable opt-out with minimal friction
12+
13+
**Non-Goals:**
14+
- Detailed error tracking or diagnostics
15+
- User identification or profiling
16+
- Complex event hierarchies
17+
- Full CLI command for telemetry management (env var sufficient for now)
18+
19+
## Decisions
20+
21+
### Opt-Out Model
22+
23+
**Decision:** Telemetry enabled by default, opt-out via environment variable.
24+
25+
```bash
26+
OPENSPEC_TELEMETRY=0 # Disable telemetry
27+
DO_NOT_TRACK=1 # Industry standard, also respected
28+
```
29+
30+
Auto-disabled when `CI=true` is detected.
31+
32+
**Rationale:**
33+
- Opt-in typically yields ~3% participation—not enough for meaningful data
34+
- Understanding usage patterns requires statistically significant sample sizes
35+
- Environment variable opt-out is simple and immediate
36+
- Respecting `DO_NOT_TRACK` follows industry convention
37+
38+
**Alternatives considered:**
39+
- Opt-in only - Insufficient data for product decisions
40+
- Config file setting - More complex, env var sufficient for MVP
41+
- Full `openspec telemetry` command - Can add later if users request
42+
43+
### Event Design
44+
45+
**Decision:** Single event type with minimal properties.
46+
47+
```typescript
48+
{
49+
event: 'command_executed',
50+
properties: {
51+
command: 'init', // Command name only
52+
version: '1.2.3' // OpenSpec version
53+
}
54+
}
55+
```
56+
57+
**Rationale:**
58+
- Answers the core questions: how much usage, which commands are popular
59+
- PostHog derives DAU/WAU/MAU from anonymous user counts over time
60+
- No arguments, paths, or content—clean privacy story
61+
- Easy to explain in disclosure notice
62+
63+
**Not tracked:**
64+
- Command arguments
65+
- File paths or contents
66+
- Error messages or stack traces
67+
- Project names or spec content
68+
- IP addresses (`$ip: null` explicitly set)
69+
70+
### Anonymous ID
71+
72+
**Decision:** Random UUID, lazily generated on first telemetry send, stored in global config.
73+
74+
```typescript
75+
// ~/.config/openspec/config.json
76+
{
77+
"telemetry": {
78+
"anonymousId": "f47ac10b-58cc-4372-a567-0e02b2c3d479"
79+
}
80+
}
81+
```
82+
83+
**Rationale:**
84+
- Random UUID has no relation to the person—can't be reversed
85+
- Stored in config so same user = same ID across sessions (needed for DAU/WAU/MAU)
86+
- Lazy generation means no ID created if user opts out before first command
87+
- User can delete config to reset identity
88+
89+
**Alternatives considered:**
90+
- Machine-derived hash (hostname, MAC) - Feels invasive, fingerprint-like
91+
- Per-session UUID - Breaks user counting metrics entirely
92+
93+
### SDK Configuration
94+
95+
**Decision:** PostHog Node SDK with immediate flush, shutdown on exit.
96+
97+
```typescript
98+
const posthog = new PostHog(API_KEY, {
99+
flushAt: 1, // Send immediately, don't batch
100+
flushInterval: 0 // No timer-based flushing
101+
});
102+
103+
// Before CLI exits
104+
await posthog.shutdown();
105+
```
106+
107+
**Rationale:**
108+
- CLI processes are short-lived; batching would lose events
109+
- `flushAt: 1` ensures each event sends immediately
110+
- `shutdown()` guarantees flush before process exit
111+
- Adds ~100-300ms to exit—negligible for typical CLI workflows
112+
113+
**Error handling:**
114+
- Network failures silently ignored (telemetry shouldn't break CLI)
115+
- `shutdown()` wrapped in try/catch
116+
117+
### Hook Location
118+
119+
**Decision:** Commander.js `preAction` and `postAction` hooks.
120+
121+
```typescript
122+
program
123+
.hook('preAction', (thisCommand) => {
124+
maybeShowTelemetryNotice();
125+
trackCommand(thisCommand.name(), VERSION);
126+
})
127+
.hook('postAction', async () => {
128+
await shutdown();
129+
});
130+
```
131+
132+
**Rationale:**
133+
- Centralized—one place for all telemetry logic
134+
- Automatic—new commands get tracked without code changes
135+
- Clean separation—command handlers don't know about telemetry
136+
137+
**Subcommand handling:**
138+
- Track full command path for nested commands (e.g., `change:apply`)
139+
140+
### First-Run Notice
141+
142+
**Decision:** One-liner on first command ever, stored "seen" flag in config.
143+
144+
```
145+
Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0
146+
```
147+
148+
**Rationale:**
149+
- First command (not just `init`) ensures notice is always seen
150+
- Non-blocking—no prompt, just informational
151+
- One-liner is visible but not intrusive
152+
- Storing "seen" in config prevents repeated display
153+
154+
**Config after first run:**
155+
```json
156+
{
157+
"telemetry": {
158+
"anonymousId": "...",
159+
"noticeSeen": true
160+
}
161+
}
162+
```
163+
164+
## Risks / Trade-offs
165+
166+
| Risk | Mitigation |
167+
|------|------------|
168+
| Users prefer opt-in | Clear disclosure, trivial opt-out, transparent about what's collected |
169+
| GDPR concerns | No personal data, no IP, user can delete config |
170+
| Slows CLI exit by ~200ms | Negligible for most workflows; can optimize if needed |
171+
| PostHog outage affects CLI | Fire-and-forget with timeout; failures are silent |
172+
173+
## Open Questions
174+
175+
None—design is intentionally minimal. Future enhancements (dedicated command, workflow tracking) can be added based on user feedback.

‎openspec/changes/archive/2026-01-09-add-posthog-analytics/proposal.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
## Why
2+
3+
OpenSpec currently has no visibility into how the tool is being used. Without analytics, we cannot:
4+
- Understand which commands and features are most valuable to users
5+
- Measure adoption and usage patterns
6+
- Make data-driven decisions about product development
7+
8+
Adding PostHog analytics enables product insights while respecting user privacy through transparent, opt-out telemetry.
9+
10+
## What Changes
11+
12+
- Add PostHog Node.js SDK as a dependency
13+
- Implement telemetry system with environment variable opt-out
14+
- Track command usage (command name and version only)
15+
- Show first-run notice informing users about telemetry
16+
- Store anonymous ID in global config (`~/.config/openspec/config.json`)
17+
- Respect `DO_NOT_TRACK` and `OPENSPEC_TELEMETRY=0` environment variables
18+
- Auto-disable in CI environments
19+
20+
## Capabilities
21+
22+
### New Capabilities
23+
24+
- `telemetry`: Anonymous usage analytics using PostHog. Covers command tracking, opt-out controls, and first-run disclosure notice.
25+
26+
### Modified Capabilities
27+
28+
- `global-config`: Add telemetry state storage (anonymous ID, notice seen flag)
29+
30+
## Impact
31+
32+
- **Dependencies**: Add `posthog-node` package
33+
- **Privacy**: Opt-out via env var, no personal data collected, clear disclosure
34+
- **Configuration**: New global config fields for telemetry state
35+
- **Network**: Async event sending with flush on exit (~100-300ms added)
36+
- **CI/CD**: Telemetry auto-disabled when `CI=true`
37+
- **Documentation**: Update README with telemetry disclosure

‎openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/global-config/spec.md‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
## MODIFIED Requirements
2+
3+
### Requirement: Global configuration storage
4+
The system SHALL store global configuration in `~/.config/openspec/config.json`, including telemetry state with `anonymousId` and `noticeSeen` fields.
5+
6+
#### Scenario: Initial config creation
7+
- **WHEN** no global config file exists
8+
- **AND** the first telemetry event is about to be sent
9+
- **THEN** the system creates `~/.config/openspec/config.json` with telemetry configuration
10+
11+
#### Scenario: Telemetry config structure
12+
- **WHEN** reading or writing telemetry configuration
13+
- **THEN** the config contains a `telemetry` object with `anonymousId` (string UUID) and `noticeSeen` (boolean) fields
14+
15+
#### Scenario: Config file format
16+
- **WHEN** storing configuration
17+
- **THEN** the system writes valid JSON that can be read and modified by users
18+
19+
#### Scenario: Existing config preservation
20+
- **WHEN** adding telemetry fields to an existing config file
21+
- **THEN** the system preserves all existing configuration fields

‎openspec/changes/archive/2026-01-09-add-posthog-analytics/specs/telemetry/spec.md‎

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
## ADDED Requirements
2+
3+
### Requirement: Command execution tracking
4+
The system SHALL send a `command_executed` event to PostHog when any CLI command executes, including only the command name and OpenSpec version as properties.
5+
6+
#### Scenario: Standard command execution
7+
- **WHEN** a user runs any openspec command
8+
- **THEN** the system sends a `command_executed` event with `command` and `version` properties
9+
10+
#### Scenario: Subcommand execution
11+
- **WHEN** a user runs a nested command like `openspec change apply`
12+
- **THEN** the system sends a `command_executed` event with the full command path (e.g., `change:apply`)
13+
14+
### Requirement: Privacy-preserving event design
15+
The system SHALL NOT include command arguments, file paths, project names, spec content, error messages, or IP addresses in telemetry events.
16+
17+
#### Scenario: Command with arguments
18+
- **WHEN** a user runs `openspec init my-project --force`
19+
- **THEN** the telemetry event contains only `command: "init"` and `version: "<version>"` without arguments
20+
21+
#### Scenario: IP address exclusion
22+
- **WHEN** the system sends a telemetry event
23+
- **THEN** the event explicitly sets `$ip: null` to prevent IP tracking
24+
25+
### Requirement: Environment variable opt-out
26+
The system SHALL disable telemetry when `OPENSPEC_TELEMETRY=0` or `DO_NOT_TRACK=1` environment variables are set.
27+
28+
#### Scenario: OPENSPEC_TELEMETRY opt-out
29+
- **WHEN** `OPENSPEC_TELEMETRY=0` is set in the environment
30+
- **THEN** the system sends no telemetry events
31+
32+
#### Scenario: DO_NOT_TRACK opt-out
33+
- **WHEN** `DO_NOT_TRACK=1` is set in the environment
34+
- **THEN** the system sends no telemetry events
35+
36+
#### Scenario: Environment variable takes precedence
37+
- **WHEN** the user has previously used the CLI (config exists)
38+
- **AND** the user sets `OPENSPEC_TELEMETRY=0`
39+
- **THEN** telemetry is disabled regardless of config state
40+
41+
### Requirement: CI environment auto-disable
42+
The system SHALL automatically disable telemetry when `CI=true` environment variable is detected.
43+
44+
#### Scenario: CI environment detection
45+
- **WHEN** `CI=true` is set in the environment
46+
- **THEN** the system sends no telemetry events
47+
48+
#### Scenario: CI with explicit enable
49+
- **WHEN** `CI=true` is set
50+
- **AND** `OPENSPEC_TELEMETRY=1` is explicitly set
51+
- **THEN** telemetry remains disabled (CI takes precedence for privacy)
52+
53+
### Requirement: First-run telemetry notice
54+
The system SHALL display a one-line telemetry disclosure notice on the first command execution, before any telemetry is sent.
55+
56+
#### Scenario: First command execution
57+
- **WHEN** a user runs their first openspec command
58+
- **AND** telemetry is enabled
59+
- **THEN** the system displays: "Note: OpenSpec collects anonymous usage stats. Opt out: OPENSPEC_TELEMETRY=0"
60+
61+
#### Scenario: Subsequent command execution
62+
- **WHEN** a user has already seen the notice (noticeSeen: true in config)
63+
- **THEN** the system does not display the notice
64+
65+
#### Scenario: Notice before telemetry
66+
- **WHEN** displaying the first-run notice
67+
- **THEN** the notice appears before any telemetry event is sent
68+
69+
### Requirement: Anonymous user identification
70+
The system SHALL generate a random UUID as an anonymous identifier on first telemetry send, stored in global config.
71+
72+
#### Scenario: First telemetry event
73+
- **WHEN** the first telemetry event is sent
74+
- **AND** no anonymousId exists in config
75+
- **THEN** the system generates a random UUID v4 and stores it in config
76+
77+
#### Scenario: Persistent identity
78+
- **WHEN** a user runs multiple commands across sessions
79+
- **THEN** the same anonymousId is used for all events
80+
81+
#### Scenario: Lazy generation with opt-out
82+
- **WHEN** a user opts out before running any command
83+
- **THEN** no anonymousId is ever generated or stored
84+
85+
### Requirement: Immediate event sending
86+
The system SHALL send telemetry events immediately without batching, using `flushAt: 1` and `flushInterval: 0` configuration.
87+
88+
#### Scenario: Event transmission timing
89+
- **WHEN** a command executes
90+
- **THEN** the telemetry event is sent immediately, not queued for batch transmission
91+
92+
### Requirement: Graceful shutdown
93+
The system SHALL call `posthog.shutdown()` before CLI exit to ensure pending events are flushed.
94+
95+
#### Scenario: Normal exit
96+
- **WHEN** a command completes successfully
97+
- **THEN** the system awaits `shutdown()` before exiting
98+
99+
#### Scenario: Error exit
100+
- **WHEN** a command fails with an error
101+
- **THEN** the system still awaits `shutdown()` before exiting
102+
103+
### Requirement: Silent failure handling
104+
The system SHALL silently ignore telemetry failures without affecting CLI functionality.
105+
106+
#### Scenario: Network failure
107+
- **WHEN** the telemetry request fails due to network error
108+
- **THEN** the CLI command completes normally without error message
109+
110+
#### Scenario: PostHog outage
111+
- **WHEN** PostHog service is unavailable
112+
- **THEN** the CLI command completes normally without error message
113+
114+
#### Scenario: Shutdown failure
115+
- **WHEN** `shutdown()` fails or times out
116+
- **THEN** the CLI exits normally without error message

0 commit comments

Comments
 (0)