prflow — PR comment trend analyser

prflow pulls GitHub pull-request review feedback into a local SQLite database, makes it full-text searchable, and uses an LLM to surface recurring themes in the feedback a given engineer receives. It's a coaching aid: instead of scrolling through dozens of PRs to work out "what kind of comments does Alice keep getting?", you ingest once and ask.

It ingests two kinds of feedback per repo:

• Inline review comments (comments left on a diff line)
• Review summaries (the body of an approve / request-changes / comment review)

General PR conversation comments are out of scope.

---

Prerequisites

• Go 1.25+ (go version) — only needed to build the binary.
• A GitHub token for ingest (see Step 2). Unauthenticated requests are limited to 60/hr and cannot see private repos (GitHub returns 404 Not Found, not 403, for private repos when unauthenticated).
• An LLM provider login for analyse (see Step 4). Not needed for ingest/search/users.

---

Step 1 — Build

go build -o prflow ./cmd/prflow

This produces a single static binary ./prflow in the project root. (A pre-built prflow binary may already be present; rebuild if you've changed the source.)

Verify it runs:

./prflow --help

---

Step 2 — Create a read-only GitHub token

prflow ingest needs a GitHub token. The recommended way is to store it once with prflow auth github (see below); it can also read the GITHUB_TOKEN environment variable. A fine-grained personal access token scoped to read-only is the safest choice — it can read PRs but cannot push, merge, or change anything.

Create the token

1. Go to GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens → Generate new token (direct link: https://github.com/settings/personal-access-tokens/new).
2. Token name: e.g. prflow-readonly.
3. Expiration: pick a sane limit (e.g. 90 days).
4. Resource owner: select the org/account that owns the repos you'll ingest (e.g. my-organisation). For org-owned repos this must be the org, not your personal account — and an org admin may need to approve the token before it works.
5. Repository access: choose Only select repositories and pick the repo(s) you want to ingest (e.g. generic-api). You can also use "All repositories" if you prefer.
6. Permissions → Repository permissions, set these to Read-only:
   • Pull requests → Read-only ← this is the one that matters
   • Metadata → Read-only ← mandatory; GitHub auto-selects it
   • Leave everything else as No access.
7. Generate token and copy it (starts with github_pat_…). You won't see it again.

Why not a classic token? Classic tokens don't have a read-only repo scope — the repo scope grants full read and write control of private repos. Use a fine-grained token to get true read-only access. (public_repo only covers public repos and still grants write.)

SAML SSO orgs: if the owning org enforces SSO, you must authorize the token for that org after creating it, or requests will still 404.

Provide the token to prflow

Recommended — store it once with the CLI:

./prflow auth github

This prompts for the token (input is hidden, so it won't land in your shell history) and saves it under your XDG config dir (~/.config/prflow/auth.json, mode 0600) — the same place prflow login keeps LLM credentials. It lives outside the project directory, so it's never committed. After that, prflow ingest picks it up automatically; no environment export needed. Remove it later with prflow auth github --clear, and check what's configured with prflow status.

Alternative — environment variable. GITHUB_TOKEN, if set, takes precedence over the stored token (handy for CI or one-off overrides):

export GITHUB_TOKEN=github_pat_xxxxxxxxxxxxxxxxxxxx
# or inline, per command:
GITHUB_TOKEN=github_pat_xxxx ./prflow ingest my-organisation/generic-api

Already use the gh CLI? You can reuse the token gh is already logged in with instead of minting a new one (note this carries whatever scopes your gh login has, which may be broader than read-only) — either pipe it into prflow auth github, or:

export GITHUB_TOKEN=$(gh auth token)

---

Step 3 — Ingest a repo

./prflow ingest <owner>/<repo>

For example:

./prflow ingest my-organisation/generic-api

This fetches PRs, inline review comments, and review summaries into a single shared database at ~/.local/share/prflow/prflow.db (honoring $XDG_DATA_HOME). Every repo you ingest goes into that one DB, so you can analyse a user's feedback across multiple repos at once. Re-running ingest is incremental — it resumes from a stored sync cursor. To force a re-pull from a specific date:

./prflow ingest my-organisation/generic-api --since 2025-05-03

ingest reports progress on stderr and a summary line like done in 4.2s — PRs=37 review_comments=210 reviews=58 when finished.

---

Step 4 — Log in to an LLM provider (for analyse)

analyse uses an LLM via stackllm. Credentials are stored under your XDG config dir — ~/.config/prflow/ (or $XDG_CONFIG_HOME/prflow/) — not in the project directory, so they're never committed. You only need to do this once:

./prflow login

This walks you through picking a provider, signing in (device-code, browser, or API key depending on provider), and choosing a default model. Check status with:

./prflow status

---

Step 5 — Explore the data

List the PR authors you've ingested

Discovery aid — shows who has feedback to analyse:

./prflow users
./prflow users --repo my-organisation/generic-api --min-comments 5

Analyse trends for one engineer

The user (PR author) is a required positional argument:

./prflow analyse alice

Options:

Flag	Default	Meaning
<user>	(required)	GitHub login of the PR author to analyse (positional)
--since	3 months ago	Window start, YYYY-MM-DD
--until	tomorrow	Window end, YYYY-MM-DD
--repo	all ingested	Restrict to one <owner>/<repo>
--prompt-only	off	Print the prompt and data that would be sent to the LLM, then exit without calling it

./prflow analyse alice --repo my-organisation/generic-api --since 2025-01-01

Use --prompt-only to copy the prompt into an interactive browser LLM session instead of calling the configured provider:

./prflow analyse alice --prompt-only > prompt.txt

Full-text search the comments

./prflow search "error handling"
./prflow search "naming" --user alice --limit 10

---

Common flags

• --db <path> (global) — path to the SQLite file. Defaults to the single shared ~/.local/share/prflow/prflow.db (or $XDG_DATA_HOME/prflow/prflow.db). Override it only if you want a separate, project-scoped database. LLM credentials live separately in ~/.config/prflow/, independent of --db.

Typical first run, end to end

go build -o prflow ./cmd/prflow         # 1. build
./prflow auth github                     # 2. store the read-only token (once)
./prflow ingest my-organisation/generic-api           # 3. pull feedback into prflow.db
./prflow login                           # 4. set up the LLM (once)
./prflow users --repo my-organisation/generic-api     # 5. see who's been reviewed
./prflow analyse alice                   # 6. surface trends

---

Troubleshooting

Symptom	Cause / fix
warning: no GitHub token	No token configured — run prflow auth github (or export GITHUB_TOKEN); see Step 2.
ingest pulls: … 404 Not Found	Private repo + no/insufficient token, or wrong owner/repo, or token not approved/SSO-authorized for the org. Confirm with curl -so /dev/null -w "%{http_code}\n" -H "Authorization: Bearer $GITHUB_TOKEN" https://api.github.com/repos/<owner>/<repo> — expect 200.
repo <owner>/<repo> not ingested yet	Run ingest for that repo before analyse/users --repo.
analyse fails on auth/model	Run ./prflow login, then ./prflow status to confirm a provider is authenticated.