Skip to content

Extract code generation logic into internal/api package#4418

Open
kyleconroy wants to merge 1 commit intomainfrom
claude/create-internal-api-package-INVIA
Open

Extract code generation logic into internal/api package#4418
kyleconroy wants to merge 1 commit intomainfrom
claude/create-internal-api-package-INVIA

Conversation

@kyleconroy
Copy link
Copy Markdown
Collaborator

@kyleconroy kyleconroy commented Apr 28, 2026

Introduce internal/api, a small programmatic interface to sqlc's code generation, inspired by esbuild's Build API. One entry point, api.Generate(ctx, api.GenerateOptions{}), returns a GenerateResult. The CLI becomes a thin wrapper.

Public surface

The package exports exactly three names:

func Generate(ctx context.Context, opts GenerateOptions) GenerateResult

type GenerateOptions struct {
    Config               io.Reader // sqlc YAML/JSON config; required
    Stderr               io.Writer
    Write                bool      // write generated files to disk
    Diff                 bool      // diff generated files against on-disk
    BaseDir              string    // resolves relative paths in Config; strips prefix in stderr labels
    EnableProcessPlugins bool      // permit process-based plugins to run
}

type GenerateResult struct {
    Files  map[string]string
    Errors []error
}

Everything else in the package — parse, codegen, the plugin shim, the result processor — is unexported.

Notable design choices

  • Config as io.Reader. No Dir/File fields. Callers hand sqlc whatever bytes they want. The CLI reads from disk; library callers can construct configs in memory.
  • BaseDir does double duty. It's the directory relative paths in the config resolve against and the prefix stripped from file paths in parse errors and diff labels. When empty it defaults to the current working directory. Six fields total, no separate "dir for resolving" vs. "dir for labels."
  • Process plugins are off by default. EnableProcessPlugins is a single bool whose zero value refuses any process plugin in the config — process plugins execute arbitrary local commands, so callers must opt in. The CLI sets it from env.Debug.ProcessPlugins, which defaults to true and flips off under SQLCDEBUG=processplugins=0.
  • Write/Diff replace separate functions. sqlc generate is api.Generate{Write: true}, sqlc compile is neither, sqlc diff is {Diff: true}. cmd.Diff is gone.

CLI

internal/cmd/cmd.go's genCmd, checkCmd, and diffCmd each:

  1. Read the config bytes.
  2. os.Chdir into the config's directory so relative paths resolve.
  3. Call api.Generate with Config: bytes.NewReader(data), BaseDir: configDir, and EnableProcessPlugins: env.Debug.ProcessPlugins.

cmd.Vet and cmd.Push still use the cmd-local helpers (parse, processQuerySets, codeGenRequest) since they have surface beyond what api covers; both packages skip joining their dir parameter when the path is already absolute, so configs with absolute paths flow through both.

Endtoend tests

internal/endtoend/endtoend_test.go calls api.Generate directly. A small mutatedConfigBytes helper parses the test's config, optionally applies a mutation (the managed-db context adds servers + sets database.managed), forces version "2" so v1 configs round-trip cleanly, and re-encodes as YAML. When mutated, the bytes are also dropped to a temp file alongside the original so cmd.Vet (which still takes a path) can use it.

Per-test environment variables from exec.json are applied via t.Setenv, and cmd.Env is then populated via opts.DebugFromEnv/ExperimentFromEnv — same path the CLI takes.

config.AnalyzerDatabase gained MarshalYAML/MarshalJSON so the parsed Config round-trips through yaml.Marshal cleanly — needed for the test helper.

Files

  • New: internal/api/{api,generate,process,parse,codegen,shim,diff}.go
  • Modified: internal/cmd/{cmd,generate,vet,process}.go, internal/config/config.go, internal/endtoend/endtoend_test.go, internal/endtoend/vet_test.go
  • Removed: internal/cmd/diff.go

Test plan

  • go test ./internal/endtoend/... (TestExamples, TestReplay base + managed-db) passes locally
  • go test ./... passes outside of pre-existing MySQL infra failures (TestExpandMySQL, TestValidSchema/*/mysql/*, TestExamplesVet/{authors,booktest,ondeck})
  • go build ./... and go vet ./... clean

@kyleconroy kyleconroy marked this pull request as ready for review May 2, 2026 04:13
Introduce a small programmatic interface to sqlc's code generation,
inspired by esbuild's Build API. The api package exports three names:

    func Generate(ctx, GenerateOptions) GenerateResult
    type GenerateOptions struct {
        Config               io.Reader
        Stderr               io.Writer
        Write                bool
        Diff                 bool
        BaseDir              string
        EnableProcessPlugins bool
    }
    type GenerateResult struct {
        Files  map[string]string
        Errors []error
    }

Everything else in the package — parse, codegen, the plugin shim, the
result processor — is unexported.

Notable design choices:

* Config as io.Reader. No Dir/File fields. Callers hand sqlc whatever
  bytes they want; the CLI reads from disk and library callers can
  construct configs in memory.
* BaseDir does double duty: it's the directory relative paths in the
  config resolve against and the prefix stripped from file paths in
  parse errors and diff labels. When empty it defaults to the current
  working directory.
* Process plugins are off by default. EnableProcessPlugins is a single
  bool whose zero value refuses any process plugin in the config. The
  CLI sets it from the `processplugins` SQLCDEBUG setting (default 1).
* Write/Diff replace separate functions. `sqlc generate` is
  api.Generate{Write: true}, `sqlc compile` is neither, `sqlc diff` is
  {Diff: true}. cmd.Diff is gone.

CLI: each of genCmd, checkCmd, and diffCmd reads the config bytes,
chdirs into the config's directory so relative paths resolve, and calls
api.Generate with BaseDir set to that directory. cmd.Vet and cmd.Push
keep their own helpers (parse, processQuerySets, codeGenRequest) since
they have surface beyond what api covers; both packages skip joining
their dir parameter when the path is already absolute, so configs with
absolute paths flow through both.

Endtoend tests: TestExamples and TestReplay call api.Generate directly.
A mutatedConfigBytes helper parses the test's config, optionally
applies a mutation (the managed-db context adds servers + sets
database.managed), forces version "2" so v1 configs round-trip cleanly,
and re-encodes as YAML. When mutated, the bytes are also dropped to a
temp file alongside the original so cmd.Vet (which still takes a path)
can use it. Per-test environment variables from exec.json are applied
via t.Setenv, and cmd.Env is then populated via opts.ExperimentFromEnv
— same path the CLI takes.

config.AnalyzerDatabase gained MarshalYAML/MarshalJSON so the parsed
Config round-trips through yaml.Marshal cleanly — needed for the test
helper.
@kyleconroy kyleconroy force-pushed the claude/create-internal-api-package-INVIA branch from 8700f30 to 8dcf3bb Compare May 6, 2026 16:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants