[build-tools] Add eas/posthog_wait_for_metric workflow function

[gwdp]

Why

Gate an EAS workflow on a PostHog metric. Hold a rollout until errors stay low (or any metric clears) before it continues, instead of watching dashboards by hand. imo this is the coolest 🔥

How

Adds eas/posthog_wait_for_metric. It polls a HogQL query until operator/threshold is met, every interval_seconds (default 30) for up to timeout_seconds (default 600). Auth is the same personal key as the flag step, and the key needs the query read scope. (good docs coming)

It also sets a value output with the query result that met the threshold, so a later step can use ${{ steps.<step id>.outputs.value }}.

There's no ignore_error here on purpose. It's a gate, so timing out fails the step. Network errors, 429 and 5xx retry until the deadline. Auth and query errors fail right away.

Test Plan

CI passes. Test workflow below. The second and third jobs fail on purpose (timeout and bad query shape). Not live-run yet, since the key from connect lacks the query scope. Will run it before undrafting.

Test workflow

name: PostHog wait-for-metric test

jobs:
  wait_cleared:
    steps:
      - uses: eas/posthog_wait_for_metric
        with:
          query: select count() from events
          operator: gte
          threshold: 0

  wait_timeout:
    steps:
      - uses: eas/posthog_wait_for_metric
        with:
          query: select count() from events
          operator: lt
          threshold: 0
          interval_seconds: 5
          timeout_seconds: 20

  wait_bad_query:
    steps:
      - uses: eas/posthog_wait_for_metric
        with:
          query: select event from events limit 1
          operator: gte
          threshold: 0
          interval_seconds: 5
          timeout_seconds: 15

Examples

These are simple workflows just to exercise the step. More fun and useful ones, mixed with EAS Update and deploys, are coming in the docs.

Send an event and wait for it

Two jobs with no needs, so they run at the same time. One sends the event, the other waits for it to land. This needs real concurrency. It won't clear on a local runner set to concurrency 1, since the jobs run one after another there.

name: Send an event and await it in parallel

jobs:
  send_ping:
    steps:
      - uses: eas/posthog_capture_event
        with:
          event: workflow_ping
          properties:
            source: parallel-demo

  await_ping:
    steps:
      - uses: eas/posthog_wait_for_metric
        with:
          query: select count() from events where event = 'workflow_ping' and timestamp > now() - interval 5 minute
          operator: gte
          threshold: 1
          interval_seconds: 10
          timeout_seconds: 300

Use the metric value in a later step

The gate exposes the reading it settled on as an output. A later step reads it with ${{ steps.gate.outputs.value }}.

name: Gate on a metric, then use its value

jobs:
  gated_announce:
    steps:
      - id: gate
        uses: eas/posthog_wait_for_metric
        with:
          query: select count() from events where event = '$exception' and timestamp > now() - interval 1 hour
          operator: lte
          threshold: 5
      - uses: eas/posthog_capture_event
        with:
          event: rollout_gate_cleared
          properties:
            error_count: ${{ steps.gate.outputs.value }}

[linear-code]

ENG-21551

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.40%. Comparing base (3745235) to head (25c9c2e).

Additional details and impacted files

@@                           Coverage Diff                           @@
##           gwdp/eng-21551-posthog-flag-rollout    #3945      +/-   ##
=======================================================================
+ Coverage                                59.33%   59.40%   +0.08%     
=======================================================================
  Files                                      938      939       +1     
  Lines                                    41223    41296      +73     
  Branches                                  8685     8699      +14     
=======================================================================
+ Hits                                     24454    24527      +73     
  Misses                                   16674    16674              
  Partials                                    95       95              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

✅ Thank you for adding the changelog entry!