Skip to content

Fix benchmark dashboard artifact downloads#178

Merged
abap34 merged 1 commit intomainfrom
codex-fix-benchmark-dashboard
Apr 13, 2026
Merged

Fix benchmark dashboard artifact downloads#178
abap34 merged 1 commit intomainfrom
codex-fix-benchmark-dashboard

Conversation

@abap34
Copy link
Copy Markdown
Owner

@abap34 abap34 commented Apr 13, 2026

Summary

  • fix benchmark dashboard artifact downloads by handling GitHub artifact redirects explicitly
  • avoid sending the GitHub Authorization header to the signed object-storage URL
  • allow the dashboard builder to run on local Python 3.9 via postponed annotations

Cause

The benchmark Pages builder found dashboard-entry artifacts, but Python urllib followed the artifact download redirect while keeping the GitHub Authorization header. The redirected signed storage URL rejected that request with 401, so every CI run was skipped and dev/bench/dashboard.json was generated with entries: [].

Validation

  • PYTHONPYCACHEPREFIX=/tmp/almo-pycache python3 -m py_compile scripts/build_benchmark_dashboard.py
  • python3 scripts/build_benchmark_dashboard.py --repo abap34/almo --out-dir /tmp/almo-dashboard-fixture-mainfix --fixture-json scripts/benchmark_pages/fixture_dashboard.json
  • verified real GitHub artifacts can be fetched: latest 3 CI runs produced entries=3 with 21 benchmarks each

generated by @codex

@chatgpt-codex-connector
Copy link
Copy Markdown

To use Codex here, create an environment for this repo.

@github-actions
Copy link
Copy Markdown

CI Summary

Check Status Details
Tests Success 16 passed / 0 failed
Benchmarks Success 21 benchmark(s) captured
Coverage Success 93.1% (1280 of 1375 lines)

Benchmark Means

Benchmark Mean (ms) Min (ms) Max (ms) Iterations
parse_small_document 9.008 8.956 9.056 5
parse_large_document 191.628 190.712 192.295 5
parse_realistic_article_document 22.843 22.778 22.942 5
parse_release_notes_document 7.402 7.358 7.519 5
parse_inline_heavy_document 347.236 346.621 347.767 5
parse_table_heavy_document 160.844 157.465 171.114 5
parse_footnote_heavy_document 56.845 56.599 57.042 5
parse_structured_document 49.583 49.163 49.996 5
parse_judge_document 0.029 0.028 0.031 5
render_small_document 18.258 18.144 18.335 5
render_large_document 228.749 216.069 243.606 5
render_realistic_article_document 32.065 31.532 32.478 5
render_release_notes_document 16.445 16.247 17.033 5
render_structured_document 67.239 66.806 67.584 5
render_table_heavy_document 216.472 215.805 218.353 5
render_footnote_heavy_document 71.605 71.493 71.714 5
render_interactive_document 12.870 12.814 12.959 5
render_judge_document 13.164 13.110 13.231 5
summary_inline_heavy_document 389.244 369.314 416.853 5
summary_realistic_article_document 32.029 31.066 33.964 5
summary_large_document 66.963 66.679 67.527 5

Benchmark Diff vs main

Benchmark Base Mean (ms) PR Mean (ms) Delta (ms) Change
parse_small_document 9.167 9.008 -0.159 ✅ 1.7% speed up
parse_large_document 198.466 191.628 -6.838 ✅ 3.4% speed up
parse_realistic_article_document 26.598 22.843 -3.755 ✅ 14.1% speed up
parse_release_notes_document 8.646 7.402 -1.244 ✅ 14.4% speed up
parse_inline_heavy_document 377.304 347.236 -30.068 ✅ 8.0% speed up
parse_table_heavy_document 156.870 160.844 +3.974 ⚠️ 2.5% slow down
parse_footnote_heavy_document 56.910 56.845 -0.065 ➖ 0.1% change
parse_structured_document 49.852 49.583 -0.269 ➖ 0.5% change
parse_judge_document 0.031 0.029 -0.002 ✅ 6.5% speed up
render_small_document 18.638 18.258 -0.380 ✅ 2.0% speed up
render_large_document 218.799 228.749 +9.950 ⚠️ 4.5% slow down
render_realistic_article_document 32.964 32.065 -0.899 ✅ 2.7% speed up
render_release_notes_document 17.182 16.445 -0.737 ✅ 4.3% speed up
render_structured_document 67.879 67.239 -0.640 ➖ 0.9% change
render_table_heavy_document 216.951 216.472 -0.479 ➖ 0.2% change
render_footnote_heavy_document 72.933 71.605 -1.328 ✅ 1.8% speed up
render_interactive_document 13.203 12.870 -0.333 ✅ 2.5% speed up
render_judge_document 13.535 13.164 -0.371 ✅ 2.7% speed up
summary_inline_heavy_document 389.409 389.244 -0.165 ➖ 0.0% change
summary_realistic_article_document 32.658 32.029 -0.629 ✅ 1.9% speed up
summary_large_document 67.901 66.963 -0.938 ✅ 1.4% speed up

Benchmark dashboard: https://abap34.github.io/almo/dev/bench/?pr=178
PR preview: https://abap34.github.io/almo/dev/previews/pr-178/

@abap34 abap34 merged commit f3009f6 into main Apr 13, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant