ci(examples): assert exact status/body instead of liveness-only check#755
Open
JamBalaya56562 wants to merge 2 commits into
Open
ci(examples): assert exact status/body instead of liveness-only check#755JamBalaya56562 wants to merge 2 commits into
JamBalaya56562 wants to merge 2 commits into
Conversation
The verify step in test-image/test-zip/test-stream treated any 2xx-4xx
response as success, so a broken route (404) or an app that booted but
served the wrong/empty content still passed.
Add .github/scripts/verify-http.sh which waits for an endpoint to return
an exact status code and (optionally) a known body substring, and convert
each job's matrix to objects carrying {path, expect_status, expect_body}
so every example asserts its real response.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
|
Thanks for improving the testing for examples. One more example needs a small change: The port is hard coded here. Should be able to overwrite with PORT. |
Contributor
Author
|
Good catch, thanks! Fixed in 0408b33 — |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Before this PR, the verify step in
test-image/test-zip/test-streamjobs treated any 2xx-4xx response as success (a plain liveness check). That means a broken route returning 404, or an app that booted but served empty/wrong content, would still pass example CI silently.This PR adds exact response assertions: every example now declares the expected HTTP status (and optionally a body substring), and the verify step fails unless the real response matches.
Changes
Adds
.github/scripts/verify-http.sh— polls an endpoint until both conditions hold:200, or200,307for redirect-then-page)Uses a wall-clock deadline (90s) plus a short per-request timeout (
curl -m 5) so a hanging endpoint can't multiply the wait window. Failure prints the last status, the expected substring, and the first 30 lines of the last body as diagnostics.Updates
.github/workflows/examples.yaml— converts the matrix in each verify job from plain example names to objects carrying{path, expect_status, expect_body}, so every example asserts its real response.examples/expressjs/— switches the local listen port from 8080 to 8000 (ENV PORT=8000+EXPOSE 8000) so the example doesn't collide with SAM's Runtime Interface Emulator on 8080 during local testing. README updated to match.examples/expressjs-zip/template.yaml— samePORT: 8000adjustment for the zip-style variant, with an inline comment explaining the reason.examples/fasthtml-response-streaming/— the app hardcodedserve(port=8080)and ignored thePORTenvironment variable that every other example (e.g.fastapi-response-streaming) already respects. ReadsPORTwith an8080fallback so the listening port can be overridden. With the app now honoringPORT, theappportworkaround in thetest-streamjob is dropped: the image case maps8000:8000like the rest instead of routing host8000to the hardcoded8080.Test plan
test-image,test-zip,test-stream) pass on this PR.fasthtml-response-streaminghonorsPORT=8000in thetest-streamimage job and the verify step passes against8000:8000.🤖 Generated with Claude Code