Skip to content

Commit e9fde4c

Browse files
NathanFlurryclaude
NathanFlurry
and
claude
committed
chore: add Postgres and TLS test coverage stories to PRD
Postgres gaps (from adversarial review): - US-021: UPDATE, DELETE, transactions (BEGIN/COMMIT/ROLLBACK) - US-022: pg.Pool connection pooling and concurrent queries - US-023: Data types (JSON, JSONB, TIMESTAMPTZ, BYTEA, arrays, UUID) - US-024: Error paths (bad SQL, constraint violations, connection errors) - US-025: Named prepared statements (Parse/Bind/Execute wire protocol) - US-026: Fix auth method discrepancy (local=trust, CI=scram-sha-256) TLS gaps: - US-027: TLS database connections (pg with ssl:true) - US-028: HTTPS error handling (expired certs, hostname mismatch) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 850681e commit e9fde4c

1 file changed

Lines changed: 143 additions & 2 deletions

File tree

‎scripts/ralph/prd.json‎

Lines changed: 143 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -290,8 +290,8 @@
290290
"Tests pass"
291291
],
292292
"priority": 16,
293-
"passes": false,
294-
"notes": "Key-based auth exercises completely different crypto paths than password auth (key parsing, signature generation). If the sandbox's crypto bridge has gaps in sign()/createSign() for RSA/Ed25519, the password-only test would not catch it."
293+
"passes": true,
294+
"notes": "Completed. Added test RSA keypair to fixture and Dockerfile. sshd.Dockerfile updated with PubkeyAuthentication and authorized_keys. Fixture connects with privateKey, runs conn.exec(), verifies parity."
295295
},
296296
{
297297
"id": "US-017",
@@ -359,6 +359,147 @@
359359
"priority": 20,
360360
"passes": false,
361361
"notes": "The current SFTP fixture only transfers 18 bytes. Larger transfers stress the sandbox's TCP buffer management, stream backpressure handling, and memory limits. Also tests createReadStream (streaming) which exercises different buffer management than readFile (buffered)."
362+
},
363+
{
364+
"id": "US-021",
365+
"title": "Expand pg-connect fixture with UPDATE, DELETE, and transactions",
366+
"description": "As a developer, I need the Postgres fixture to test all CRUD operations and transactions so we validate the sandbox handles the full pg wire protocol.",
367+
"acceptanceCriteria": [
368+
"Add UPDATE statement to pg-connect fixture (update the inserted row, verify the change with SELECT)",
369+
"Add DELETE statement (delete the row, verify with SELECT returning 0 rows)",
370+
"Add a transaction block: BEGIN, INSERT, ROLLBACK, verify the row was NOT inserted",
371+
"Add a transaction block: BEGIN, INSERT, COMMIT, verify the row WAS inserted",
372+
"fixture.json expectation remains 'pass'",
373+
"Host and sandbox produce identical normalized output",
374+
"Typecheck passes",
375+
"Tests pass"
376+
],
377+
"priority": 21,
378+
"passes": false,
379+
"notes": "The current fixture only tests CREATE TABLE, INSERT, SELECT, DROP TABLE — missing UPDATE, DELETE, and all transaction semantics. If the sandbox breaks transaction state tracking, nothing catches it."
380+
},
381+
{
382+
"id": "US-022",
383+
"title": "Add pg-pool e2e-docker fixture for connection pooling",
384+
"description": "As a developer, I need a Postgres fixture that tests pg.Pool so we validate pool acquire/release and concurrent queries work through the sandbox.",
385+
"acceptanceCriteria": [
386+
"New fixture pg-pool in tests/e2e-docker/ that uses pg.Pool instead of pg.Client",
387+
"Fixture acquires a client from the pool, runs a query, releases it back",
388+
"Fixture runs multiple concurrent queries via pool.query() shorthand",
389+
"Fixture calls pool.end() for cleanup",
390+
"fixture.json expectation is 'pass'",
391+
"Host and sandbox produce identical normalized output",
392+
"Typecheck passes",
393+
"Tests pass"
394+
],
395+
"priority": 22,
396+
"passes": false,
397+
"notes": "pg.Pool is how everyone uses pg in production. The current fixture only uses pg.Client. Pool acquire/release, idle timeout, and concurrent query dispatch are completely untested."
398+
},
399+
{
400+
"id": "US-023",
401+
"title": "Add pg-types e2e-docker fixture for data type coverage",
402+
"description": "As a developer, I need a Postgres fixture that tests JSON, timestamps, bytea, arrays, and other data types so we validate the sandbox's type parser support.",
403+
"acceptanceCriteria": [
404+
"New fixture pg-types in tests/e2e-docker/ that creates a table with columns: JSON, JSONB, TIMESTAMPTZ, BOOLEAN, BYTEA, INTEGER[], TEXT[], UUID, NUMERIC",
405+
"Fixture inserts a row with values for each type, reads it back, verifies round-trip fidelity",
406+
"Fixture verifies that pg type parsers return the correct JavaScript types (object for JSON, Date for timestamp, Buffer for bytea, Array for arrays)",
407+
"fixture.json expectation is 'pass'",
408+
"Host and sandbox produce identical normalized output",
409+
"Typecheck passes",
410+
"Tests pass"
411+
],
412+
"priority": 23,
413+
"passes": false,
414+
"notes": "Only TEXT and SERIAL are tested. Type parsing in node-pg relies on Buffer, crypto, and type parser functions that the sandbox may not fully support. JSON/JSONB round-trip is especially important for real-world apps."
415+
},
416+
{
417+
"id": "US-024",
418+
"title": "Add pg-errors e2e-docker fixture for error path coverage",
419+
"description": "As a developer, I need a Postgres fixture that tests error handling so we validate the sandbox propagates Postgres errors correctly.",
420+
"acceptanceCriteria": [
421+
"New fixture pg-errors in tests/e2e-docker/ that tests multiple error scenarios",
422+
"Test 1: syntax error (bad SQL) — verify error message and code match host",
423+
"Test 2: query nonexistent table — verify ENOENT-style error matches host",
424+
"Test 3: unique constraint violation (INSERT duplicate) — verify error matches host",
425+
"Test 4: connection to wrong port — verify connection error matches host",
426+
"All errors are caught and their properties (message, code, detail) are printed as JSON for parity comparison",
427+
"fixture.json expectation is 'pass'",
428+
"Host and sandbox produce identical normalized output",
429+
"Typecheck passes",
430+
"Tests pass"
431+
],
432+
"priority": 24,
433+
"passes": false,
434+
"notes": "Zero error paths are tested for Postgres. If the sandbox swallows, reshapes, or corrupts pg error objects, nothing detects it. Error propagation is a common sandbox failure mode."
435+
},
436+
{
437+
"id": "US-025",
438+
"title": "Add pg-prepared e2e-docker fixture for prepared statements",
439+
"description": "As a developer, I need a Postgres fixture that tests named prepared statements so we validate the sandbox handles the Parse/Bind/Execute wire protocol correctly.",
440+
"acceptanceCriteria": [
441+
"New fixture pg-prepared in tests/e2e-docker/ that uses named prepared statements",
442+
"Fixture uses client.query({ name: 'insert-row', text: 'INSERT ...', values: [...] }) syntax",
443+
"Fixture reuses the same named statement multiple times (exercises the prepared statement cache)",
444+
"Fixture verifies results match host behavior",
445+
"fixture.json expectation is 'pass'",
446+
"Host and sandbox produce identical normalized output",
447+
"Typecheck passes",
448+
"Tests pass"
449+
],
450+
"priority": 25,
451+
"passes": false,
452+
"notes": "Named prepared statements use different wire protocol messages (Parse/Bind/Execute vs SimpleQuery). The current fixture only uses unnamed parameterized queries. Prepared statement caching and reuse is a production-critical path."
453+
},
454+
{
455+
"id": "US-026",
456+
"title": "Fix pg auth method discrepancy between local and CI",
457+
"description": "As a developer, I need local and CI Postgres containers to use the same authentication method so we don't silently test different code paths.",
458+
"acceptanceCriteria": [
459+
"Remove POSTGRES_HOST_AUTH_METHOD: trust from local Docker setup in e2e-docker.test.ts",
460+
"Or add the same trust setting to CI — both environments must match",
461+
"If removing trust: verify SCRAM-SHA-256 auth works through the sandbox locally",
462+
"If keeping trust: document why and add a separate fixture that tests SCRAM auth explicitly",
463+
"Tests pass both locally and in CI",
464+
"Typecheck passes"
465+
],
466+
"priority": 26,
467+
"passes": false,
468+
"notes": "Local uses POSTGRES_HOST_AUTH_METHOD: trust (password bypassed), CI defaults to scram-sha-256. This means the SCRAM crypto path is only tested in CI and never locally. If a developer breaks crypto support needed for SCRAM, they won't catch it locally."
469+
},
470+
{
471+
"id": "US-027",
472+
"title": "Add TLS database connection e2e-docker fixtures",
473+
"description": "As a developer, I need e2e-docker fixtures that test TLS-encrypted database connections so we validate the sandbox's HTTPS/TLS bridge handles SSL database connections correctly.",
474+
"acceptanceCriteria": [
475+
"New fixture pg-ssl in tests/e2e-docker/ that connects to Postgres with ssl: { rejectUnauthorized: false }",
476+
"Configure the Postgres container with SSL enabled (self-signed cert)",
477+
"Fixture connects, runs a query, verifies the connection was encrypted (check pg_stat_ssl or similar)",
478+
"fixture.json expectation is 'pass'",
479+
"Host and sandbox produce identical normalized output",
480+
"Typecheck passes",
481+
"Tests pass"
482+
],
483+
"priority": 27,
484+
"passes": false,
485+
"notes": "No database connection in the test suite uses TLS/SSL. In production, pg connections are almost always SSL-encrypted. The pg library's SSL support goes through a different code path (tls.connect wrapper) than plain TCP. Since the tls module is deferred (Tier 4), this may require implementing enough of tls for pg's SSL to work, or documenting the limitation."
486+
},
487+
{
488+
"id": "US-028",
489+
"title": "Add HTTPS error handling e2e fixture for TLS edge cases",
490+
"description": "As a developer, I need tests that verify the sandbox correctly handles TLS error cases like expired certs, hostname mismatches, and self-signed certs with rejectUnauthorized:true.",
491+
"acceptanceCriteria": [
492+
"New test cases in tests/runtime-driver/node/ that test HTTPS error scenarios through the sandbox",
493+
"Test 1: fetch() to HTTPS server with expired cert — verify error matches host",
494+
"Test 2: fetch() to HTTPS server with hostname mismatch — verify error matches host",
495+
"Test 3: fetch() with rejectUnauthorized:true to self-signed cert — verify rejection matches host",
496+
"All errors caught and compared for parity with host Node.js behavior",
497+
"Typecheck passes",
498+
"Tests pass"
499+
],
500+
"priority": 28,
501+
"passes": false,
502+
"notes": "The existing https-streams.test.ts tests the happy path (self-signed with rejectUnauthorized:false and valid cert chain). Error cases for TLS validation failures are untested. If the sandbox silently accepts bad certs, this is a security issue."
362503
}
363504
]
364505
}

0 commit comments

Comments
 (0)