Skip to content

feat(db): Database connection multiplexing for serverless environments (#600)#651

Merged
Smartdevs17 merged 2 commits into
Smartdevs17:mainfrom
jadonamite:feat/serverless-db-multiplexing
Jul 3, 2026
Merged

feat(db): Database connection multiplexing for serverless environments (#600)#651
Smartdevs17 merged 2 commits into
Smartdevs17:mainfrom
jadonamite:feat/serverless-db-multiplexing

Conversation

@jadonamite

Copy link
Copy Markdown
Contributor

Summary

Implements database connection multiplexing for serverless workloads. Serverless functions (webhook handlers, auth callbacks, scheduled jobs) currently open a new DB connection per invocation, exhausting the connection limit during traffic spikes. This routes all connections through a transaction-pooling proxy (AWS RDS Proxy or self-hosted PgBouncer), so a small set of backend connections is multiplexed across hundreds of concurrent functions.

Changes

  • backend/shared/db/serverlessPool.tsServerlessConnectionPool: transaction-pooling adapter over the existing pg pool, with withClient/withTransaction/query helpers, IAM and SCRAM-256 auth modes, per-connect credential refresh (rotating RDS tokens), and connection-leak detection that force-closes any client checked out longer than 30s.
  • backend/serverless/dbConfig.ts — env-driven pool configuration, including an RDS IAM auth-token credential provider (@aws-sdk/rds-signer, lazily imported).
  • backend/serverless/withDatabase.ts — Lambda handler wrapper that hands each invocation a pooled client and calls release() in a finally block, regardless of success/throw. Pool singleton is reused across warm invocations.
  • backend/monitoring/connectionPoolMetrics.ts — Prometheus pool gauges (total/idle/waiting/checked-out), a *_leaked_total counter, and structured leak alerting.
  • infra/terraform/rds_proxy.tf — RDS Proxy with IAM auth, transaction pooling (~50 backend connections), TLS required, 30s idle timeout, EXCLUDE_VARIABLE_SETS pinning filter.
  • infra/terraform/pgbouncer.tf — self-hosted PgBouncer alternative (transaction mode, SCRAM-256, MAX_PREPARED_STATEMENTS, 500 client cap / 50 server pool).
  • docker-compose.yml — local Postgres + PgBouncer for dev/test parity.

Acceptance criteria

  • PgBouncer deployment (docker-compose + Terraform) or RDS Proxy configuration
  • Transaction pooling mode: connections reused across invocations
  • Prepared-statement support (PgBouncer MAX_PREPARED_STATEMENTS / RDS Proxy)
  • Authentication: IAM-based (AWS RDS) or SCRAM-256 (self-hosted)
  • Connection limits: max ~50 pooled connections serving 500+ concurrent functions
  • Lambda handler adaptation: release() in a finally block per invocation
  • Edge case: connection-leak detection (abandoned >30s, log and force-close)

Closes #600

@drips-wave

drips-wave Bot commented Jun 24, 2026

Copy link
Copy Markdown

@jadonamite Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@gitguardian

gitguardian Bot commented Jun 24, 2026

Copy link
Copy Markdown

️✅ There are no secrets present in this pull request anymore.

If these secrets were true positive and are still valid, we highly recommend you to revoke them.
While these secrets were previously flagged, we no longer have a reference to the
specific commits where they were detected. Once a secret has been leaked into a git
repository, you should consider it compromised, even if it was deleted immediately.
Find here more information about risks.


🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

@jadonamite jadonamite force-pushed the feat/serverless-db-multiplexing branch from 615f491 to 7167381 Compare June 24, 2026 23:15
@Smartdevs17

Copy link
Copy Markdown
Owner

⚠️ Merge failed — could not merge.

Possible reasons:

  • CI checks not passing
  • Branch protection rules
  • Other merge requirements not met

🤖 Drips Wave Merge Agent

…evs17#600)

Route serverless DB access through a transaction-pooling proxy (RDS Proxy
or PgBouncer) so a small set of backend connections is multiplexed across
many concurrent function invocations, preventing connection exhaustion.

- backend/shared/db/serverlessPool.ts: transaction-pooling adapter with
  IAM/SCRAM-256 auth, credential refresh, withClient/withTransaction
  helpers, and abandoned-connection leak detection (>30s force-close).
- backend/serverless/dbConfig.ts + withDatabase.ts: env-driven pool config
  (RDS IAM token provider) and a Lambda wrapper that releases the client in
  a finally block after every invocation.
- backend/monitoring/connectionPoolMetrics.ts: Prometheus pool/leak metrics
  and structured leak alerting.
- infra/terraform/{rds_proxy,pgbouncer}.tf: proxy provisioning (max ~50
  pooled connections serving 500+ functions, transaction pooling).
- docker-compose.yml + .env.example: local PgBouncer + Postgres for parity;
  all credentials read from a gitignored .env (no hardcoded secrets).

Closes Smartdevs17#600
The dependencies block had a duplicate "zustand" entry and missing
commas, producing invalid JSON that breaks `npm install` in CI. Keep a
single zustand ^5.0.0 and the redis ^4.6.7 entry.
@jadonamite jadonamite force-pushed the feat/serverless-db-multiplexing branch from 7167381 to 4e848fa Compare July 3, 2026 07:35
@jadonamite

Copy link
Copy Markdown
Contributor Author

@Smartdevs17 Changes made, waiting for your review.

@Smartdevs17 Smartdevs17 merged commit 57d61c5 into Smartdevs17:main Jul 3, 2026
1 check passed
@Smartdevs17

Copy link
Copy Markdown
Owner

@jadonamite merged. Thanks for contributing. Kindly drop a review 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement database connection multiplexing for serverless environments

2 participants