Commit ec49ef5
authored
feat(pam): add MongoDB support (#162)
* feat: add MongoDB support to PAM database proxy
Implement MongoDB handler using the mongo.Client bridge approach (Approach B).
The proxy parses client OP_MSG commands, executes them via RunCommand on a
real mongo.Client connection, and encodes responses back as OP_MSG.
- Wire protocol helpers for reading/writing OP_MSG with Kind 0/1 sections
- mongo.Client setup with SetDirect(true), SetMaxPoolSize(1), SRV support
- Command bridge with full BSON session recording
- Auth interception (saslStart/saslContinue faked as success)
- Hello response sanitization (strips compression/auth fields)
- Connection string display for MongoDB resources
* fix: handle legacy OP_QUERY (opcode 2004) for mongosh handshake
mongosh sends OP_QUERY for the initial isMaster/hello handshake even on
modern servers. Add OP_QUERY parsing and OP_REPLY response building so
the proxy can complete the handshake before switching to OP_MSG.
* fix: add TCP pre-check and better error visibility for MongoDB proxy
- Verify raw TCP connectivity before creating mongo.Client
- Reduce timeouts (5s connect, 10s server selection, 2s heartbeat)
- Add ServerMonitor to log actual heartbeat failures (auth/TLS errors)
* fix: strip first-hello-only fields before forwarding via RunCommand
The mongo.Client already sent its own hello with client metadata during
connection setup. Strip client, compression, saslSupportedMechs, and
speculativeAuthenticate from hello/isMaster commands before forwarding,
since MongoDB only allows these in the first hello on a connection.
* fix: strip all driver-managed fields and clean up error handling
- Strip lsid, $clusterTime, $readPreference, txnNumber, startTransaction,
autocommit from all commands before RunCommand to prevent duplicate BSON
field errors (fixes 'ping.lsid is a duplicate field')
- Consolidate all field stripping into executeAndLog
- Handle client EOF as graceful disconnect instead of error
- Use fresh context for client.Disconnect to ensure clean shutdown
- Suppress expected 'context canceled' heartbeat errors during shutdown
* fix: handle moreToCome flag on client OP_MSG requests
When the client sets moreToCome (bit 1), it will send more messages
before expecting a response. Execute the command for side-effects but
don't reply — responding to moreToCome messages injects unexpected bytes
into the TCP stream, corrupting subsequent commands.
Also add named constants for OP_MSG flag bits.
* fix: increase timeouts for remote MongoDB servers
Increase connect timeout to 10s, server selection to 15s, and reduce
heartbeat frequency to 30s. Prevents connection storms when mongosh
opens multiple connections to high-latency remote servers.
* fix: resolve remote MongoDB connectivity issues
Three fixes for high-latency remote MongoDB servers:
1. Synthetic initial handshake: respond to mongosh's first hello
immediately before connectToTarget, preventing 2s timeout when
the target takes 1-5s to connect (TCP + TLS + SCRAM).
2. Strip monitoring long-poll fields: remove topologyVersion and
maxAwaitTimeMS from hello/isMaster commands so the server responds
immediately instead of blocking the bridge for 10+ seconds.
3. Clean up debug logging to Debug level.
* fix: clean up MongoDB session logs by stripping protocol noise
Strip driver/protocol fields ($clusterTime, lsid, $readPreference, etc.)
from logged input so the actual command data is visible. Skip logging
internal commands (ismaster, hello, ping) that are not user activity.
* perf: replace mongo driver with direct TCP/SCRAM for MongoDB proxy
mongosh opens 4+ connections and cycles them every 10-20s. Each
connection previously created a full mongo.Connect() with pooling,
heartbeats, and topology monitoring — adding 2-5s per reconnect.
Replace the Go mongo driver with direct TCP/TLS connections and raw
wire protocol forwarding, matching the pattern used by the Postgres,
MySQL, MSSQL, and Redis handlers. Implement SCRAM-SHA-256/SHA-1 auth
using the xdg-go/scram library (already a transitive dep). Add SRV
resolution via net.LookupSRV/LookupTXT for mongodb+srv:// URLs.
Client commands are now forwarded as raw wire messages instead of being
parsed, field-stripped, and re-executed via RunCommand. Session logging
still inspects BSON for recording.
* chore: update e2e go.mod dependencies for mongo-driver v1.17.9
* perf: delegate MongoDB proxy to driver topology for connection pooling and correctness
The old MongoDB proxy reimplemented SRV resolution, SCRAM auth, and TLS
from scratch, causing per-connection overhead and a fabricated server hello.
Replace with the official driver's topology which handles all of this
internally, pools connections per session, and returns real server responses.
- Use topology.Topology with driver-managed SRV, TLS, auth, and pooling
- Share topology per session in the gateway (first connection creates, others reuse)
- Send warmup connection on session start so topology is ready before first client
- Replace custom wire protocol parsing with driver's wiremessage/bsoncore packages
- Block auth commands with proper wire protocol errors instead of fake SCRAM responses
- Add message size validation from server hello
- Accept MongoDB URIs in host field (authSource and other options in the URI)
- Fix TLS ServerName for MongoDB (let driver set per replica set member)
* fix: close warmup connection immediately and set minPoolSize for warm pool
The warmup connection's bridge was holding a pooled Atlas connection
hostage, forcing every real client to create a new TCP+TLS+auth
connection (~1-2s), exceeding mongosh's 2-second timeout.
- Close warmup connection immediately after triggering topology creation
- Set minPoolSize=2 so the pool proactively maintains warm connections
* refactor: migrate MongoDB proxy from driver v1 to v2
v1.17.x is no longer actively developed (bug fixes only until early 2026).
Migrates to go.mongodb.org/mongo-driver/v2 and removes the unnecessary
bsoncore dependency in favor of bson.Raw native methods.
- Update all imports to v2 paths
- Replace driver.Connection with *mnet.Connection (Read/Write API)
- Replace description.ReadPrefSelector with local primarySelector
- Remove x/bsonx/bsoncore, use bson.Raw.IndexErr/LookupErr directly
* chore: remove unused wire protocol constants and buildOpReply
Remove buildOpReply (never called), opReplyOpCode (only used by it),
flagChecksumPresent (redundant with wiremessage.ChecksumPresent),
and flagExhaustAllowed (proxy doesn't handle exhaust cursors).
* fix: strip client metadata from hello requests on reused connections
MongoDB 6.1+ rejects hello commands with client metadata on connections
that already completed a handshake. Since the proxy forwards through
driver-pooled connections that are already handshaked, strip the `client`
and `compression` fields from hello requests before forwarding.
* fix: strip client metadata from OP_QUERY hello on reused connections
mongosh sends the initial isMaster as OP_QUERY (not OP_MSG). The
previous fix only sanitized OP_MSG hello requests, missing this path.
* fix: convert OP_QUERY to OP_MSG and preserve BSON key order in hello sanitization
Two bugs fixed:
1. sanitizeHelloRequest used bson.M (map, random key order) which could
reorder BSON fields so the command name was no longer first, causing
"no such command: topologyVersion" errors. Switched to bson.D.
2. MongoDB 8.0+ removed OP_QUERY support entirely. The proxy now converts
OP_QUERY hello/isMaster to OP_MSG before forwarding, and converts the
OP_MSG response back to OP_REPLY for the client.
* fix: guard TLS config with EnableTLS flag and add server selection timeout
The caller passes a non-nil TLSConfig even when EnableTLS=false, which
caused the driver to attempt TLS on non-TLS servers (standalone Docker
MongoDB). Now only applies TLSConfig when EnableTLS is true.
Also adds directConnection for non-SRV hosts (needed for standalone
servers) and a 10s server selection timeout to surface connection
errors instead of blocking forever.
* fix: strip awaitable hello fields and moreToCome flag to prevent connection corruption
- Strip topologyVersion and maxAwaitTimeMS from hello requests to prevent
the server from entering streaming mode (moreToCome responses)
- Clear moreToCome flag from all responses sent to the client
- Clear exhaustAllowed flag from client requests to prevent exhaust cursors
- Fix bson.M → bson.D in sanitizeHelloWireMessage (response key ordering)
- Remove minPoolSize to avoid warm connection interference
* fix: strip topologyVersion from hello responses and pre-warm proxy connection
The root cause of "Server ended moreToCome unexpectedly" was that hello
responses included topologyVersion, causing mongosh's SDAM monitor to use
exhaustCommand() for streaming hello. The proxy strips awaitable fields
from requests but the client still attempted exhaust mode because it saw
topologyVersion in the response. Stripping it forces polling mode, which
is fully compatible with a request/response proxy.
Additional fixes:
- Sanitize hello responses in the OP_QUERY path (was missing)
- Add clearMoreToCome to forwardRaw and handleOpQuery non-hello paths
- Drain server-side moreToCome continuations as a safety net
- Pre-warm a pool connection during NewMongoDBProxy
- Make warmup synchronous: send a real hello through the full chain
(local → relay → gateway → MongoDB) and block until the response
confirms the topology is ready, so mongosh connects on first try
* refactor: move warmup protocol logic to mongodb package
Move the MongoDB wire protocol hello/response logic from database-proxy.go
into mongodb.Warmup(), keeping the local proxy thin and protocol-agnostic.
* refactor: use serverSelectionTimeoutMS instead of protocol-level warmup
Instead of sending MongoDB wire protocol data to detect when the gateway's
topology is ready, simply include serverSelectionTimeoutMS=15000 in the
displayed connection string. This gives the first connection enough time
for topology creation while keeping the local proxy protocol-agnostic.
The async warmup is kept to trigger topology creation early on the gateway.
* fix: correct misleading comment on connection pre-warm in NewMongoDBProxy
* chore: remove unused fields and clean up comments in mongodb proxy
- Remove unused opQuery fields (Flags, Skip, Return)
- Replace client-specific references (mongosh) with generic language
- Simplify overly verbose comments to focus on intent over mechanics
- Fix misleading moreToCome comment (was describing wrong direction)
* chore: remove warmup connection from database proxy
The serverSelectionTimeoutMS=15000 in the connection string is sufficient
for the first connection to succeed while the gateway creates the topology.
The warmup optimization is unnecessary complexity.
* fix: sync e2e go.mod with mongo-driver/v2 transitive dependencies
* fix: inject database into MongoDB URI path for full connection strings
When the backend sends a full MongoDB URI as the connection string,
buildURI only injected credentials but silently dropped the database
field. Also fixed credential escaping in the plain-host-spec branch
to use url.UserPassword instead of url.PathEscape.
* refactor: address PR review comments for MongoDB proxy
- Extract packOpMsg helper to deduplicate OP_MSG rebuild logic
- Add max iteration cap on moreToCome drain loop
- Add comment explaining request ID offset at 1000
* fix: remove broken URI fallback in injectCredentials, return error instead
The fallback path on url.Parse failure silently produced a URI missing
the database argument. Since url.Parse practically never fails on valid
URIs, replace the fallback with a proper error return and propagate it
through buildURI → NewMongoDBProxy.
---------
Co-authored-by: saif <11242541+saifsmailbox98@users.noreply.github.com>1 parent fa03013 commit ec49ef5
13 files changed
Lines changed: 1375 additions & 11 deletions
File tree
- e2e
- packages
- api
- gateway-v2
- pam
- handlers/mongodb
- local
- session
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
272 | 272 | | |
273 | 273 | | |
274 | 274 | | |
| 275 | + | |
| 276 | + | |
| 277 | + | |
275 | 278 | | |
276 | 279 | | |
277 | 280 | | |
278 | 281 | | |
279 | 282 | | |
280 | 283 | | |
281 | | - | |
| 284 | + | |
| 285 | + | |
282 | 286 | | |
283 | 287 | | |
284 | 288 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
947 | 947 | | |
948 | 948 | | |
949 | 949 | | |
950 | | - | |
951 | 950 | | |
952 | 951 | | |
953 | 952 | | |
| |||
983 | 982 | | |
984 | 983 | | |
985 | 984 | | |
| 985 | + | |
986 | 986 | | |
987 | 987 | | |
| 988 | + | |
| 989 | + | |
988 | 990 | | |
| 991 | + | |
| 992 | + | |
989 | 993 | | |
990 | 994 | | |
991 | 995 | | |
| |||
1008 | 1012 | | |
1009 | 1013 | | |
1010 | 1014 | | |
1011 | | - | |
1012 | 1015 | | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
1013 | 1020 | | |
1014 | 1021 | | |
1015 | 1022 | | |
| |||
1301 | 1308 | | |
1302 | 1309 | | |
1303 | 1310 | | |
| 1311 | + | |
1304 | 1312 | | |
1305 | 1313 | | |
1306 | 1314 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
| |||
103 | 104 | | |
104 | 105 | | |
105 | 106 | | |
106 | | - | |
| 107 | + | |
107 | 108 | | |
108 | 109 | | |
109 | 110 | | |
| |||
159 | 160 | | |
160 | 161 | | |
161 | 162 | | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
162 | 166 | | |
163 | 167 | | |
164 | | - | |
| 168 | + | |
165 | 169 | | |
166 | 170 | | |
167 | 171 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
271 | 271 | | |
272 | 272 | | |
273 | 273 | | |
274 | | - | |
275 | 274 | | |
| 275 | + | |
| 276 | + | |
276 | 277 | | |
277 | 278 | | |
278 | 279 | | |
| |||
582 | 583 | | |
583 | 584 | | |
584 | 585 | | |
585 | | - | |
586 | 586 | | |
587 | 587 | | |
588 | 588 | | |
| |||
594 | 594 | | |
595 | 595 | | |
596 | 596 | | |
| 597 | + | |
597 | 598 | | |
598 | 599 | | |
| 600 | + | |
| 601 | + | |
599 | 602 | | |
| 603 | + | |
| 604 | + | |
600 | 605 | | |
601 | 606 | | |
602 | 607 | | |
| |||
613 | 618 | | |
614 | 619 | | |
615 | 620 | | |
616 | | - | |
617 | 621 | | |
| 622 | + | |
| 623 | + | |
| 624 | + | |
| 625 | + | |
618 | 626 | | |
619 | 627 | | |
620 | 628 | | |
| |||
863 | 871 | | |
864 | 872 | | |
865 | 873 | | |
| 874 | + | |
866 | 875 | | |
867 | 876 | | |
868 | 877 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
853 | 853 | | |
854 | 854 | | |
855 | 855 | | |
| 856 | + | |
856 | 857 | | |
857 | 858 | | |
858 | 859 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
18 | 18 | | |
19 | 19 | | |
20 | 20 | | |
| 21 | + | |
21 | 22 | | |
22 | 23 | | |
23 | 24 | | |
| |||
119 | 120 | | |
120 | 121 | | |
121 | 122 | | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
122 | 135 | | |
123 | 136 | | |
124 | 137 | | |
| |||
147 | 160 | | |
148 | 161 | | |
149 | 162 | | |
| 163 | + | |
150 | 164 | | |
151 | 165 | | |
152 | 166 | | |
| |||
157 | 171 | | |
158 | 172 | | |
159 | 173 | | |
160 | | - | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
161 | 178 | | |
162 | 179 | | |
163 | 180 | | |
| |||
188 | 205 | | |
189 | 206 | | |
190 | 207 | | |
| 208 | + | |
191 | 209 | | |
192 | 210 | | |
193 | 211 | | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
194 | 266 | | |
195 | 267 | | |
196 | 268 | | |
| |||
309 | 381 | | |
310 | 382 | | |
311 | 383 | | |
| 384 | + | |
| 385 | + | |
| 386 | + | |
| 387 | + | |
| 388 | + | |
| 389 | + | |
| 390 | + | |
| 391 | + | |
| 392 | + | |
| 393 | + | |
| 394 | + | |
312 | 395 | | |
313 | 396 | | |
314 | 397 | | |
| |||
823 | 906 | | |
824 | 907 | | |
825 | 908 | | |
| 909 | + | |
826 | 910 | | |
827 | 911 | | |
828 | 912 | | |
| |||
0 commit comments