diff --git a/source/causal-consistency/causal-consistency.md b/source/causal-consistency/causal-consistency.md index 026619b7fb..c0e4e81f3b 100644 --- a/source/causal-consistency/causal-consistency.md +++ b/source/causal-consistency/causal-consistency.md @@ -101,7 +101,7 @@ options = new SessionOptions(causalConsistency = true); session = client.startSession(options); ``` -All read operations performed using this session will now be causally consistent. +All read and write operations performed using this session will now be causally consistent. If no value is provided for `causalConsistency` and snapshot reads are not requested a value of true is implied. See the `causalConsistency` section. @@ -125,7 +125,7 @@ class SessionOptions { In order to support causal consistency a new property named `causalConsistency` is added to `SessionOptions`. Applications set `causalConsistency` when starting a client session to indicate whether they want causal consistency. -All read operations performed using that client session are then causally consistent. +All read and write operations performed using that client session are then causally consistent. Each new member is documented below. @@ -200,7 +200,7 @@ There are no new server commands related to causal consistency. Instead, causal The server reports the `operationTime` whether the operation succeeded or not and drivers MUST save the `operationTime` in the `ClientSession` whether the operation succeeded or not. 2. Passing that `operationTime` in the `afterClusterTime` field of the `readConcern` field for subsequent causally - consistent read operations (for all commands that support a `readConcern`) + consistent read and write operations (for all commands that support a `readConcern`) 3. Gossiping clusterTime (described in the Driver Session Specification) ## Server Command Responses @@ -218,8 +218,8 @@ and write operations). ``` The `operationTime` MUST be stored in the `ClientSession` to later be passed as the `afterClusterTime` field of the -`readConcern` field in subsequent read operations. The `operationTime` is returned whether the command succeeded or not -and MUST be stored in either case. +`readConcern` field in subsequent causally consistent read and write operations. The `operationTime` is returned whether +the command succeeded or not and MUST be stored in either case. Drivers MUST examine all responses from the server for the presence of an `operationTime` field and store the value in the `ClientSession`. @@ -230,14 +230,14 @@ standalone node are causally consistent automatically because there is only one When connected to a deployment that supports cluster times the command response also includes a field called `$clusterTime` that drivers MUST use to gossip the cluster time. See the Sessions Specification for details. -## Causally consistent read commands +## Causally consistent read and write commands For causal consistency the driver MUST send the `operationTime` saved in the `ClientSession` as the value of the -`afterClusterTime` field of the `readConcern` field: +`afterClusterTime` field of the `readConcern` field for read and write commands: ```typescript { - find : , // or other read command + find : , // or other read or write command ... // the rest of the command parameters readConcern : { @@ -247,7 +247,7 @@ For causal consistency the driver MUST send the `operationTime` saved in the `Cl } ``` -For the lists of commands that support causally consistent reads, see +For the list of commands that support causally consistent reads, see the [ReadConcern](../read-write-concern/read-write-concern.md#read-concern) spec. The driver MUST merge the `ReadConcern` specified for the operation with the `operationTime` from the `ClientSession` @@ -259,15 +259,16 @@ level does not support causal consistency. The Read and Write Concern specification states that when a user has not specified a `ReadConcern` or has specified the server's default `ReadConcern`, drivers MUST omit the `ReadConcern` parameter when sending the command. For causally -consistent reads this requirement is modified to state that when the `ReadConcern` parameter would normally be omitted -drivers MUST send a `ReadConcern` after all because that is how the `afterClusterTime` value is sent to the server. +consistent reads and writes this requirement is modified to state that when the `ReadConcern` parameter would normally +be omitted drivers MUST send a `ReadConcern` after all because that is how the `afterClusterTime` value is sent to the +server. The Read and Write Concern Specification states that drivers MUST NOT add a `readConcern` field to commands that are run using a generic `runCommand` method. The same is true for causal consistency, so commands that are run using `runCommand` MUST NOT have an `afterClusterTime` field added to them. -When executing a causally consistent read, the `afterClusterTime` field MUST be sent when connected to a deployment that -supports cluster times, and MUST NOT be sent when connected to a deployment that does not support cluster times. +When executing a causally consistent operation, the `afterClusterTime` field MUST be sent when connected to a deployment +that supports cluster times, and MUST NOT be sent when connected to a deployment that does not support cluster times. ## Unacknowledged writes @@ -276,7 +277,7 @@ a write. Since unacknowledged writes don't receive a response from the server (o `ClientSession`'s `operationTime` is not updated after an unacknowledged write. That means that a causally consistent read after an unacknowledged write cannot be causally consistent with the unacknowledged write. Rather than prohibiting unacknowledged writes in a causally consistent session we have decided to accept this limitation. Drivers MUST document -that causally consistent reads are not causally consistent with unacknowledged writes. +that causally consistent operations are not causally consistent with unacknowledged writes. ## Test Plan @@ -403,6 +404,9 @@ resolving many discussions of spec details. A final reference implementation mus ## Changelog +- 2026-05-04: Require `afterClusterTime` on all write commands in causally-consistent sessions, not only on read + commands. + - 2024-02-08: Migrated from reStructuredText to Markdown. - 2022-11-11: Require `causalConsistency=false` for implicit sessions. diff --git a/source/causal-consistency/tests/causal-consistency-clientBulkWrite.json b/source/causal-consistency/tests/causal-consistency-clientBulkWrite.json new file mode 100644 index 0000000000..8dd201d191 --- /dev/null +++ b/source/causal-consistency/tests/causal-consistency-clientBulkWrite.json @@ -0,0 +1,151 @@ +{ + "description": "causal consistency write commands include afterClusterTime", + "schemaVersion": "1.3", + "runOnRequirements": [ + { + "minServerVersion": "8.0", + "topologies": [ + "replicaset", + "sharded", + "load-balanced" + ] + } + ], + "createEntities": [ + { + "client": { + "id": "client0", + "useMultipleMongoses": false, + "uriOptions": { + "retryWrites": false + }, + "observeEvents": [ + "commandStartedEvent" + ] + } + }, + { + "database": { + "id": "database0", + "client": "client0", + "databaseName": "causal-consistency-tests" + } + }, + { + "collection": { + "id": "collection0", + "database": "database0", + "collectionName": "test" + } + }, + { + "session": { + "id": "session0", + "client": "client0", + "sessionOptions": { + "causalConsistency": true + } + } + } + ], + "initialData": [ + { + "collectionName": "test", + "databaseName": "causal-consistency-tests", + "documents": [ + { + "_id": 1, + "x": 11 + }, + { + "_id": 2, + "x": 22 + }, + { + "_id": 3, + "x": 33 + } + ] + } + ], + "tests": [ + { + "description": "clientBulkWrite includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "clientBulkWrite", + "object": "client0", + "arguments": { + "session": "session0", + "models": [ + { + "insertOne": { + "namespace": "causal-consistency-tests.test", + "document": { + "_id": 4 + } + } + } + ] + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "bulkWrite", + "command": { + "bulkWrite": 1, + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + } + ] +} diff --git a/source/causal-consistency/tests/causal-consistency-clientBulkWrite.yml b/source/causal-consistency/tests/causal-consistency-clientBulkWrite.yml new file mode 100644 index 0000000000..d1902e188e --- /dev/null +++ b/source/causal-consistency/tests/causal-consistency-clientBulkWrite.yml @@ -0,0 +1,75 @@ +description: "causal consistency bulkWrite include afterClusterTime" + +schemaVersion: "1.3" + +runOnRequirements: + - minServerVersion: "8.0" + topologies: [replicaset, sharded, load-balanced] + +createEntities: + - client: + id: &client0 client0 + useMultipleMongoses: false + uriOptions: + retryWrites: false + observeEvents: [commandStartedEvent] + - database: + id: &database0 database0 + client: *client0 + databaseName: &databaseName causal-consistency-tests + - collection: + id: &collection0 collection0 + database: *database0 + collectionName: &collectionName test + - session: + id: &session0 session0 + client: *client0 + sessionOptions: + causalConsistency: true + +initialData: + - collectionName: *collectionName + databaseName: *databaseName + documents: + - { _id: 1, x: 11 } + - { _id: 2, x: 22 } + - { _id: 3, x: 33 } + +# In a causally consistent session, once an operationTime has been established by a prior +# operation, subsequent write commands MUST include readConcern.afterClusterTime so the +# server can apply the write causally after the previously-observed data. + +tests: + - description: "clientBulkWrite includes afterClusterTime in causally consistent session" + operations: + - name: find + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + expectResult: [{ _id: 1, x: 11 }] + - name: clientBulkWrite + object: *client0 + arguments: + session: *session0 + models: + - insertOne: + namespace: causal-consistency-tests.test + document: { _id: 4 } + expectEvents: + - client: *client0 + events: + - commandStartedEvent: + commandName: find + command: + find: *collectionName + readConcern: { $$exists: false } + lsid: { $$sessionLsid: *session0 } + - commandStartedEvent: + commandName: bulkWrite + command: + bulkWrite: 1 + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } diff --git a/source/causal-consistency/tests/causal-consistency-write-commands.json b/source/causal-consistency/tests/causal-consistency-write-commands.json new file mode 100644 index 0000000000..30460f5912 --- /dev/null +++ b/source/causal-consistency/tests/causal-consistency-write-commands.json @@ -0,0 +1,1283 @@ +{ + "description": "causal consistency write commands include afterClusterTime", + "schemaVersion": "1.3", + "runOnRequirements": [ + { + "topologies": [ + "replicaset", + "sharded", + "load-balanced" + ] + } + ], + "createEntities": [ + { + "client": { + "id": "client0", + "useMultipleMongoses": false, + "uriOptions": { + "retryWrites": false + }, + "observeEvents": [ + "commandStartedEvent" + ] + } + }, + { + "database": { + "id": "database0", + "client": "client0", + "databaseName": "causal-consistency-tests" + } + }, + { + "collection": { + "id": "collection0", + "database": "database0", + "collectionName": "test" + } + }, + { + "session": { + "id": "session0", + "client": "client0", + "sessionOptions": { + "causalConsistency": true + } + } + } + ], + "initialData": [ + { + "collectionName": "test", + "databaseName": "causal-consistency-tests", + "documents": [ + { + "_id": 1, + "x": 11 + }, + { + "_id": 2, + "x": 22 + }, + { + "_id": 3, + "x": 33 + } + ] + } + ], + "tests": [ + { + "description": "insertOne includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "insertOne", + "object": "collection0", + "arguments": { + "session": "session0", + "document": { + "_id": 4 + } + }, + "expectResult": { + "$$unsetOrMatches": { + "insertedId": { + "$$unsetOrMatches": 4 + } + } + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "insert", + "command": { + "insert": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "insertMany includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "insertMany", + "object": "collection0", + "arguments": { + "session": "session0", + "documents": [ + { + "_id": 4 + }, + { + "_id": 5 + } + ] + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "insert", + "command": { + "insert": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "updateOne includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "updateOne", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + }, + "update": { + "$set": { + "x": 100 + } + } + }, + "expectResult": { + "matchedCount": 1, + "modifiedCount": 1, + "upsertedCount": 0 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "update", + "command": { + "update": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "updateMany includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "updateMany", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": { + "$gt": 0 + } + }, + "update": { + "$set": { + "updated": true + } + } + }, + "expectResult": { + "matchedCount": 3, + "modifiedCount": 3, + "upsertedCount": 0 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "update", + "command": { + "update": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "replaceOne includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "replaceOne", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + }, + "replacement": { + "x": 100 + } + }, + "expectResult": { + "matchedCount": 1, + "modifiedCount": 1, + "upsertedCount": 0 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "update", + "command": { + "update": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "deleteOne includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "deleteOne", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": { + "deletedCount": 1 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "delete", + "command": { + "delete": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "deleteMany includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "deleteMany", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": { + "$gt": 0 + } + } + }, + "expectResult": { + "deletedCount": 3 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "delete", + "command": { + "delete": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "findOneAndUpdate includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "findOneAndUpdate", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + }, + "update": { + "$set": { + "x": 100 + } + } + }, + "expectResult": { + "_id": 1, + "x": 11 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "findAndModify", + "command": { + "findAndModify": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "findOneAndDelete includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "findOneAndDelete", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": { + "_id": 1, + "x": 11 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "findAndModify", + "command": { + "findAndModify": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "findOneAndReplace includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "findOneAndReplace", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + }, + "replacement": { + "x": 100 + } + }, + "expectResult": { + "_id": 1, + "x": 11 + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "findAndModify", + "command": { + "findAndModify": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "bulkWrite includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "bulkWrite", + "object": "collection0", + "arguments": { + "session": "session0", + "requests": [ + { + "insertOne": { + "document": { + "_id": 4 + } + } + }, + { + "updateOne": { + "filter": { + "_id": 2 + }, + "update": { + "$set": { + "x": 100 + } + } + } + }, + { + "deleteOne": { + "filter": { + "_id": 3 + } + } + } + ] + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "insert", + "command": { + "insert": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "update", + "command": { + "update": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "delete", + "command": { + "delete": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "create includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "dropCollection", + "object": "database0", + "arguments": { + "session": "session0", + "collection": "causal-consistency-createCollection-test" + } + }, + { + "name": "createCollection", + "object": "database0", + "arguments": { + "session": "session0", + "collection": "causal-consistency-createCollection-test" + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "drop", + "command": { + "drop": "causal-consistency-createCollection-test", + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "create", + "command": { + "create": "causal-consistency-createCollection-test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "createIndexes includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "createIndex", + "object": "collection0", + "arguments": { + "session": "session0", + "keys": { + "x": 1 + }, + "name": "x_1" + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "createIndexes", + "command": { + "createIndexes": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "drop includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "dropCollection", + "object": "database0", + "arguments": { + "session": "session0", + "collection": "test" + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "drop", + "command": { + "drop": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + }, + { + "description": "dropIndexes includes afterClusterTime in causally consistent session", + "operations": [ + { + "name": "find", + "object": "collection0", + "arguments": { + "session": "session0", + "filter": { + "_id": 1 + } + }, + "expectResult": [ + { + "_id": 1, + "x": 11 + } + ] + }, + { + "name": "dropIndexes", + "object": "collection0", + "arguments": { + "session": "session0" + } + } + ], + "expectEvents": [ + { + "client": "client0", + "events": [ + { + "commandStartedEvent": { + "commandName": "find", + "command": { + "find": "test", + "readConcern": { + "$$exists": false + }, + "lsid": { + "$$sessionLsid": "session0" + } + } + } + }, + { + "commandStartedEvent": { + "commandName": "dropIndexes", + "command": { + "dropIndexes": "test", + "lsid": { + "$$sessionLsid": "session0" + }, + "readConcern": { + "afterClusterTime": { + "$$exists": true + }, + "level": { + "$$exists": false + } + } + } + } + } + ] + } + ] + } + ] +} diff --git a/source/causal-consistency/tests/causal-consistency-write-commands.yml b/source/causal-consistency/tests/causal-consistency-write-commands.yml new file mode 100644 index 0000000000..ffe2d23c4e --- /dev/null +++ b/source/causal-consistency/tests/causal-consistency-write-commands.yml @@ -0,0 +1,430 @@ +description: "causal consistency write commands include afterClusterTime" + +schemaVersion: "1.3" + +runOnRequirements: + - topologies: [replicaset, sharded, load-balanced] + +createEntities: + - client: + id: &client0 client0 + useMultipleMongoses: false + uriOptions: + retryWrites: false + observeEvents: [commandStartedEvent] + - database: + id: &database0 database0 + client: *client0 + databaseName: &databaseName causal-consistency-tests + - collection: + id: &collection0 collection0 + database: *database0 + collectionName: &collectionName test + - session: + id: &session0 session0 + client: *client0 + sessionOptions: + causalConsistency: true + +initialData: + - collectionName: *collectionName + databaseName: *databaseName + documents: + - { _id: 1, x: 11 } + - { _id: 2, x: 22 } + - { _id: 3, x: 33 } + +# In a causally consistent session, once an operationTime has been established by a prior +# operation, subsequent write commands MUST include readConcern.afterClusterTime so the +# server can apply the write causally after the previously-observed data. + +tests: + - description: "insertOne includes afterClusterTime in causally consistent session" + operations: + - &find + name: find + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + expectResult: [{ _id: 1, x: 11 }] + - name: insertOne + object: *collection0 + arguments: + session: *session0 + document: { _id: 4 } + expectResult: + $$unsetOrMatches: { insertedId: { $$unsetOrMatches: 4 } } + expectEvents: + - client: *client0 + events: + - &findEvent + commandStartedEvent: + commandName: find + command: + find: *collectionName + readConcern: { $$exists: false } + lsid: { $$sessionLsid: *session0 } + - commandStartedEvent: + commandName: insert + command: + insert: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "insertMany includes afterClusterTime in causally consistent session" + operations: + - *find + - name: insertMany + object: *collection0 + arguments: + session: *session0 + documents: + - { _id: 4 } + - { _id: 5 } + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: insert + command: + insert: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "updateOne includes afterClusterTime in causally consistent session" + operations: + - *find + - name: updateOne + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + update: { $set: { x: 100 } } + expectResult: + matchedCount: 1 + modifiedCount: 1 + upsertedCount: 0 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: update + command: + update: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "updateMany includes afterClusterTime in causally consistent session" + operations: + - *find + - name: updateMany + object: *collection0 + arguments: + session: *session0 + filter: { _id: { $gt: 0 } } + update: { $set: { updated: true } } + expectResult: + matchedCount: 3 + modifiedCount: 3 + upsertedCount: 0 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: update + command: + update: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "replaceOne includes afterClusterTime in causally consistent session" + operations: + - *find + - name: replaceOne + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + replacement: { x: 100 } + expectResult: + matchedCount: 1 + modifiedCount: 1 + upsertedCount: 0 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: update + command: + update: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "deleteOne includes afterClusterTime in causally consistent session" + operations: + - *find + - name: deleteOne + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + expectResult: + deletedCount: 1 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: delete + command: + delete: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "deleteMany includes afterClusterTime in causally consistent session" + operations: + - *find + - name: deleteMany + object: *collection0 + arguments: + session: *session0 + filter: { _id: { $gt: 0 } } + expectResult: + deletedCount: 3 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: delete + command: + delete: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "findOneAndUpdate includes afterClusterTime in causally consistent session" + operations: + - *find + - name: findOneAndUpdate + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + update: { $set: { x: 100 } } + expectResult: { _id: 1, x: 11 } + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: findAndModify + command: + findAndModify: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "findOneAndDelete includes afterClusterTime in causally consistent session" + operations: + - *find + - name: findOneAndDelete + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + expectResult: { _id: 1, x: 11 } + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: findAndModify + command: + findAndModify: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "findOneAndReplace includes afterClusterTime in causally consistent session" + operations: + - *find + - name: findOneAndReplace + object: *collection0 + arguments: + session: *session0 + filter: { _id: 1 } + replacement: { x: 100 } + expectResult: { _id: 1, x: 11 } + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: findAndModify + command: + findAndModify: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "bulkWrite includes afterClusterTime in causally consistent session" + operations: + - *find + - name: bulkWrite + object: *collection0 + arguments: + session: *session0 + requests: + - insertOne: + document: { _id: 4 } + - updateOne: + filter: { _id: 2 } + update: { $set: { x: 100 } } + - deleteOne: + filter: { _id: 3 } + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: insert + command: + insert: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + - commandStartedEvent: + commandName: update + command: + update: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + - commandStartedEvent: + commandName: delete + command: + delete: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "create includes afterClusterTime in causally consistent session" + operations: + - *find + # Drop the collection first to make sure there's no name conflict during + # createCollection. + - name: dropCollection + object: *database0 + arguments: + session: *session0 + collection: &newCollectionName causal-consistency-createCollection-test + - name: createCollection + object: *database0 + arguments: + session: *session0 + collection: *newCollectionName + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: drop + command: + drop: *newCollectionName + lsid: { $$sessionLsid: *session0 } + - commandStartedEvent: + commandName: create + command: + create: *newCollectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "createIndexes includes afterClusterTime in causally consistent session" + operations: + - *find + - name: createIndex + object: *collection0 + arguments: + session: *session0 + keys: { x: 1 } + name: x_1 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: createIndexes + command: + createIndexes: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "drop includes afterClusterTime in causally consistent session" + operations: + - *find + - name: dropCollection + object: *database0 + arguments: + session: *session0 + collection: *collectionName + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: drop + command: + drop: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } + + - description: "dropIndexes includes afterClusterTime in causally consistent session" + operations: + - *find + - name: dropIndexes + object: *collection0 + arguments: + session: *session0 + expectEvents: + - client: *client0 + events: + - *findEvent + - commandStartedEvent: + commandName: dropIndexes + command: + dropIndexes: *collectionName + lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } + level: { $$exists: false } diff --git a/source/read-write-concern/read-write-concern.md b/source/read-write-concern/read-write-concern.md index ad779bf2c4..4455b5fa5b 100644 --- a/source/read-write-concern/read-write-concern.md +++ b/source/read-write-concern/read-write-concern.md @@ -68,7 +68,7 @@ class ReadConcern { } ``` -The read concern option is available for the following operations: +The read concern option is available for most operations, including the following: - `aggregate` command - `count` command @@ -78,6 +78,16 @@ The read concern option is available for the following operations: - `parallelCollectionScan` command - `geoNear` command - `geoSearch` command +- `insert` command +- `update` command +- `findAndModify` command +- `delete` command +- `bulkWrite` command +- `create` command +- `createIndexes` command +- `drop` command +- `dropDatabase` command +- `dropIndexes` command Starting in MongoDB 4.2, an `aggregate` command with a write stage (e.g. `$out`, `$merge`) supports a `readConcern`; however, it does not support the "linearizable" level (attempting to do so will result in a server error). diff --git a/source/transactions-convenient-api/tests/unified/callback-aborts.json b/source/transactions-convenient-api/tests/unified/callback-aborts.json index 206428715c..dc8f7fb5a4 100644 --- a/source/transactions-convenient-api/tests/unified/callback-aborts.json +++ b/source/transactions-convenient-api/tests/unified/callback-aborts.json @@ -308,10 +308,12 @@ "lsid": { "$$sessionLsid": "session0" }, - "autocommit": { - "$$exists": false - }, "readConcern": { + "afterClusterTime": { + "$$exists": true + } + }, + "autocommit": { "$$exists": false }, "startTransaction": { diff --git a/source/transactions-convenient-api/tests/unified/callback-aborts.yml b/source/transactions-convenient-api/tests/unified/callback-aborts.yml index 9414040eca..d5a95d7776 100644 --- a/source/transactions-convenient-api/tests/unified/callback-aborts.yml +++ b/source/transactions-convenient-api/tests/unified/callback-aborts.yml @@ -164,9 +164,10 @@ tests: - { _id: 2 } ordered: true lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } # omitted fields autocommit: { $$exists: false } - readConcern: { $$exists: false } startTransaction: { $$exists: false } writeConcern: { $$exists: false } commandName: insert diff --git a/source/transactions-convenient-api/tests/unified/callback-commits.json b/source/transactions-convenient-api/tests/unified/callback-commits.json index 06f791e9ae..edda386418 100644 --- a/source/transactions-convenient-api/tests/unified/callback-commits.json +++ b/source/transactions-convenient-api/tests/unified/callback-commits.json @@ -381,10 +381,12 @@ "lsid": { "$$sessionLsid": "session0" }, - "autocommit": { - "$$exists": false - }, "readConcern": { + "afterClusterTime": { + "$$exists": true + } + }, + "autocommit": { "$$exists": false }, "startTransaction": { diff --git a/source/transactions-convenient-api/tests/unified/callback-commits.yml b/source/transactions-convenient-api/tests/unified/callback-commits.yml index b5cbb04151..7bd3e7feae 100644 --- a/source/transactions-convenient-api/tests/unified/callback-commits.yml +++ b/source/transactions-convenient-api/tests/unified/callback-commits.yml @@ -193,9 +193,10 @@ tests: - { _id: 3 } ordered: true lsid: { $$sessionLsid: *session0 } + readConcern: + afterClusterTime: { $$exists: true } # omitted fields autocommit: { $$exists: false } - readConcern: { $$exists: false } startTransaction: { $$exists: false } writeConcern: { $$exists: false } commandName: insert diff --git a/source/transactions/tests/unified/commit.json b/source/transactions/tests/unified/commit.json index ab778d8df2..f033906940 100644 --- a/source/transactions/tests/unified/commit.json +++ b/source/transactions/tests/unified/commit.json @@ -1040,7 +1040,9 @@ ], "ordered": true, "readConcern": { - "$$exists": false + "afterClusterTime": { + "$$exists": true + } }, "lsid": { "$$sessionLsid": "session1" @@ -1196,7 +1198,9 @@ ], "ordered": true, "readConcern": { - "$$exists": false + "afterClusterTime": { + "$$exists": true + } }, "lsid": { "$$sessionLsid": "session1" diff --git a/source/transactions/tests/unified/commit.yml b/source/transactions/tests/unified/commit.yml index d9af084894..eaf4ee39ba 100644 --- a/source/transactions/tests/unified/commit.yml +++ b/source/transactions/tests/unified/commit.yml @@ -615,7 +615,8 @@ tests: documents: - { _id: 2 } ordered: true - readConcern: { $$exists: false } + readConcern: + afterClusterTime: { $$exists: true } lsid: { $$sessionLsid: *session1 } txnNumber: { $$exists: false } startTransaction: { $$exists: false } @@ -698,7 +699,8 @@ tests: documents: - { _id: 2 } ordered: true - readConcern: { $$exists: false } + readConcern: + afterClusterTime: { $$exists: true } lsid: { $$sessionLsid: *session1 } txnNumber: { $$exists: false } startTransaction: { $$exists: false } diff --git a/source/transactions/tests/unified/retryable-writes.json b/source/transactions/tests/unified/retryable-writes.json index c196e68622..21ef22b8ab 100644 --- a/source/transactions/tests/unified/retryable-writes.json +++ b/source/transactions/tests/unified/retryable-writes.json @@ -217,7 +217,9 @@ ], "ordered": true, "readConcern": { - "$$exists": false + "afterClusterTime": { + "$$exists": true + } }, "lsid": { "$$sessionLsid": "session0" @@ -306,7 +308,9 @@ ], "ordered": true, "readConcern": { - "$$exists": false + "afterClusterTime": { + "$$exists": true + } }, "lsid": { "$$sessionLsid": "session0" diff --git a/source/transactions/tests/unified/retryable-writes.yml b/source/transactions/tests/unified/retryable-writes.yml index aa9c037d41..81339c9ad9 100644 --- a/source/transactions/tests/unified/retryable-writes.yml +++ b/source/transactions/tests/unified/retryable-writes.yml @@ -132,7 +132,8 @@ tests: documents: - { _id: 2 } ordered: true - readConcern: { $$exists: false } + readConcern: + afterClusterTime: { $$exists: true } lsid: { $$sessionLsid: *session0 } txnNumber: { $numberLong: '2' } startTransaction: { $$exists: false } @@ -175,7 +176,8 @@ tests: - { _id: 4 } - { _id: 5 } ordered: true - readConcern: { $$exists: false } + readConcern: + afterClusterTime: { $$exists: true } lsid: { $$sessionLsid: *session0 } txnNumber: { $numberLong: '4' } startTransaction: { $$exists: false } diff --git a/source/unified-test-format/tests/Makefile b/source/unified-test-format/tests/Makefile index 4893f267ac..dda5a8ec3e 100644 --- a/source/unified-test-format/tests/Makefile +++ b/source/unified-test-format/tests/Makefile @@ -2,6 +2,7 @@ SCHEMA=../schema-latest.json .PHONY: all \ auth \ + causal-consistency \ change-streams \ client-side-encryption \ client-side-operations-timeout \ @@ -33,6 +34,7 @@ SCHEMA=../schema-latest.json HAS_AJV all: auth \ + causal-consistency \ change-streams \ client-side-encryption \ client-side-operations-timeout \ @@ -62,6 +64,9 @@ all: auth \ auth: HAS_AJV @ajv --spec=draft2019 test -s $(SCHEMA) -d "../../auth/tests/unified/*.yml" --valid +causal-consistency: HAS_AJV + @ajv --spec=draft2019 test -s $(SCHEMA) -d "../../causal-consistency/tests/*.yml" --valid + change-streams: HAS_AJV @ajv --spec=draft2019 test -s $(SCHEMA) -d "../../change-streams/tests/unified/*.yml" --valid