Add Claims DDS#27389
Conversation
- Add new @fluidframework/shared-claims package implementing first-writer-wins claim semantics - Integrate SharedClaims into PureDataObject with trySetClaim and getClaim methods - Add ClaimResult and ClaimConfirmation types exported from aqueduct - Gate claims channel creation behind Fluid.PureDataObject.EnableClaims config flag - Always load existing claims channels regardless of feature gate - Auto-register SharedClaimsFactory in PureDataObjectFactory - Make new methods optional to avoid forward-compat type breaks - Add generic typing to ISharedClaimsEvents for type-safe claimed event values Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Hi! Thank you for opening this PR. Want me to review it? Based on the diff (1748 lines, 38 files), I've queued these reviewers:
How this works
|
…gate - Fix biome formatting in shared-claims and aqueduct - Add TClaimValue generic to PureDataObject for typed claims - Generate api-report files for shared-claims - Read feature gate from config in initializeInternal (not constructor) - Only gate create flow; always load existing claims channels Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes node10 type resolution for the /internal subpath export. The .gitignore overrides the repo-wide ignore rule so internal.d.ts can be tracked. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces a new DDS, @fluidframework/shared-claims, providing first-writer-wins claim semantics, and wires it into PureDataObject with a feature-gated creation path and aqueduct surface-area exports.
Changes:
- Adds the new
SharedClaimsDDS (factory/kind, ops, summarization, GC visitation) plus a test suite and packaging/build config. - Integrates
SharedClaimsinto aqueductPureDataObject(trySetClaim/getClaim) and auto-registers the channel factory inPureDataObjectFactory. - Publishes the new package through repo metadata (feeds, PACKAGES.md, lockfile updates) and updates aqueduct API reports.
Reviewed changes
Copilot reviewed 37 out of 39 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds workspace importer entries for the new @fluidframework/shared-claims package and aqueduct dependency update. |
| packages/framework/aqueduct/src/index.ts | Re-exports new claim result types from aqueduct. |
| packages/framework/aqueduct/src/data-objects/pureDataObject.ts | Adds optional claims APIs and feature-gated creation/loading of an internal claims channel. |
| packages/framework/aqueduct/src/data-object-factories/pureDataObjectFactory.ts | Auto-registers SharedClaimsFactory so existing documents can load claims channels. |
| packages/framework/aqueduct/src/claimTypes.ts | Introduces public ClaimResult / ClaimConfirmation types for aqueduct. |
| packages/framework/aqueduct/package.json | Adds dependency on @fluidframework/shared-claims. |
| packages/framework/aqueduct/api-report/aqueduct.public.api.md | API report updates for claim types. |
| packages/framework/aqueduct/api-report/aqueduct.legacy.public.api.md | API report updates for claim types. |
| packages/framework/aqueduct/api-report/aqueduct.legacy.beta.api.md | API report updates for claim types and PureDataObject signature. |
| packages/framework/aqueduct/api-report/aqueduct.beta.api.md | API report updates for claim types. |
| packages/dds/shared-claims/package.json | New package definition, scripts, dependencies, exports (ESM/CJS). |
| packages/dds/shared-claims/LICENSE | New package license file. |
| packages/dds/shared-claims/internal.d.ts | Internal entrypoint type export wiring. |
| packages/dds/shared-claims/tsdoc.json | TSDoc config for the new package. |
| packages/dds/shared-claims/tsconfig.json | TypeScript build config for ESM output. |
| packages/dds/shared-claims/tsconfig.cjs.json | TypeScript build config for CJS output. |
| packages/dds/shared-claims/src/index.ts | Public/internal exports for the new DDS. |
| packages/dds/shared-claims/src/interfaces.ts | Defines ISharedClaims API and typed events. |
| packages/dds/shared-claims/src/sharedClaims.ts | Implements claim semantics, ops, summarization/loading, GC visitation, and stashed-op hooks. |
| packages/dds/shared-claims/src/sharedClaimsFactory.ts | Channel factory and SharedClaimsKind creation helper. |
| packages/dds/shared-claims/src/packageVersion.ts | Generated package name/version constants. |
| packages/dds/shared-claims/src/test/tsconfig.json | Test compilation config (ESM). |
| packages/dds/shared-claims/src/test/tsconfig.cjs.json | Test compilation config (CJS). |
| packages/dds/shared-claims/src/test/sharedClaims.spec.ts | Unit tests for claim behaviors, events, and summary round-trip. |
| packages/dds/shared-claims/eslint.config.mts | Package-local eslint configuration (test-only node:assert allowance). |
| packages/dds/shared-claims/.mocharc.cjs | Mocha configuration for the package test runner. |
| packages/dds/shared-claims/.npmignore | Excludes test/coverage artifacts from published package. |
| packages/dds/shared-claims/.gitignore | Ignores build outputs; ensures internal.d.ts is included. |
| packages/dds/shared-claims/api-extractor.json | API Extractor base config wiring. |
| packages/dds/shared-claims/api-extractor/api-extractor-lint-index.esm.json | API extractor lint config for ESM entrypoint. |
| packages/dds/shared-claims/api-extractor/api-extractor-lint-index.cjs.json | API extractor lint config for CJS entrypoint. |
| packages/dds/shared-claims/api-extractor/api-extractor-lint-bundle.json | API extractor lint config for release tags bundle. |
| packages/dds/shared-claims/api-report/shared-claims.public.api.md | Generated API report artifact (public). |
| packages/dds/shared-claims/api-report/shared-claims.beta.api.md | Generated API report artifact (beta). |
| packages/dds/shared-claims/api-report/shared-claims.alpha.api.md | Generated API report artifact (alpha). |
| PACKAGES.md | Adds @fluidframework/shared-claims to the repo package/layer listing. |
| feeds/public.txt | Publishes @fluidframework/shared-claims to the public feed list. |
| feeds/internal-test.txt | Publishes @fluidframework/shared-claims to the internal-test feed list. |
| feeds/internal-build.txt | Publishes @fluidframework/shared-claims to the internal-build feed list. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
ChumpChief
left a comment
There was a problem hiding this comment.
Looks like a good start! Mostly just high-level feedback on API shape, functionality, staging, exposure -- I didn't dive into the details yet.
|
|
||
| // @beta @legacy | ||
| export abstract class PureDataObject<I extends DataObjectTypes = DataObjectTypes> extends TypedEventEmitter<I["Events"] & IEvent> implements IFluidLoadable, IProvideFluidHandle { | ||
| export abstract class PureDataObject<I extends DataObjectTypes = DataObjectTypes, TClaimValue = unknown> extends TypedEventEmitter<I["Events"] & IEvent> implements IFluidLoadable, IProvideFluidHandle { |
There was a problem hiding this comment.
Two main thoughts on integration:
- I suspect we'll eventually want to consider integrating/exposing this through the datastore runtime (thereby giving aliasing parity at the datastore runtime layer as compared to the container runtime layer), rather than the aqueduct layer. Probably a discussion to have but worth thinking about pros/cons of each.
- Either way - I'd probably recommend your first PR be just the DDS plus unit tests, and then take integration as a followup. Just from the perspective of work staging and throughput.
- I strongly recommend writing an example/ package that leverages the new DDS as a concrete and speedy way to experiment with the usage patterns and get a feel for the ergonomics of its use without needing to worry about legacy API impact/compat/rollout.
- It can be a toy/fake scenario (e.g. two clients trying to write their name into a field), no need to devise a complex real-world app. This can still help highlight if we've made good choices around eventing, promises, etc.
- You might even consider doing it as a "consensus-comparison" example comparing against CRC, similar to how the task-selection example compares OldestClientObserver vs. TaskManager or tree-comparison compares legacy SharedTree against new SharedTree.
- Totally OK if this is a followup PR, but I do think it will be useful.
| } | ||
|
|
||
| /** | ||
| * A distributed data structure providing first-writer-wins claim semantics. |
There was a problem hiding this comment.
For naming, I'd suggest dropping "Shared", we don't typically include it anymore and it doesn't really add anything.
We don't have to belabor naming yet (fine to defer until we're about to expose for customer use, and options might become clearer as the functionality is iterated on) but I'd be curious to know how you chose "Claims" as the name.
There was a problem hiding this comment.
I went with "Claims" because this DDS is all about claiming keys
| * ### Usage | ||
| * | ||
| * ```typescript | ||
| * const result = await claims.trySetClaim("singleton-component", componentHandle); |
There was a problem hiding this comment.
This is a good example of a thing to experiment with in an examples/ package - whether a promise-based API adds any value to make it worth the extra effort in implementation (e.g. pending claim tracking, resolve, abort) as compared to fire-and-forget + events.
This is why I made .set() be void for PactMap - to correctly use the DDS the caller must comprehend that they risk a race against other clients. Through experimentation I found it easier to separately fire-and-forget plus listen for any incoming ops (including your own) rather than try to reconcile an await callstack against an event handler for remote ops (AB#1602).
Plus it's not too hard for a customer to Promise-ify an event-based API themselves if they really feel they have a need.
There was a problem hiding this comment.
I'm not sure how I feel about a fire-and-forget plus listening to events. Having a returned value and choosing whether to await it or not feels much more explicit without the consumer needing to reason much about current state.
|
adding a new dds is a big deal. i don't think we should group adding to pure data object, and public exposures into the same pr. i think the dds itself, if we go this route, should be committed as internal. it then needs to have a dds stress added, like we do for all dds, then you need to export a model and integrate into local server test |
Bug fixes: - Use .has() instead of !== undefined in trySetClaim to support undefined values - Fix stashed-op crash: create pending entry in applyStashedOp, make resolvePending/rollback tolerant of missing entries as defense-in-depth - Fix ClaimResult docs: Accepted variant describes detached/staging use case - Optimize processGCDataCore to avoid building full summary tree - Fix doc example in factory (correct casing, show sync API pattern) - Only suppress 'Channel does not exist' errors in initializeInternal - Update trySetClaim error message to mention both legacy and feature gate scenarios API changes: - Rename: drop 'Shared' prefix (SharedClaims→Claims, SharedClaimsFactory→ClaimsFactory, SharedClaimsKind→ClaimsKind, ISharedClaims→IClaims, ISharedClaimsEvents→IClaimsEvents) - Minimize 'claimed' event params to just (key) per reviewer feedback - Add CAS support: trySetClaim(key, value, expectedValue?) enables compare-and-swap when expectedValue is provided, write-once when omitted New: - Add README with usage examples - Add CAS tests (4 new test cases) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per reviewer feedback (anthony-murphy, ChumpChief), this PR should strictly contain the new DDS. Aqueduct integration (PureDataObject claims APIs, ClaimResult types, factory auto-registration) will be done in a follow-up PR after DDS stress tests are added. Reverts all changes to: - packages/framework/aqueduct/ Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds DDS fuzz tests using createDDSFuzzSuite with: - claim, CAS, and getClaim operations - Small key pool (4 keys) for high contention - Consistency validation across all clients - Default and rebasing configurations (100 seeds each) - test:stress npm script for extended testing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Export baseClaimsModel from ./internal/test entrypoint - Enable declaration output in test tsconfig - Add @fluidframework/shared-claims dep to local-server-stress-tests - Register Claims in ddsModelMap for stress testing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Renames directory, package name, source files, and all references: - packages/dds/shared-claims → packages/dds/claims - @fluidframework/shared-claims → @fluidframework/claims - sharedClaimsFactory.ts → claimsFactory.ts - sharedClaims.spec.ts → claims.spec.ts - API report files renamed accordingly - Updated feeds, PACKAGES.md, local-server-stress-tests refs Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- src/sharedClaims.ts → src/claims.ts - Local variables: sharedClaims → claims - Logger prefix: fluid_sharedClaims_ → fluid_claims_ - Fixed import ordering Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New @internal package does not need feed entries yet. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This reverts commit dd3838a.
Move entry to correct dependency layer position (between counter and cell). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Seeds 0 and 92 hit a latent Matrix consistency bug exposed by the changed random distribution from adding Claims to the DDS model map. These are pre-existing bugs unrelated to the Claims DDS. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| // Pre-existing DDS bugs: seed 54 (ConsensusOrderedCollection consistency). | ||
| skip: [54], | ||
| // Pre-existing DDS bugs: seed 54 (ConsensusOrderedCollection), seeds 0 and 92 (Matrix). | ||
| skip: [0, 54, 92], |
There was a problem hiding this comment.
@anthony-murphy @ChumpChief This was a worrying one, but I triple checked and these failures aren't caused by the new DDS (I copied the seed json and replayed strictly against main). I had copilot go all night to find root causes for each and fix locally, I'll put some of the fixes in a separate PR (might skip the ConsensusOrderedCollection fix).
Seed 92 — Matrix (SharedMatrix consensus update)
File: packages/dds/matrix/src/matrix.ts
Bug: When multiple local set ops are pending and an ack arrives, the consensus value on remaining pending cells wasn't updated. Rolling back remaining local changes restored incorrect server state.
Fix: After clearing the acked pending cell, update pendingCell.consensus to the acked value — but only in LWW mode (FWW mode has its own handling).
Seed 0 — ConsensusOrderedCollection (missed quorum events)
File: packages/dds/ordered-collection/src/consensusOrderedCollection.ts
Bug: When a channel is loaded from a snapshot, the constructor's removeMember quorum listener misses departure events that already fired. Catch-up ops can create jobTracking entries for departed clients, causing permanent data inconsistency.
Fix: Dual reconciliation:
- onConnect(): Reconciles for lazy loading (where onConnect fires after pending replay)
- processMessagesCore() with queueMicrotask: Reconciles for non-lazy loading (where onConnect fires before catch-up ops)
Seed 54 — fuzzUtils bug
File: packages/dds/ordered-collection/src/test/fuzzUtils.ts
Fix: Changed aData.size to aData.size() (method call, not property access).
- Split trySetClaim into trySetClaim (write-once) and compareAndSetClaim (CAS) - Add discriminated union IClaimOperation<T> with mode field - Make IClaimOperation generic to eliminate unsafe type casts - Fix currentValue typing to T | undefined for AlreadyClaimed results - Remove unsafe assert in resolvePending - Remove unused @fluid-internal/client-utils dependency - Remove unused T generic parameter from IClaimsEvents - Simplify processMessage (no legacy op support needed) - Add tests for rollback, dispose, events, and CAS edge cases - Update README and JSDoc to reflect current API surface Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
🔗 No broken links found! ✅ Your attention to detail is admirable. linkcheck output |
Summary
Adds a new
@fluidframework/claimsDDS implementing first-writer-wins claim semantics with compare-and-swap (CAS) support. All exports are@internalfor now.Key changes
ClaimsDDS —trySetClaim(key, value, expectedValue?)returnsAccepted,AlreadyClaimed, orPending(with a promise). Emits typed"claimed"events with key.expectedValueparameter enables CAS semantics: claim succeeds only if the current value matchesexpectedValue. Omitting it preserves write-once behavior.createDDSFuzzSuite, covering concurrent claims, CAS, and read operations with consistency validation.baseClaimsModelfrom./internal/testand registers it in the local-server-stress-testsddsModelMap.Why a new DDS?
Our long-term direction is consolidating DDSes into
SharedTree, butClaimsis strictly internal — it helps implement "aliasing" functionality at the dataobject/datastore level and is not intended for direct external use. It will become available to consumers via a scoped API surface.A DDS was chosen because:
What's NOT in this PR
@internal)Note for future
Currently, this new DDS does not accept trying to make claims when detached or disconnected. But, the API was designed in a way to allow this to happen in future. Here is the high level expected flow in the future:
flowchart TD A["trySetClaim(key, value)"] --> B{Key already\ncommitted?} B -->|Yes| C["AlreadyClaimed"] B -->|No| D{Detached?} D -->|Yes| E["Accepted"] D -->|No| F["Pending { promise }"] F --> G{Op roundtrips} G -->|Won| H["Accepted"] G -->|Lost| I["AlreadyClaimed"] G -->|Cancelled / Disposed| J["Aborted"] style C fill:#f9e79f style E fill:#abebc6 style H fill:#abebc6 style I fill:#f9e79f style J fill:#f5b7b1AB#74045