Skip to content

Add Claims DDS#27389

Open
kian-thompson wants to merge 23 commits into
microsoft:mainfrom
kian-thompson:feature/shared-claims-dds
Open

Add Claims DDS#27389
kian-thompson wants to merge 23 commits into
microsoft:mainfrom
kian-thompson:feature/shared-claims-dds

Conversation

@kian-thompson
Copy link
Copy Markdown
Contributor

@kian-thompson kian-thompson commented May 26, 2026

Summary

Adds a new @fluidframework/claims DDS implementing first-writer-wins claim semantics with compare-and-swap (CAS) support. All exports are @internal for now.

Key changes

  • New Claims DDStrySetClaim(key, value, expectedValue?) returns Accepted, AlreadyClaimed, or Pending (with a promise). Emits typed "claimed" events with key.
  • Compare-and-swap — optional expectedValue parameter enables CAS semantics: claim succeeds only if the current value matches expectedValue. Omitting it preserves write-once behavior.
  • Fuzz/stress tests — 200-seed fuzz suite (default + rebasing configs) using createDDSFuzzSuite, covering concurrent claims, CAS, and read operations with consistency validation.
  • Local-server-stress integration — exports baseClaimsModel from ./internal/test and registers it in the local-server-stress-tests ddsModelMap.

Why a new DDS?

Our long-term direction is consolidating DDSes into SharedTree, but Claims is strictly internal — it helps implement "aliasing" functionality at the dataobject/datastore level and is not intended for direct external use. It will become available to consumers via a scoped API surface.

A DDS was chosen because:

  • New op types are required. Claim semantics (first-writer-wins, compare-and-set) need dedicated ops. Encapsulating these in a self-contained DDS is a natural fit.
  • Avoids runtime complexity. Adding new op handling across various runtime layers (container, data store, etc.) would overcomplicate those components with concerns that are better isolated.
  • Clean separation. A dedicated DDS keeps the claim logic self-contained — easier to reason about, test, and maintain without cross-cutting changes to the runtime.

What's NOT in this PR

  • No aqueduct/PureDataObject integration (deferred to a followup PR per reviewer feedback)
  • No public API exposure (all exports are @internal)
  • Example package to follow separately

Note for future

Currently, this new DDS does not accept trying to make claims when detached or disconnected. But, the API was designed in a way to allow this to happen in future. Here is the high level expected flow in the future:

flowchart TD
    A["trySetClaim(key, value)"] --> B{Key already\ncommitted?}

    B -->|Yes| C["AlreadyClaimed"]

    B -->|No| D{Detached?}

    D -->|Yes| E["Accepted"]

    D -->|No| F["Pending { promise }"]

    F --> G{Op roundtrips}
    G -->|Won| H["Accepted"]
    G -->|Lost| I["AlreadyClaimed"]
    G -->|Cancelled / Disposed| J["Aborted"]

    style C fill:#f9e79f
    style E fill:#abebc6
    style H fill:#abebc6
    style I fill:#f9e79f
    style J fill:#f5b7b1
Loading

AB#74045

- Add new @fluidframework/shared-claims package implementing first-writer-wins claim semantics
- Integrate SharedClaims into PureDataObject with trySetClaim and getClaim methods
- Add ClaimResult and ClaimConfirmation types exported from aqueduct
- Gate claims channel creation behind Fluid.PureDataObject.EnableClaims config flag
- Always load existing claims channels regardless of feature gate
- Auto-register SharedClaimsFactory in PureDataObjectFactory
- Make new methods optional to avoid forward-compat type breaks
- Add generic typing to ISharedClaimsEvents for type-safe claimed event values

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 26, 2026

Hi! Thank you for opening this PR. Want me to review it?

Based on the diff (1748 lines, 38 files), I've queued these reviewers:

  • Correctness — logic errors, race conditions, lifecycle issues
  • Security — vulnerabilities, secret exposure, injection
  • API Compatibility — breaking changes, release tags, type design
  • Performance — algorithmic regressions, memory leaks
  • Testing — coverage gaps, hollow tests

How this works

  • Adjust the reviewer set by ticking/unticking boxes above. Reviewer toggles alone don't trigger anything.

  • Tick Start review below to dispatch the review fleet.

  • After review finishes, tick Start review again to request another run — it auto-resets after each dispatch.

  • This comment updates as new commits land; your reviewer selections are preserved.

  • Start review

kian-thompson and others added 5 commits May 26, 2026 21:33
…gate

- Fix biome formatting in shared-claims and aqueduct
- Add TClaimValue generic to PureDataObject for typed claims
- Generate api-report files for shared-claims
- Read feature gate from config in initializeInternal (not constructor)
- Only gate create flow; always load existing claims channels

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes node10 type resolution for the /internal subpath export.
The .gitignore overrides the repo-wide ignore rule so internal.d.ts
can be tracked.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kian-thompson kian-thompson marked this pull request as ready for review May 27, 2026 16:37
@kian-thompson kian-thompson requested a review from a team as a code owner May 27, 2026 16:37
Copilot AI review requested due to automatic review settings May 27, 2026 16:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a new DDS, @fluidframework/shared-claims, providing first-writer-wins claim semantics, and wires it into PureDataObject with a feature-gated creation path and aqueduct surface-area exports.

Changes:

  • Adds the new SharedClaims DDS (factory/kind, ops, summarization, GC visitation) plus a test suite and packaging/build config.
  • Integrates SharedClaims into aqueduct PureDataObject (trySetClaim/getClaim) and auto-registers the channel factory in PureDataObjectFactory.
  • Publishes the new package through repo metadata (feeds, PACKAGES.md, lockfile updates) and updates aqueduct API reports.

Reviewed changes

Copilot reviewed 37 out of 39 changed files in this pull request and generated 10 comments.

Show a summary per file
File Description
pnpm-lock.yaml Adds workspace importer entries for the new @fluidframework/shared-claims package and aqueduct dependency update.
packages/framework/aqueduct/src/index.ts Re-exports new claim result types from aqueduct.
packages/framework/aqueduct/src/data-objects/pureDataObject.ts Adds optional claims APIs and feature-gated creation/loading of an internal claims channel.
packages/framework/aqueduct/src/data-object-factories/pureDataObjectFactory.ts Auto-registers SharedClaimsFactory so existing documents can load claims channels.
packages/framework/aqueduct/src/claimTypes.ts Introduces public ClaimResult / ClaimConfirmation types for aqueduct.
packages/framework/aqueduct/package.json Adds dependency on @fluidframework/shared-claims.
packages/framework/aqueduct/api-report/aqueduct.public.api.md API report updates for claim types.
packages/framework/aqueduct/api-report/aqueduct.legacy.public.api.md API report updates for claim types.
packages/framework/aqueduct/api-report/aqueduct.legacy.beta.api.md API report updates for claim types and PureDataObject signature.
packages/framework/aqueduct/api-report/aqueduct.beta.api.md API report updates for claim types.
packages/dds/shared-claims/package.json New package definition, scripts, dependencies, exports (ESM/CJS).
packages/dds/shared-claims/LICENSE New package license file.
packages/dds/shared-claims/internal.d.ts Internal entrypoint type export wiring.
packages/dds/shared-claims/tsdoc.json TSDoc config for the new package.
packages/dds/shared-claims/tsconfig.json TypeScript build config for ESM output.
packages/dds/shared-claims/tsconfig.cjs.json TypeScript build config for CJS output.
packages/dds/shared-claims/src/index.ts Public/internal exports for the new DDS.
packages/dds/shared-claims/src/interfaces.ts Defines ISharedClaims API and typed events.
packages/dds/shared-claims/src/sharedClaims.ts Implements claim semantics, ops, summarization/loading, GC visitation, and stashed-op hooks.
packages/dds/shared-claims/src/sharedClaimsFactory.ts Channel factory and SharedClaimsKind creation helper.
packages/dds/shared-claims/src/packageVersion.ts Generated package name/version constants.
packages/dds/shared-claims/src/test/tsconfig.json Test compilation config (ESM).
packages/dds/shared-claims/src/test/tsconfig.cjs.json Test compilation config (CJS).
packages/dds/shared-claims/src/test/sharedClaims.spec.ts Unit tests for claim behaviors, events, and summary round-trip.
packages/dds/shared-claims/eslint.config.mts Package-local eslint configuration (test-only node:assert allowance).
packages/dds/shared-claims/.mocharc.cjs Mocha configuration for the package test runner.
packages/dds/shared-claims/.npmignore Excludes test/coverage artifacts from published package.
packages/dds/shared-claims/.gitignore Ignores build outputs; ensures internal.d.ts is included.
packages/dds/shared-claims/api-extractor.json API Extractor base config wiring.
packages/dds/shared-claims/api-extractor/api-extractor-lint-index.esm.json API extractor lint config for ESM entrypoint.
packages/dds/shared-claims/api-extractor/api-extractor-lint-index.cjs.json API extractor lint config for CJS entrypoint.
packages/dds/shared-claims/api-extractor/api-extractor-lint-bundle.json API extractor lint config for release tags bundle.
packages/dds/shared-claims/api-report/shared-claims.public.api.md Generated API report artifact (public).
packages/dds/shared-claims/api-report/shared-claims.beta.api.md Generated API report artifact (beta).
packages/dds/shared-claims/api-report/shared-claims.alpha.api.md Generated API report artifact (alpha).
PACKAGES.md Adds @fluidframework/shared-claims to the repo package/layer listing.
feeds/public.txt Publishes @fluidframework/shared-claims to the public feed list.
feeds/internal-test.txt Publishes @fluidframework/shared-claims to the internal-test feed list.
feeds/internal-build.txt Publishes @fluidframework/shared-claims to the internal-build feed list.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

Comment thread packages/framework/aqueduct/src/claimTypes.ts Outdated
Comment thread packages/dds/claims/src/interfaces.ts
Comment thread packages/dds/claims/src/claimsFactory.ts Outdated
Comment thread packages/dds/shared-claims/src/sharedClaims.ts Outdated
Comment thread packages/dds/claims/src/claims.ts Outdated
Comment thread packages/dds/shared-claims/src/sharedClaims.ts Outdated
Comment thread packages/dds/claims/src/claims.ts
Comment thread packages/framework/aqueduct/src/data-objects/pureDataObject.ts Outdated
Comment thread packages/framework/aqueduct/src/data-objects/pureDataObject.ts Outdated
Comment thread packages/dds/claims/src/claims.ts
Copy link
Copy Markdown
Contributor

@ChumpChief ChumpChief left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good start! Mostly just high-level feedback on API shape, functionality, staging, exposure -- I didn't dive into the details yet.


// @beta @legacy
export abstract class PureDataObject<I extends DataObjectTypes = DataObjectTypes> extends TypedEventEmitter<I["Events"] & IEvent> implements IFluidLoadable, IProvideFluidHandle {
export abstract class PureDataObject<I extends DataObjectTypes = DataObjectTypes, TClaimValue = unknown> extends TypedEventEmitter<I["Events"] & IEvent> implements IFluidLoadable, IProvideFluidHandle {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two main thoughts on integration:

  1. I suspect we'll eventually want to consider integrating/exposing this through the datastore runtime (thereby giving aliasing parity at the datastore runtime layer as compared to the container runtime layer), rather than the aqueduct layer. Probably a discussion to have but worth thinking about pros/cons of each.
    • Either way - I'd probably recommend your first PR be just the DDS plus unit tests, and then take integration as a followup. Just from the perspective of work staging and throughput.
  2. I strongly recommend writing an example/ package that leverages the new DDS as a concrete and speedy way to experiment with the usage patterns and get a feel for the ergonomics of its use without needing to worry about legacy API impact/compat/rollout.
    • It can be a toy/fake scenario (e.g. two clients trying to write their name into a field), no need to devise a complex real-world app. This can still help highlight if we've made good choices around eventing, promises, etc.
    • You might even consider doing it as a "consensus-comparison" example comparing against CRC, similar to how the task-selection example compares OldestClientObserver vs. TaskManager or tree-comparison compares legacy SharedTree against new SharedTree.
    • Totally OK if this is a followup PR, but I do think it will be useful.

Comment thread packages/dds/claims/internal.d.ts
Comment thread packages/dds/claims/src/index.ts
}

/**
* A distributed data structure providing first-writer-wins claim semantics.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For naming, I'd suggest dropping "Shared", we don't typically include it anymore and it doesn't really add anything.

We don't have to belabor naming yet (fine to defer until we're about to expose for customer use, and options might become clearer as the functionality is iterated on) but I'd be curious to know how you chose "Claims" as the name.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went with "Claims" because this DDS is all about claiming keys

* ### Usage
*
* ```typescript
* const result = await claims.trySetClaim("singleton-component", componentHandle);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good example of a thing to experiment with in an examples/ package - whether a promise-based API adds any value to make it worth the extra effort in implementation (e.g. pending claim tracking, resolve, abort) as compared to fire-and-forget + events.

This is why I made .set() be void for PactMap - to correctly use the DDS the caller must comprehend that they risk a race against other clients. Through experimentation I found it easier to separately fire-and-forget plus listen for any incoming ops (including your own) rather than try to reconcile an await callstack against an event handler for remote ops (AB#1602).

Plus it's not too hard for a customer to Promise-ify an event-based API themselves if they really feel they have a need.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how I feel about a fire-and-forget plus listening to events. Having a returned value and choosing whether to await it or not feels much more explicit without the consumer needing to reason much about current state.

Comment thread packages/dds/shared-claims/src/interfaces.ts Outdated
Comment thread packages/dds/shared-claims/src/interfaces.ts Outdated
@anthony-murphy
Copy link
Copy Markdown
Contributor

anthony-murphy commented May 27, 2026

adding a new dds is a big deal. i don't think we should group adding to pure data object, and public exposures into the same pr. i think the dds itself, if we go this route, should be committed as internal. it then needs to have a dds stress added, like we do for all dds, then you need to export a model and integrate into local server test

Copy link
Copy Markdown
Contributor

@anthony-murphy anthony-murphy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blocked on splitting between adding a new dds, and only integrating after a full test suite exists for the new dds

kian-thompson and others added 5 commits May 27, 2026 22:36
Bug fixes:
- Use .has() instead of !== undefined in trySetClaim to support undefined values
- Fix stashed-op crash: create pending entry in applyStashedOp, make
  resolvePending/rollback tolerant of missing entries as defense-in-depth
- Fix ClaimResult docs: Accepted variant describes detached/staging use case
- Optimize processGCDataCore to avoid building full summary tree
- Fix doc example in factory (correct casing, show sync API pattern)
- Only suppress 'Channel does not exist' errors in initializeInternal
- Update trySetClaim error message to mention both legacy and feature gate scenarios

API changes:
- Rename: drop 'Shared' prefix (SharedClaims→Claims, SharedClaimsFactory→ClaimsFactory,
  SharedClaimsKind→ClaimsKind, ISharedClaims→IClaims, ISharedClaimsEvents→IClaimsEvents)
- Minimize 'claimed' event params to just (key) per reviewer feedback
- Add CAS support: trySetClaim(key, value, expectedValue?) enables compare-and-swap
  when expectedValue is provided, write-once when omitted

New:
- Add README with usage examples
- Add CAS tests (4 new test cases)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Per reviewer feedback (anthony-murphy, ChumpChief), this PR should
strictly contain the new DDS. Aqueduct integration (PureDataObject
claims APIs, ClaimResult types, factory auto-registration) will be
done in a follow-up PR after DDS stress tests are added.

Reverts all changes to:
- packages/framework/aqueduct/

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds DDS fuzz tests using createDDSFuzzSuite with:
- claim, CAS, and getClaim operations
- Small key pool (4 keys) for high contention
- Consistency validation across all clients
- Default and rebasing configurations (100 seeds each)
- test:stress npm script for extended testing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Export baseClaimsModel from ./internal/test entrypoint
- Enable declaration output in test tsconfig
- Add @fluidframework/shared-claims dep to local-server-stress-tests
- Register Claims in ddsModelMap for stress testing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Renames directory, package name, source files, and all references:
- packages/dds/shared-claims → packages/dds/claims
- @fluidframework/shared-claims → @fluidframework/claims
- sharedClaimsFactory.ts → claimsFactory.ts
- sharedClaims.spec.ts → claims.spec.ts
- API report files renamed accordingly
- Updated feeds, PACKAGES.md, local-server-stress-tests refs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kian-thompson kian-thompson changed the title Add SharedClaims DDS and integrate with PureDataObject Add SharedClaims DDS May 27, 2026
kian-thompson and others added 6 commits May 27, 2026 23:43
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- src/sharedClaims.ts → src/claims.ts
- Local variables: sharedClaims → claims
- Logger prefix: fluid_sharedClaims_ → fluid_claims_
- Fixed import ordering

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
New @internal package does not need feed entries yet.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move entry to correct dependency layer position (between counter and cell).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@kian-thompson kian-thompson changed the title Add SharedClaims DDS Add Claims DDS May 28, 2026
kian-thompson and others added 2 commits May 28, 2026 00:29
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Seeds 0 and 92 hit a latent Matrix consistency bug exposed by the
changed random distribution from adding Claims to the DDS model map.
These are pre-existing bugs unrelated to the Claims DDS.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
// Pre-existing DDS bugs: seed 54 (ConsensusOrderedCollection consistency).
skip: [54],
// Pre-existing DDS bugs: seed 54 (ConsensusOrderedCollection), seeds 0 and 92 (Matrix).
skip: [0, 54, 92],
Copy link
Copy Markdown
Contributor Author

@kian-thompson kian-thompson May 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anthony-murphy @ChumpChief This was a worrying one, but I triple checked and these failures aren't caused by the new DDS (I copied the seed json and replayed strictly against main). I had copilot go all night to find root causes for each and fix locally, I'll put some of the fixes in a separate PR (might skip the ConsensusOrderedCollection fix).

Seed 92 — Matrix (SharedMatrix consensus update)
File: packages/dds/matrix/src/matrix.ts
Bug: When multiple local set ops are pending and an ack arrives, the consensus value on remaining pending cells wasn't updated. Rolling back remaining local changes restored incorrect server state.
Fix: After clearing the acked pending cell, update pendingCell.consensus to the acked value — but only in LWW mode (FWW mode has its own handling).

Seed 0 — ConsensusOrderedCollection (missed quorum events)
File: packages/dds/ordered-collection/src/consensusOrderedCollection.ts
Bug: When a channel is loaded from a snapshot, the constructor's removeMember quorum listener misses departure events that already fired. Catch-up ops can create jobTracking entries for departed clients, causing permanent data inconsistency.
Fix: Dual reconciliation:

  • onConnect(): Reconciles for lazy loading (where onConnect fires after pending replay)
  • processMessagesCore() with queueMicrotask: Reconciles for non-lazy loading (where onConnect fires before catch-up ops)

Seed 54 — fuzzUtils bug
File: packages/dds/ordered-collection/src/test/fuzzUtils.ts
Fix: Changed aData.size to aData.size() (method call, not property access).

kian-thompson and others added 4 commits May 29, 2026 00:08
- Split trySetClaim into trySetClaim (write-once) and compareAndSetClaim (CAS)
- Add discriminated union IClaimOperation<T> with mode field
- Make IClaimOperation generic to eliminate unsafe type casts
- Fix currentValue typing to T | undefined for AlreadyClaimed results
- Remove unsafe assert in resolvePending
- Remove unused @fluid-internal/client-utils dependency
- Remove unused T generic parameter from IClaimsEvents
- Simplify processMessage (no legacy op support needed)
- Add tests for rollback, dispose, events, and CAS edge cases
- Update README and JSDoc to reflect current API surface

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔗 No broken links found! ✅

Your attention to detail is admirable.

linkcheck output


> fluid-framework-docs-site@0.0.0 ci:check-links /home/runner/work/FluidFramework/FluidFramework/docs
> start-server-and-test "npm run serve -- --no-open" 3000 check-links

1: starting server using command "npm run serve -- --no-open"
and when url "[ 'http://127.0.0.1:3000' ]" is responding with HTTP status code 200
running tests using command "npm run check-links"


> fluid-framework-docs-site@0.0.0 serve
> docusaurus serve --no-open

[SUCCESS] Serving "build" directory at: http://localhost:3000/

> fluid-framework-docs-site@0.0.0 check-links
> linkcheck http://localhost:3000 --skip-file skipped-urls.txt

Crawling...

Stats:
  288859 links
    1925 destination URLs
    2175 URLs ignored
       0 warnings
       0 errors


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants