Tighten canonical-repo match in queryTargetsDependingOnModules#338
Open
rdark wants to merge 2 commits into
Open
Tighten canonical-repo match in queryTargetsDependingOnModules#338rdark wants to merge 2 commits into
rdark wants to merge 2 commits into
Conversation
e031d63 to
1f737d4
Compare
Collaborator
|
@rdark There is a conflict now. Thanks again for this |
1f737d4 to
0d4922c
Compare
On MODULE.bazel-touching commits the impacted-targets path did
`contains(moduleName)` against full target labels to find canonical repos
to query. A module named "cpp" pulled in every `abseil-cpp+//...` label;
in workspaces with many bzlmod module extensions the resulting unioned
rdeps query swept far more of the workspace than it should have.
Replace the substring scan with a name-prefix predicate against the parsed
canonical repo (`label.substring(2).substringBefore("//")`): R belongs to
M if R starts with `"{M.name}+"` or `"{M.name}~"`. Bazel's bzlmod canonical
names are deterministic from the module name, and extension-created
`name++ext+repo` / `name~~ext~repo` forms are subsumed by these prefixes.
Also fix a pre-existing bug in the same function: the trailing
`computeSimpleImpactedTargets(emptyMap(), allTargets)` call returned
`allTargets.keys`, silently unioning every target back into the rdeps
result. Thread `from` through the call chain so the union uses the real
hash diff. Without this, the rdeps filtering would be cost-only — the
impacted set would still equal the full workspace.
Reject empty `module.name` in `ModuleGraphParser.extractModules`: Bazel's
MODULE.bazel spec disallows empty names, but `bazel mod graph
--output=json` emits `name=""` for the synthetic root entry of a
MODULE.bazel that omits `module(name=…)`. Filtering at the parser stops
that case from reaching the canonical-repo predicate.
Tests: new CalculateImpactedTargetsInteractorModuleQueryTest covers
canonical-plus, canonical-tilde, extension-created subsume, the
"cpp must not match abseil-cpp" regression, transitive-by-name-prefix,
ghost-module skip, multiple-disjoint-modules, and the version-bump-as-
add+remove dedup case. New parser test covers empty-name rejection.
The per-repo rdeps query-failure fallback filtered out every `@@` label on the assumption that the hash set was mostly workspace-local `//...` targets. That assumption does not hold on bzlmod-only workspaces where `generate-hashes` emits almost entirely `@@canonical` labels plus a small number of `//external:<apparent>` bzlmod-synthetic bridges, with zero workspace-local targets. On such workspaces, a single failed `rdeps(//..., @@<repo>//...)` query left the filter with only the `//external:*` bridges, which the downstream `excludeExternalTargets=true` default (Tinder#334) then strips entirely — producing silently-empty impacted output on what should be a "conservatively rebuild everything" signal. Replace the unconditional `!startsWith("@@")` filter with a shape-aware fallback: if the buildable workspace subset (`//...` minus `//external:`) is non-empty, emit it; otherwise fall through to every hashed label so the downstream filter has something to keep. The "buildable workspace" predicate is extracted as `isBuildableWorkspaceTarget` and explicitly distinguished from the user-controlled `excludeExternalTargets` filter. Adds bzlmod-only and mixed-workspace regression tests covering both branches of the fallback.
0d4922c to
9c3b424
Compare
Contributor
Author
|
@tinder-maxwellelliott fixed up the conflicts and simplified this one a bit - thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partial fix for #335.
Two commits:
1. Restrict canonical-repo match in
queryTargetsDependingOnModulesOn
MODULE.bazel-touching commits the impacted-targets path usedcontains(moduleName)against full target labels to find which canonical repos to feed into rdeps. A module named"cpp"pulled in everyabseil-cpp+//...label; in workspaces with many bzlmod module extensions the unionedrdeps(//..., @@a//... + @@b//... + ...)query swept far more of the workspace than it should have.Replace the substring scan with a name-prefix predicate against the parsed canonical repo (
label.substring(2).substringBefore("//")):Rbelongs toMifRstarts with"{M.name}+"or"{M.name}~". Bazel's bzlmod canonical names are deterministic from the module name (per themodule()spec), and extension-createdname++ext+repo/name~~ext~repoforms are subsumed by these prefixes.Also fixes a pre-existing bug in the same function:
computeSimpleImpactedTargets(emptyMap(), allTargets)returnedallTargets.keys, silently unioning every target back into the rdeps result. Threadsfromthrough the call chain so the union uses the real hash diffRejects empty
module.nameinModuleGraphParser.extractModules. Bazel'sMODULE.bazelspec disallows empty names, butbazel mod graph --output=jsonemitsname=""for the synthetic root entry of aMODULE.bazelthat omitsmodule(name=…). Filtering at the parser stops that case from reaching the canonical-repo predicate.2. Make rdeps-failure fallback shape-aware for bzlmod-only workspaces
The rdeps-failure catch block filtered out every
@@label on the assumption that the hash set was mostly workspace-local//...targets. That assumption breaks on bzlmod-only workspaces where generate-hashes emits almost entirely@@canonicallabels plus a small number of//external:<apparent>synthetic targets. A single failed rdeps query would leave the filter with only the synthetic targets, whichexcludeExternalTargets=true(#334) then strips entirely. Replaces the unconditional filter with a shape-aware fallback: emit the buildable workspace subset (//...minus//external:) when it exists, otherwise fall through to every hashed label.Tests in
CalculateImpactedTargetsInteractorModuleQueryTestcover: canonical+form, canonical~form, extension-created-repo subsume, the "cpp must not match abseil-cpp" regression, transitive-by-name-prefix, ghost-module skip, multiple-disjoint-modules, version-bump-as-add+remove dedup, both branches of the bzlmod-only/mixed-workspace fallback, and theexecuteWithDistancesmodule-query path. Output assertions on the empty-rdeps cases guard against regression to theemptyMap()union bug. NewModuleGraphParserTestcase covers the empty-name rejection.Notable changes vs. the original description:
module.name, so the Tier A repo-mapping lookup added nothing the name-prefix match didn't already cover).