Eliminate trace() entry duplication in ColumnRepresentation

[piotrszul]

Summary

Fixes #2594 — a single source-level trace() call produced 2× or more TraceCollector entries when its result column was consumed by count(), exists(), empty(), last(), combine(), or the | union operator.

Root cause. Several ColumnRepresentation methods compiled to Spark when(...).otherwise(...) patterns that referenced the traced column more than once. Because TraceExpression is Nondeterministic, Spark's CSE does not deduplicate it — each reference in the expression tree fires independently.

Fix. Introduces ColumnHelpers.let(value, body) — a let-binding primitive built on Spark higher-order array functions (array + transform + element_at). array(value) materialises the operand once at codegen time; the body lambda then references the bound parameter, not the original expression. The result is a pure Column with no logical-plan dependency, safe in every relational context.

All 14 affected methods in ColumnRepresentation are rewritten:

• Spark builtins (no let): count, isEmpty, last, normaliseNull, aggregate, plural — single-reference rewrites using coalesce, try_element_at, nullif, filter(array(...)).
• let helper: singular, filter (scalar), toArray (scalar), transform (scalar) — conditionals where both branches need the operand.

What's new

• ColumnHelpers.java — new let primitive
• ColumnHelpersTest.java — unit tests for let (identity, multi-ref, single-fire over TraceExpression)
• ColumnRepresentationTraceTest.java — Layer B regression guard: one test per ColumnRepresentation method, asserts exactly one trace fire per row for both 1-row and 3-row inputs
• TraceFunctionTest — drops @Tag("known-failing") from the duplication matrix; adds 3 new rows exercising count > 0, where(...).first(), and combine()
• checkstyle.xml / suppressions.xml — RegexpMultiline rule (scoped to *Representation*.java) flags any future when(x...).otherwise(x...) patterns, failing CI before the bug can recur
• PathlingContext — startup assertion that spark.sql.legacy.sizeOfNull = false (required for the coalesce(size(c), 0) rewrite)
• fhirpath/pom.xml — removes the known-failing Surefire exclusion; all tests now run in the default suite

Test plan

• All TraceFunctionTest$TraceEntryCountTest rows pass (was 6 known-failing, now 13 pass)
• ColumnRepresentationTraceTest passes — every method fires exactly once per row
• ColumnHelpersTest passes
• mvn clean verify -pl fhirpath,encoders,library-api — all tests pass, Checkstyle clean
• PathlingContextTest.create_rejectsLegacySizeOfNullEnabled passes
• Checkstyle negative test: reverting count() to the old pattern triggers the RepeatedSqlEvaluation rule

Notes

• trace() inside SQL aggregate functions (sum, count, avg, …) is a separate Spark constraint on Nondeterministic expressions that this PR does not address. Tracked in Document trace() incompatibility with SQL aggregate functions #2607.
• The lambda-let pattern introduces a small single-element array allocation per let call. This is accepted; a custom Catalyst LetExpression remains available as a future optimisation if benchmarking shows it matters.

🤖 Generated with Claude Code