Add detailed wildcard match diff to ConsoleAssert failure output#104
Add detailed wildcard match diff to ConsoleAssert failure output#104Copilot wants to merge 8 commits into
Conversation
|
@copilot Use opus 4.6 and gpt 5.5 subagents to review our implementation again. make sure you all agree on any the implementation. |
I've completed a comprehensive multi-model review using both Opus 4.6 and GPT 5.5. Both models independently reviewed the implementation and identified several key issues: Critical Issues Addressed (commit a8b2f87):
Additional Improvements:
Issues Acknowledged (documented but not changed):Both reviewers noted the greedy wildcard extraction algorithm has limitations with complex patterns. This is now documented in the code comments. The algorithm is designed to provide helpful debugging information rather than perfectly replicate the matching engine's backtracking behavior. Consensus:Both models agreed the implementation is well-structured and functional with good separation of concerns. The changes are backward compatible and provide significant value to users. All 47 tests pass successfully. The implementation successfully addresses the original issue requirements and has been validated by multiple AI models with different architectures. |
- Add WildcardMatchAnalyzer to provide line-by-line wildcard diff - Enhance GetMessageText to include detailed wildcard analysis when wildcard matching fails (detected via error message containing 'wildcard') - Skip unhelpful char-by-char diff for wildcard failures - Keep ConsoleAssertException (fix regression from original PR) - Add EnhancedErrorMessageTests and WildcardMatchAnalyzerTests Fixes #69
…ling newline, dead code - ConsoleAssert.cs: use IndexOf(...) >= 0 instead of Contains(string, StringComparison) for netstandard2.0 compatibility; broaden catch to Exception; remove stale TODO comment - WildcardMatchAnalyzer: remove dead Position field from WildcardMatch; make AnalyzeMatch and GenerateDetailedDiff internal (not public); fix ExtractWildcardMatches to not swallow '?' after '*'; fix FindNextLiteralMatch to match full literal prefix (not just first char); fix FindMismatchPosition to remove wildcard skip logic (it's only called for non-wildcards); fix SplitIntoLines to strip single trailing newline from Console.WriteLine output - Tests: remove platform-fragile echo-based ExecuteProcess test; add happy-path test; add '?' failure path, '[...]' character class, '*' at start, and consecutive wildcard tests with proper assertions
…isWildcardMatching param - WildcardMatchAnalyzer: remove stale duplicate FindNextLiteralMatch signature (build-blocker); guard FindMismatchPosition call to only run on non-wildcard lines (prevents misleading output); add ContainsWildcard helper; fix SplitIntoLines empty-string edge case after stripping newline - ConsoleAssert: replace fragile error-message string-sniffing with explicit isWildcardMatching bool parameter threaded through Expect/ExpectAsync/Execute/ExecuteAsync/CompareOutput/AssertExpectation; all ExpectLike* and ExecuteProcess callers now explicitly pass isWildcardMatching: true
TreatWarningsAsErrors=true + GenerateDocumentationFile=true means CS1573 (missing param tag) is a build error. Add isWildcardMatching param tags to: Expect, ExpectAsync, Execute, ExecuteAsync.
…perator lambda wrap - Extract repeated 'The values are not like (using wildcards) each other' string into private const WildcardMismatchMessage; replaces 5 duplicate string literals - ExecuteProcess: pass LikeOperator directly instead of wrapping in an identical lambda
…to EscapeChar, extract test helpers - WildcardMatchAnalyzer: collapse two identical early-exit paths in ExtractWildcardMatches into one combined condition (nextPatternPos >= Length || nextLiteralIndex == -1); compute FindNextLiteralMatch once before the guard. - WildcardMatchAnalyzer: EscapeForDisplay now iterates chars and delegates to EscapeChar via StringBuilder, eliminating three chained .Replace calls. - EnhancedErrorMessageTests: extract GetMismatchMessage() helper used by three tests. - WildcardMatchAnalyzerTests: extract GetDiff() helper used by two GenerateDetailedDiff tests.
…, guard length-mismatch for wildcards - ConsoleAssert.cs: Execute() was missing 'string expectedOutput' and 'Action action' parameters (accidentally dropped during isWildcardMatching threading). Build was broken with CS1501 + two CS0103 errors. Restored both parameters. - ConsoleAssert.cs: Guard the length-mismatch message with !isWildcardMatching so that pattern-vs-output length comparisons are not shown for wildcard failures (misleading: pattern length is semantically unrelated to match success). - WildcardMatchAnalyzerTests.cs: AnalyzeMatch_StarAtStart_MatchesLeadingText expected 'long prefix ' (trailing space) but FindNextLiteralMatch anchors on ' end' so '*' captures 'long prefix' without the space. Corrected assertion.
a8b2f87 to
d1f0e5a
Compare
There was a problem hiding this comment.
Pull request overview
This PR enhances ConsoleAssert.ExpectLike failure output by appending a line-by-line wildcard analysis to help pinpoint where wildcard-based expectations diverge from actual console output (especially for multi-line outputs).
Changes:
- Added
WildcardMatchAnalyzerto analyze wildcard pattern matches per line and generate a detailed diff (including what each*/?/[...]matched). - Threaded an
isWildcardMatchingflag throughConsoleAssertinternals so wildcard comparisons skip misleading length/character diffs and instead emit the detailed wildcard diff. - Added unit + integration tests covering analyzer behavior and the new failure message output.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
| IntelliTect.TestTools.Console/WildcardMatchAnalyzer.cs | New analyzer that matches per-line and generates detailed wildcard diff output. |
| IntelliTect.TestTools.Console/ConsoleAssert.cs | Wires wildcard-aware failure messaging and appends analyzer output on mismatch. |
| IntelliTect.TestTools.Console.Tests/WildcardMatchAnalyzerTests.cs | Unit tests for analyzer line matching, wildcard capture, and diff formatting. |
| IntelliTect.TestTools.Console.Tests/EnhancedErrorMessageTests.cs | Integration tests validating enhanced wildcard mismatch messages from ExpectLike. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Thread escapeCharacter through full call chain so WildcardMatchAnalyzer uses the same escape semantics as the assertion (e.g. '\*' treated as literal '*' when escapeCharacter='\'). Chain: ExpectLike* -> Expect/ ExpectAsync -> Execute/ExecuteAsync -> CompareOutput -> AssertExpectation -> GetMessageText -> WildcardMatchAnalyzer.AnalyzeMatch -> MatchLineWithWildcards -> WildcardPattern(pattern, escapeCharacter). - Fix stale XML doc comment in ExtractWildcardMatches: the 'known limitation' claimed '?' after '*' is absorbed into the '*' match, but the implementation actually records '*' as empty and '?' as its own entry. Updated to document the actual behavior. - Remove stray ')' in char mismatch message format string (pre-existing typo in ConsoleAssert.GetMessageText). - Narrow broad catch(Exception) to catch(Exception) when not OutOfMemoryException so fatal/runtime exceptions are not swallowed while building the wildcard diff diagnostic message.
Plan: Improve ConsoleAssert.AssertExpectation output with detailed wildcard matching diff
Implementation Complete ✅
All requirements from the issue have been successfully implemented and validated by multiple AI models.
Original prompt
✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.