v2.2.0 Release#147
Conversation
…ng and entity type Object. (#132) Signed-off-by: Mike Fuller <mike@finops.org> --------- Signed-off-by: Mike Fuller <mike@finops.org>
Bumped version Signed-off-by: Mike Fuller <mike@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Signed-off-by: Mike Fuller <mike@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Signed-off-by: Mike Fuller <mike@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org> Signed-off-by: Mike Fuller <mike@finops.org>
Apply review feedback across the FOCUS 1.4 check generators: - Make every leaf generator row-condition-aware by wrapping its predicate_sql in _apply_condition, matching FormatJSON. Previously only FormatJSON did this; the predicate is read directly off the stored SQLQuery, so the condition must be baked in at construction. - Fix CheckColumnComparison null handling. The violation condition now requires both columns non-null before comparing, so null rows are no longer flagged. This aligns it with CheckSameValue's convention. - Collapse CheckGreaterOrEqual, CheckGreaterThan, and CheckLessOrEqual into a shared _CheckScalarComparisonGenerator base. Each subclass now sets only its operators, message phrase, and check type. This also standardizes literal quoting and predicate gating across the three. Tests: - Add a row-condition predicate test covering the named comparison generators to lock in the now-uniform behavior. - Update the CheckStringEndsWith assertion to expect the ends_with SQL. - Add tests/config_objects/test_generator_edge_cases.py, which executes generated SQL against a real in-memory DuckDB to cover null handling, multi-byte suffixes, malformed JSON paths, a missing jsonschema dependency, and row-filtered de-duplication. Signed-off-by: Mike Fuller <mike@finops.org>
Co-authored-by: Matt Cowsert <matthew@finops.org>
The Utf8 datetime parsing strategies each required a zero null_count, so a single null cell (an originally-null value or one unparseable entry) forced every strategy to fail and the column to stay a string. Nullable datetime columns are common in FOCUS, so this path was hit in practice. Judge parse success against the original null count instead of zero: a strategy now succeeds when parsing introduces no new nulls beyond those already present and the column has at least one real value. A nullable single-format column converts, a genuinely mixed or unparseable column is still rejected, and an all-null column is left as a string rather than coerced. Add regression tests covering nullable ISO (Z and offset), date-only, and space-separated columns, plus a mixed-format column that must be dropped. Also fix a stale TypeJSON test assertion: the generator now emits NOT json_valid(...) rather than typeof(...) != 'JSON'. Signed-off-by: Mike Fuller <mike@finops.org>
Signed-off-by: Mike Fuller <mike@finops.org>
Signed-off-by: Mike Fuller <mike@finops.org>
Clean up issue with main having a squash marge on it Signed-off-by: Mike Fuller <mike@finops.org>
Matt-Cowsert
left a comment
Matt-Cowsert
left a comment
There was a problem hiding this comment.
FOCUS 1.4 support. One item to address in the explain path; the inline notes are optional.
explain type rename is incomplete. _explain_check_sql now returns "type": "special" for special-executor leaf checks where it previously returned "type": "reference" (focus_to_duckdb_converter.py:6453). Three places still expect "reference":
focus_to_duckdb_converter.py:6544—print_sql_map'selif t == "reference":is now unreachable, so theReference to: …line no longer prints.validator.py:327—if child_type == "reference":no longer matches, so compositeexplainoutput drops thechild_id -> referenced_idmapping for reference children.focus_to_duckdb_converter.py:6459-6463— thenoteternary checksspecial_kind == "reference", but the conformance-reference executor (:2166-2168) setsexec_mode = "reference"and never setsmeta["special_executor_kind"] = "reference"(only the JSON-schema path sets that key,:1327), so reference checks fall through to the genericexecuted via special executornote.
These affect explain/introspection output only, not validation results, which is why the suite stays green. One way to close it: set chk.meta["special_executor_kind"] = "reference" in the reference executor, then branch the two consumers on type == "special" (plus the kind) rather than type == "reference".
| sql = query.replace("{table_name}", table_name) | ||
| sql = sql.replace("{table_name}", table_name) |
There was a problem hiding this comment.
The second replace is a no-op — the first call already substituted the only {table_name} placeholder.
| sql = query.replace("{table_name}", table_name) | |
| sql = sql.replace("{table_name}", table_name) | |
| sql = query.replace("{table_name}", table_name) |
| "CheckJSONSchema requires the 'jsonschema' package to be installed" | ||
| ) from exc | ||
|
|
||
| Draft202012Validator.check_schema(schema) |
There was a problem hiding this comment.
check_schema() runs on every execution and raises jsonschema.SchemaError for a malformed model schema, surfacing as an unhandled exception mid-run. Validating it in generateCheck() would fail fast as InvalidRuleException, consistent with how a missing SchemaId is already handled here.
|
|
||
| failure_messages: list[str] = [] | ||
| violations = 0 | ||
| for row_num, row in enumerate(rows, start=1): |
There was a problem hiding this comment.
row_num counts position within the filtered result set, so row {row_num} in the failure message won't match the source data row once nulls or a row condition drop rows. Minor, but worth noting if these messages reach users.
| def get_sample_sql(self) -> str: | ||
| col_a = self.params.ColumnAName | ||
| col_b = self.params.ColumnBName | ||
| comparator = self.params.Comparator |
There was a problem hiding this comment.
Comparator isn't re-validated here the way generateSql() does at :1927. Safe today since generateSql() runs first and raises on an unknown comparator, but factoring the check (and the shared violation condition) out would keep the two paths from drifting.
2 participants
FOCUS 1.4 Support
Added new check functions and support for 1.4 validation