Add STRUCT/ARRAY/MAP nested column support to RAB (CSA-371)#2392
Open
bladata1990 wants to merge 4 commits intomainfrom
Open
Add STRUCT/ARRAY/MAP nested column support to RAB (CSA-371)#2392bladata1990 wants to merge 4 commits intomainfrom
bladata1990 wants to merge 4 commits intomainfrom
Conversation
Expands complex SQL types (STRUCT, ARRAY<STRUCT>, MAP<K,STRUCT>) into child Column assets linked via parentColumn hierarchy. Sub-columns are excluded from the table's flat Columns list by clearing all tableQualifiedName/tableName/table/view refs — navigation is via parentColumn chain only. New fields on sub-columns: - parentColumnQualifiedName / parentColumn / parentColumnName - columnHierarchy: newline-delimited JSON ancestor entries (enables breadcrumb display in the UI, e.g. struct_col > city) - columnDepthLevel: 1 for direct fields, 2+ for deeper nesting - nestedColumnOrder: ordinal position within parent - subType=nested QN format (matching Databricks connector): - STRUCT field: tableQN/parentCol/fieldName - ARRAY<STRUCT>: tableQN/parentCol/items/fieldName - MAP<K,STRUCT>: tableQN/parentCol/values/fieldName - Deeply nested: tableQN/parentCol/outer/inner Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: bladata1990 <balakrishnan.r@atlan.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: bladata1990 <balakrishnan.r@atlan.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> Signed-off-by: bladata1990 <balakrishnan.r@atlan.com>
The dependencies { include(project(":samples:packages:asset-import")) }
filter was accidentally removed from the shadowJar block, causing the fat
jar to bundle all transitive SDK dependencies instead of just asset-import.
This conflicted with the base container image jars and caused
ClassNotFoundException: com.atlan.pkg.rab.Importer at runtime.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Signed-off-by: bladata1990 <balakrishnan.r@atlan.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
parentColumnhierarchyKey design decisions
Why sub-columns don't appear in the flat Columns list:
Atlan auto-creates the
table_columnsrelationship fromtableQualifiedNameserver-side. Sub-columns have all table/view reference fields cleared (tableQualifiedName,tableName,table,viewQualifiedName,viewName,view,materializedView) so they are NOT added totable_columns. Navigation is via theparentColumnchain.QN format (matching Databricks connector):
tableQN/parentCol/fieldNametableQN/parentCol/items/fieldNametableQN/parentCol/values/fieldNametableQN/parentCol/outerField/innerFieldFields set on sub-columns:
parentColumnQualifiedName/parentColumn/parentColumnNamecolumnHierarchy: newline-delimited JSON ancestor entries (enables breadcrumb display)columnDepthLevel: 1 for direct fields, 2+ for deeper nestingnestedColumnOrder: ordinal within parentsubType=nestedFiles changed
ComplexTypeParser.kt— NEW: bracket-aware recursive parser for STRUCT/ARRAY/MAP type stringsAssetXformer.kt— added nested column fields toBASE_OUTPUT_HEADERSColumnXformer.kt— overridesmapRow()to recursively emit child column rows for complex typesComplexTypeParserTest.kt— NEW: unit tests for the parserassets-complex.csv— NEW: test fixture with STRUCT/ARRAY/MAP typesbuild.gradle.kts(RAB + AIM) — minor build fixesTest plan
ci_noqn_testtable)plain_col,struct_col,nested_struct_colappear in table's flat Columns list (3 total)city,zipappear only insidestruct_col's nested viewouter,labelappear only insidenested_struct_col's nested viewinner,countappear only insideouter's nested view (depth=2)columnHierarchybreadcrumbs display correctly in the UI./gradlew :samples:packages:relational-assets-builder:test -PpackageTests🤖 Generated with Claude Code