Commit 0772110
committed
store: Implement Parquet dump for subgraph data
Wire dump queries and Parquet writing into Layout::dump(). This is the
core of the dump feature, covering both entity tables and the special
data_sources$ table.
For entity tables: build a DynamicSelectClause that selects vid, block
columns (split into lower/upper for mutable tables), causality_region,
and data columns. Use VidBatcher for adaptive batching, convert rows
to Arrow RecordBatch, and write via ParquetChunkWriter.
For data_sources$: use a concrete QueryableByName struct with raw SQL
(fixed schema, no DynamicSelectClause needed). Check table existence
via catalog::table_exists before attempting dump.
Metadata includes version, network, block pointers, entity count,
graft info, health, indexes, and per-table chunk tracking. Written
atomically via tmp+rename so its presence signals a complete dump.
Add entity_count() helper in detail.rs. Wire dump() through
DeploymentStore and SubgraphStore.1 parent ff71687 commit 0772110
6 files changed
Lines changed: 716 additions & 1 deletion
File tree
- store/postgres/src
- parquet
- relational
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
33 | 34 | | |
34 | 35 | | |
35 | 36 | | |
| |||
896 | 897 | | |
897 | 898 | | |
898 | 899 | | |
| 900 | + | |
| 901 | + | |
| 902 | + | |
| 903 | + | |
| 904 | + | |
| 905 | + | |
| 906 | + | |
| 907 | + | |
| 908 | + | |
| 909 | + | |
| 910 | + | |
| 911 | + | |
899 | 912 | | |
900 | 913 | | |
901 | 914 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
657 | 657 | | |
658 | 658 | | |
659 | 659 | | |
| 660 | + | |
| 661 | + | |
| 662 | + | |
| 663 | + | |
| 664 | + | |
| 665 | + | |
| 666 | + | |
| 667 | + | |
| 668 | + | |
| 669 | + | |
| 670 | + | |
| 671 | + | |
660 | 672 | | |
661 | 673 | | |
662 | 674 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
2 | 2 | | |
| 3 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| |||
0 commit comments