Skip to content

validate & status: pre-flight checks and a dataset progress report#60

Merged
bbarad merged 3 commits into
mainfrom
validate-command
Jun 28, 2026
Merged

validate & status: pre-flight checks and a dataset progress report#60
bbarad merged 3 commits into
mainfrom
validate-command

Conversation

@bbarad

@bbarad bbarad commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Two read-only utility commands for setting up and tracking a dataset.

morphometrics validate config.yml

Pre-flights a config + folder setup before a run:

  • seg_dir exists (error) and tomo_dir exists (warning only — tomograms are needed only for density sampling, thickness, and refinement);
  • each segmentation has a matching tomogram under the pipeline's shared-prefix naming, and flags tomograms with no matching segmentation — so a name mismatch shows up here instead of as a silently skipped tomogram;
  • which configured segmentation_values labels actually appear in each segmentation (flags segmentations where none appear, or where an unmapped label value is present).

Exits non-zero on errors; warnings do not fail. Label scan reads slice-by-slice via mmap (~0.6 s for a 1000×1000×200 segmentation).

morphometrics status config.yml

Prints a per-segmentation, per-surface summary of what has been computed — mesh, curvature, refinement (and the accepted iteration), and which self/inter distances, verticality, and thickness measurements exist. Everything is derived from the files in work_dir and the per-surface CSV column headers, so it always reflects reality with no state to maintain. --output PATH also writes the summary to a text file.

YTC041_1_lam4_2_ts_002_labels   [tomogram ✓]
  IMM   mesh ✓  curv ✓  refine accepted@iter4  self-dist ✓  vert ✓  inter OMM  thickness ✓
  OMM   mesh ✓  curv ✓  refine accepted@iter4  self-dist ✓  vert ✓  inter IMM  thickness ✓

The one non-derivable piece — which refinement iteration was accepted — is now recorded by accept_refinement as a one-line {surface}.accepted_iter marker; older datasets without it report accepted (step unknown) from the .orig.bak backups.

Notes

  • Both are light Setup subcommands (no graph-tool); registered in --help.
  • Tests: 11 (6 status incl. the accepted-iter marker + dry-run, 5 validate).
  • Docs already landed on main (commit d423a8e) ahead of this merge.

🤖 Generated with Claude Code

bbarad and others added 3 commits June 27, 2026 11:54
…ders

`validate config.yml` checks setup before a run:
- seg_dir exists (error) and tomo_dir exists (warning only -- tomograms are needed
  only for density sampling, thickness, and refinement);
- each segmentation has a matching tomogram under the pipeline's prefix convention
  (the tomogram name is a prefix of the segmentation name), so a name mismatch shows
  up here rather than as a silently skipped tomogram later;
- which configured segmentation_values labels actually appear in each segmentation,
  flagging segmentations where none of them appear or where an unmapped label value
  is present.

Exits non-zero on errors; warnings do not fail. Registered as a Setup subcommand.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the reverse (tomogram -> segmentation) direction: a Tomograms section warns
about any tomogram whose name does not prefix any segmentation, catching name
mismatches from the other side. Test added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ntation

`status config.yml` scans the work_dir and prints, per segmentation and surface,
which steps have run -- mesh, curvature, refinement (and the accepted iteration),
and which self/inter distances, verticality, and thickness measurements exist. It
is fully derived from the artifacts on disk and the per-surface CSV column headers,
so it always reflects reality with no separate state to maintain. --output also
writes the summary to a text file.

The one non-derivable piece -- which refinement iteration was accepted -- is now
recorded by accept_refinement as a one-line {surface}.accepted_iter marker; without
it (older datasets) status reports "accepted (step unknown)" from the .orig.bak
backups.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@bbarad bbarad merged commit e7f8683 into main Jun 28, 2026
2 checks passed
@bbarad bbarad deleted the validate-command branch June 28, 2026 00:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant