2.0.0b2: robust bilayer fitting, thickness quality flag, and config defaults#59
Merged
Conversation
Mesh refinement sometimes mis-assigns the membrane center and the surface diverges. The dual-Gaussian thickness fit previously let the two leaflet peak positions float independently, so both could collapse onto the same leaflet and yield a bogus center/thickness. Re-parameterize the fit by (center, half_sep): the two leaflet peaks sit at center +/- half_sep, with half_sep bounded to a physical 2-7 nm range, so one Gaussian is forced into each leaflet. Adds _dual_gaussian_centered and a find_peaks-based _seed_bilayer_center seeder in _thickness_worker, promotes the fit-quality thresholds to module-level constants, restricts per-triangle fits to a window around the seeded center, and applies the same parameterization to the global average fit in refine_mesh. Work in progress: divergence issue still under investigation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Restrict the global-average dual-Gaussian fit to a window around the seeded bilayer center (like the per-triangle fit), so adjacent membranes / CTF ringing can't pull the global midpoint (the poor-fit fallback) off-center. - Add tests covering the center+half-separation model, the two-leaflet seeder, and that the windowed fit recovers the true bilayer center on an asymmetric bilayer + confounder (the "both Gaussians in one leaflet" failure) and leaves a well-centered bilayer unmoved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…solved Forcing the center+half_sep dual Gaussian (half_sep >= 1 nm) onto a single, poorly-resolved peak still centered correctly but reported a spurious ~2 nm bilayer. _seed_bilayer_center now counts prominent peaks; the per-triangle and global fits only attempt the dual Gaussian when two leaflets are actually resolved (n_peaks >= 2). One-peak regions fall through to the single-Gaussian step (method 2), which centers on the peak without a fake thickness. Tests cover the single-peak fallback. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…founders) Seed the bilayer by anchoring on the most prominent peak (the membrane) and pairing it with a partner peak that is a physical bilayer thickness (2-7 nm) away and prominent enough (>= 30% of the anchor). This: - rejects small flanking shoulders (neighboring protein density) by prominence, - rejects distant confounders (other membranes/protein > 7 nm) by separation, - keeps genuinely asymmetric bilayers as two leaflets. Regions with no qualifying partner resolve as a single peak -> single-Gaussian fallback. Tests cover the shoulder and asymmetric+confounder cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ow dip Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…membranes The density-profile bilayer fit (used by both mesh refinement and thickness measurement) had three problems that hurt real, well-behaved surfaces: * Leaflet detection used find_peaks prominence, which is asymmetric between the two halves of a bilayer: a hair-taller leaflet gets prominence measured to the solvent base while the other only reaches the inter-leaflet saddle, so their ratio is ~0.2 even for a symmetric bilayer. This made essentially every clean bilayer fall back to a single Gaussian. Detect leaflets by height above the shared solvent base instead (and enumerate all local maxima, no prominence floor), which also keeps a thin/merged bilayer's shallow-saddle second leaflet. * The dual model used independent leaflet widths, letting the fit absorb an asymmetric outer flank into unequal widths; because both peaks share one center, that width tilt shifted the fitted bilayer center ~0.1 nm to one side. Switch to a shared-width, centered (center + half-separation) model fit over a window symmetric about the seed, so the leaflets stay symmetric and the center sits on the true midpoint. * On a surface whose global average clearly resolves a bilayer, a locally-merged triangle was either force-fit into a spurious ~2 nm bilayer (mis-centering skewed peaks) or dropped entirely. Add a two-tier gate: strictly-resolved triangles fit and validate on R^2 + physical thickness; merged triangles are recovered from the global prior but accepted only when non-degenerate and center-stable (_dual_recovery_ok), so genuine single/skewed peaks are still rejected to the single Gaussian. measure_thickness now computes the global bilayer prior, applies the recovery tier, and reports a continuous per-triangle bilayer_resolution score (the height-above-base leaflet ratio) in the exported CSV -- a reliability flag that separates strictly-resolved (>= 0.5) from prior-recovered (slightly thin, < 0.5) measurements, since R^2 cannot. The whole-surface average fit and its _fit.svg plot use the same shared-width model so the published fit matches what drives the measurements. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…vergence Refinement stops early once a surface converges, so different surfaces can have different numbers of _refined_iter* files. When the requested STEP is not available for a surface, use the largest available iteration <= STEP and print a warning rather than skipping the surface. Also fix the stale "python run_pycurv.py" hint to the current "morphometrics pycurv" command. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…keys Config handling was scattered and inconsistent across commands: directory paths had to end in "/" or string-concatenated paths broke, and missing settings either raised a raw KeyError traceback (older commands using config["seg_dir"]) or were silently defaulted, with no uniform behavior. Add surface_morphometrics/config_utils.py with load_config(configfile, require=()): * normalize_dirs appends a trailing "/" to seg_dir / tomo_dir / work_dir when set but missing one, so a config written without trailing slashes (e.g. work_dir: ./morphometrics) behaves identically to one with them. * require_keys raises a single clean click.ClickException listing every missing or empty required setting (with its purpose), instead of a KeyError partway through a run. Route all 13 commands through load_config and declare per-command required keys: seg_dir+work_dir for the mesh/curvature/distance/stats steps, work_dir+tomo_dir for the tomogram steps (sample_density, refine_mesh) so a missing tomo_dir fails cleanly there, and work_dir only for the analysis/export steps. Remove the old per-command dir-validation blocks (the KeyError-prone bracket checks, redundant slash handling, and silent work_dir=seg_dir fallbacks); segmentation_to_meshes validates after its data_dir->seg_dir migration warning so migrating users still see the hint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…figs run
Add a single source of truth for default parameter values (config_utils.DEFAULTS,
mirroring config_template.yml) and merge it into every loaded config, so a
stripped-down config that omits parameter sections runs as documented instead of
raising a KeyError on the first bracket access. merge_defaults fills each section
shallowly (user-set values win, omitted params take their default), deep-copies so
the module DEFAULTS is never mutated, and preserves the legacy
density-sampling-under-thickness_measurements layout. A test keeps DEFAULTS in sync
with the template.
Where the code's old omitted-key defaults had drifted from the template (the
mesh_refinement settings: average_radius, average_radius_min, max_total_offset,
laplacian_iterations/lambda, convergence_threshold, xcorr_iterations), the
documented template values are now authoritative.
segmentation_values has no sensible default, so it is now required by the commands
that use it (make_meshes, stats, distances).
Distance/orientation measurements default to all-vs-all: omitting intra/inter
measures every surface against itself and every unordered pair against each other,
while an explicit empty (intra: [] / inter: {}) opts out. The command announces the
default and how to change it. resolve_distance_targets implements this and is unit
tested.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`morphometrics new_config` gains `--simple/--verbose` (default `--verbose`, the full annotated template as before). `--simple` writes config_template_simple.yml: just directories, segmentation_values, cores, the common surface_generation knobs (angstroms, isotropic_remesh, target_area, simplify, simplify_max_triangles), thickness average_radius, and refinement average_radius/iterations/xcorr_iterations. Every omitted setting falls back to the centralized defaults, so the minimal config is fully runnable; the command points the user to --verbose for all options. Ship the new template as package data and test that its values match DEFAULTS, that it loads with omitted sections filled, and that both flags write the right file. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Beta release 2.0.0b2. Builds on 2.0.0b1 with a substantial overhaul of the
density-profile bilayer fitting (used by both mesh refinement and thickness
measurement) plus a centralized, forgiving config system.
Bilayer fitting (refinement + thickness)
find_peaksprominence, which was asymmetric between the two halves of a bilayerand made essentially every clean bilayer fall back to a single Gaussian. Also keeps
a thin/merged bilayer's shallow-saddle second leaflet.
removes a ~0.1 nm systematic centering bias (the old independent-width fit absorbed
asymmetric baselines into unequal widths, shifting the center) and can't collapse
both peaks into one leaflet.
triangles that merge to one peak on a surface whose global average is a bilayer
are recovered from the global prior, accepted only when non-degenerate and
center-stable. Genuine single/skewed peaks still fall back to a single Gaussian.
_fit.svgplot use the same shared-widthmodel, so the published fit matches what drives the measurements.
Thickness measurement
triangles report
NaNinstead of a spurious value.bilayer_resolutionscore (0–1) in the exported CSV — areliability flag (R² is uninformative here) that separates strictly-resolved
(≥0.5) from prior-recovered (slightly thin, <0.5) measurements.
Refinement workflow
accept_refinementfalls back to the last available iteration (with a warning)when a surface converged before the requested step.
Config handling
config_utils): normalizes directory trailing slashes,validates per-command required keys with one clean error (no more
KeyErrortracebacks), and fills omitted parameter sections from a single source of truth
(
DEFAULTS, kept in sync with the template by a test) so partial configs run.with explicit-empty to opt out).
new_config --simple/--verbose: a minimal, fully-runnable starter config inaddition to the full annotated template.
Tests
36 unit tests (thickness fitting helpers + config) pass under pytest; the full
measure_thicknesspipeline was verified end-to-end on real data.🤖 Generated with Claude Code