Skip to content

Add decoder inference Colab (segmentation, depth, normals)#32

Open
bingyic wants to merge 2 commits into
mainfrom
add-decoder-inference-colab
Open

Add decoder inference Colab (segmentation, depth, normals)#32
bingyic wants to merge 2 commits into
mainfrom
add-decoder-inference-colab

Conversation

@bingyic
Copy link
Copy Markdown
Collaborator

@bingyic bingyic commented May 11, 2026

Summary

Adds a new Colab notebook demonstrating inference with all three TIPSv2 DPT decoder heads:

  1. Semantic Segmentation — ADE20K (150 classes) with color palette visualization
  2. Monocular Depth Estimation — NYU Depth V2 samples with turbo colormap
  3. Surface Normal Estimation — NYU samples with RGB normal map visualization

Notebook Structure

Section Description
Setup Install deps, clone repo
Download Checkpoints (vision encoder + 3 DPT heads) + sample images (ADE20K, NYU)
Load Models Vision encoder + SegmentationDecoder, DepthDecoder, NormalsDecoder
Inference Segmentation on ADE20K, Depth + Normals on NYU (3 sample images each)

Notes

  • Supports all variants: B, L, So, g (selectable via dropdown)
  • GPU runtime recommended for faster inference
  • Uses bicubic upsampling + L2 re-normalization for smooth surface normal maps

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant