Image Frame Stitching

Samsung PRISM · MIT MMS14 Project

A real-time image stitching tool that takes overlapping photos from multiple cameras (or a single camera panned across a scene) and combines them into a single panoramic frame.

Built originally as part of the Samsung PRISM research program at MIT, and re-implemented here as an interactive web demo using modern Python tooling.

---

What it does

You feed in 2–4 overlapping images, and the pipeline:

1. Detects keypoints in each image using ORB (Oriented FAST and Rotated BRIEF)
2. Matches corresponding keypoints between adjacent images using Brute Force matching with Hamming distance
3. Estimates the homography (perspective transform) between each pair using RANSAC
4. Warps and composites the images into a single stitched panorama
5. Crops out black border artifacts left by the warping

The web app shows you each stage — detected keypoints, matched feature pairs, and the final result — and lets you tune the algorithm parameters live.

---

Demo

streamlit run streamlit_app.py

Open http://localhost:8502 in your browser.

Upload tab — drag in 2–4 overlapping photos, click Stitch. Webcam tab — take 2–4 photos with your webcam (pan slightly between shots), then stitch.

Tips for good results

• Overlap between adjacent frames: 20–40% works well
• Keep the camera at roughly the same height and level
• Scenes with lots of texture and distinct features stitch better than flat walls or sky
• If stitching fails, try increasing Max keypoints in the sidebar

---

Setup

git clone https://github.com/DoomDust7/Samsung-PRISM-Project.git
cd Samsung-PRISM-Project
pip install -r requirements.txt
streamlit run streamlit_app.py

Requirements: Python 3.9+

streamlit>=1.43.0
opencv-python-headless>=4.9.0
numpy>=1.26.0
Pillow>=10.0.0

opencv-python-headless is used instead of opencv-python so the app runs server-side without display dependencies (works on Streamlit Community Cloud too).

---

Project structure

.
├── streamlit_app.py          # entry point
├── requirements.txt
├── .streamlit/
│   └── config.toml           # dark theme
└── app/
    ├── core/
    │   ├── config.py         # StitcherConfig dataclass — all tunable params
    │   ├── detector.py       # ORB feature detection
    │   ├── matcher.py        # BF / KNN feature matching
    │   ├── stitcher.py       # homography + warp + crop
    │   └── pipeline.py       # orchestrates detect → match → stitch for N images
    ├── ui/
    │   ├── sidebar.py        # parameter controls
    │   ├── upload_tab.py     # file upload flow
    │   ├── webcam_tab.py     # webcam capture flow
    │   └── components.py     # shared result display widgets
    └── utils/
        ├── image_utils.py    # load / encode / resize / draw helpers
        └── logging_config.py

---

How the algorithm works

Feature detection — ORB

ORB (Rublee et al., 2011) is a fast, patent-free alternative to SIFT and SURF. It combines:

• FAST corner detection to find candidate keypoints
• BRIEF descriptors, rotated to be orientation-invariant
• A scale pyramid to handle size differences between frames

For each image, ORB returns a list of keypoints (x, y, angle, scale) and a 256-bit binary descriptor per keypoint.

Feature matching — Brute Force / Hamming distance

ORB descriptors are binary, so the right distance metric is Hamming distance (number of differing bits). The default mode uses cross-check matching — a match is only kept if A→B and B→A agree on the same pair. Optionally, KNN matching (k=2) with the Lowe ratio test can be enabled — useful for noisier images.

Matches are sorted by distance and the top N% are kept (the Good match % slider).

Homography estimation — RANSAC

Given matched point pairs, cv2.findHomography estimates the 3×3 projective transformation matrix H that maps points in image A to their corresponding positions in image B. RANSAC is used to reject outlier matches — points that don't fit a consistent geometric transform.

The RANSAC threshold slider sets how many pixels of reprojection error are tolerated before a match is considered an outlier.

Warping and compositing

cv2.warpPerspective applies H to warp image A into image B's coordinate frame. The two images are composited by placing warped A on a canvas and pasting B on top (B acts as the fixed anchor). For more than 2 images, this is done progressively: each new image is stitched onto the accumulated result from the previous step.

Black regions introduced by warping are removed by finding the largest contour in the binary mask of the result and cropping to its bounding rectangle.

---

Tuning parameters

Parameter	What it does	When to change
Max keypoints	How many ORB features to detect per image	Increase if stitching fails on low-texture images
Good match %	Fraction of top matches to keep	Lower = stricter, fewer but better matches
Lowe ratio test	Switch to KNN + ratio test matching	Try if default matching gives too few good matches
RANSAC threshold	Max pixel error for an inlier	Increase for images with noticeable perspective distortion
Crop black borders	Remove warping artifacts	Keep on unless you want the full warped canvas

---

What changed from the original

The original submission was a single Python script (Stitcher Code.py) that read video files from a local folder and showed output via cv2.imshow. It had no error handling, hardcoded paths, imports at the bottom of the file, and required the raw video dataset to run.

This version:

• Replaces cv2.imshow with a Streamlit web UI
• Restructures into a proper Python package
• Adds type hints, error handling, and structured logging
• Drops the imutils dependency (replaced with direct OpenCV 4.x idioms)
• Adds interactive parameter controls and step-by-step visualizations
• Adds webcam capture mode and PNG export
• Works entirely on uploaded images — no dataset required

---

References

• Rublee, E., Rabaud, V., Konolige, K., Bradski, G. (2011). ORB: An efficient alternative to SIFT or SURF. ICCV.
• Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. IJCV.
• Fischler, M. A., Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting. CACM.
• OpenCV docs: Feature Detection and Description