Skip to content

Add ask_vlm method for cloud VLM alert verification#442

Draft
srnangi wants to merge 2 commits into
mainfrom
feature/vlm-verification-sdk
Draft

Add ask_vlm method for cloud VLM alert verification#442
srnangi wants to merge 2 commits into
mainfrom
feature/vlm-verification-sdk

Conversation

@srnangi

@srnangi srnangi commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

What

Adds Groundlight.ask_vlm(images, query, model_id) which verifies one or two images against a natural-language query by calling the new POST /v1/vlm-queries cloud endpoint. Returns a VLMVerificationResult dataclass with verdict (YES/NO/UNSURE), confidence, reasoning, and token cost.

Pairs with the janzu endpoint (separate PR). The VLM runs in the Groundlight cloud (Bedrock) — no local inference.

How

  • Accepts a single image or [full_frame, roi] for the dual-image strategy, reusing parse_supported_image_types for encoding (numpy BGR, PIL, bytes, filename, etc.).
  • POSTs multipart images parts with query/model_id as query params, using the existing requests-based pattern from internalapi.py.
  • Moves import requests to module level.
  • Exports VLMVerificationResult from the package.

Usage

gl = Groundlight()
result = gl.ask_vlm(images=[full_frame, roi], query="Is there a fire?")
if result.verdict == "YES":
    emit_alert()
print(result.confidence, result.reasoning, result.total_cost_usd)

Testing

  • 7 unit tests with mocked HTTP (image encoding, dual-image, model_id passthrough, >2-image guard, result parsing) — all passing.

🤖 Generated with Claude Code

buildci and others added 2 commits June 17, 2026 17:01
Add Groundlight.ask_vlm(images, query, model_id) which verifies one or two
images against a natural-language query by calling POST /v1/vlm-queries.
Returns a VLMVerificationResult dataclass with verdict (YES/NO/UNSURE),
confidence, reasoning, and token cost.

- Accepts a single image or [full_frame, roi] for the dual-image strategy,
  reusing parse_supported_image_types for encoding.
- Moves the requests import to module level.
- Exports VLMVerificationResult from the package.
- Unit tests with mocked HTTP.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant