Skip to content

user_guide.md

Latest commit

 

History

History
179 lines (139 loc) · 6.37 KB

user_guide.md

File metadata and controls

179 lines (139 loc) · 6.37 KB

User Manual

0. Table of Contents

1. How to Support New Models

Integrating a custom model requires 4 simple steps:

  1. Create a model class in app/models/your_model.py
  2. Register it in app/core/registry.py
  3. Configure it in configs/auto_labeling/your_model_id.yaml
  4. Enable it in configs/models.yaml

1.1 Create Model Class

Your model must inherit from BaseModel and implement three methods:

from . import BaseModel
from app.schemas.shape import Shape

class YourModel(BaseModel):
    def load(self):
        """Load model weights and initialize resources"""
        model_path = self.params.get("model_path")
        self.model = load_your_model(model_path)
    
    def predict(self, image, params):
        """Run inference and return results"""
        results = self.model(image)
        shapes = [Shape(label="...", shape_type="rectangle", points=[...])]
        return {"shapes": shapes, "description": ""}
    
    def unload(self):
        """Free resources on shutdown"""
        del self.model

Key Points:

  • Input image is in BGR format (OpenCV style)
  • Return a dict with shapes and description fields (see Response Schema)
  • Check app/models/ for complete implementation examples

1.2 Register Model

Just only use the @register_model decorator to register your model class.

...
from app.core.registry import register_model

@register_model("your_model_id")
class YourModel(BaseModel):
    # ... your implementation

You can also register multiple model IDs with the same class:

@register_model(
    "yolo11n", "yolo11s", "yolo11m", "yolo11l", "yolo11x"
)
class YOLO11Detection(BaseModel):
    # ... your implementation

Tip

  • The system automatically imports all modules in app/models/ directory (first level only)
  • Multiple model IDs can share the same class (e.g., yolo11n and yolo11s both use YOLO11nDetection)

1.3 Create Configuration

Create configs/auto_labeling/your_model_id.yaml:

model_id: your_model_id          # Required: Must be globally unique
display_name: "Your Model Name"  # Required: Displayed in X-AnyLabeling UI
batch_processing_mode: "default" # Optional: "default" or "text_prompt" (default: "default")
capabilities:                    # Optional: Module-scoped capability metadata
  ppocr_pipeline: true           # Example: consumed by PPOCR panel clients

params:                          # Optional: All params are passed to your model's __init__
  model_path: "path/to/weights.pt"
  device: "cuda:0"
  conf_threshold: 0.25
  # Add any custom parameters your model needs

widgets:
  - name: button_run
    value: null
  - name: edit_conf
    value: 0.25          # Widgets with ✅ must have default values
  - name: edit_iou
    value: 0.45
  - name: toggle_preserve_existing_annotations
    value: false
  ...

Capabilities Guidance:

  • capabilities is used for module-specific client routing rather than generic auto-labeling UI presentation.
  • If a model declares capabilities, clients may hide it from the general Remote-Server dropdown and surface it only in dedicated panels.
  • Use clear keys and stable semantics to keep client behavior predictable.

See Widget Reference for details.

1.4 Enable Model

Add to configs/models.yaml:

enabled_models:
  - yolo11n
  - yolo11n_seg
  - your_model_id  # Models will be displayed in this order
  # - yolo11s      # Comment out to disable a model

Note

  • Models are displayed in X-AnyLabeling UI in the order listed here
  • Comment out any model you don't want to load to save resources

2. Response Schema

Your model's predict() method must return a dictionary with the following fields:

{
    "shapes": [...],      # List of Shape objects (can be empty for caption tasks)
    "description": "...", # Text description (can be empty for detection tasks)
    "replace": None       # Optional: Boolean flag to replace existing annotations (excluded if None)
}

Each Shape object should have the following properties:

Field Type Required Description
label String ✅ Yes Category label of the object
shape_type String ✅ Yes Type of shape (see supported types below)
points List ✅ Yes List of [x, y] coordinates defining the shape vertices
score Float No Confidence score from model inference (default: None)
attributes Dict No Custom object attributes (default: {})
description String No Optional text description for the shape
difficult Boolean No Flag if object is difficult to identify (default: False)
direction Float No Direction in radians, 0-2π (default: 0)
flags Dict No Additional flags or metadata
group_id Integer No ID to group related shapes (e.g., pose keypoints)
kie_linking List No Key Information Extraction linking data (default: [])

Supported Shape Types:

  • rectangle: Horizontal bounding box defined by 4 corner points
  • polygon: Closed polygon with 3+ vertices
  • quadrilateral: Four-sided polygon with 4 vertices
  • rotation: Oriented/rotated bounding box
  • point: Single point coordinate
  • line: Line segment with start and end points
  • circle: Circle defined by center and radius point
  • linestrip: Connected line segments (polyline)

For detailed shape specifications, see the X-AnyLabeling User Guide and app/schemas/shape.py.

3. Troubleshooting

Error Solution
Model 'xxx' not registered Add to _build_registry() in registry.py
Configuration file not found Check YAML file exists and model_id matches filename
Widget 'edit_conf' requires a default value Set value: 0.25 in widgets config
Duplicate model_id found Each model_id must be unique