EvalBaseModel extra='forbid' rejects custom fields in session_input and eval cases

[tottenjordan]

Description

EvalBaseModel in src/google/adk/evaluation/common.py uses extra="forbid" in its Pydantic model_config. This causes all eval-related models (EvalCase, SessionInput, IntermediateData, Invocation, ConversationScenario, etc.) to reject any fields not explicitly defined in the model.

This blocks common use cases like adding custom metadata to session_input for downstream analysis (e.g., category, tier, complexity tags on eval cases).

Reproduction

from google.adk.evaluation.eval_case import SessionInput

# This raises ValidationError because 'category' is not a defined field
session_input = SessionInput(
    app_name="my_app",
    user_id="eval_user",
    category="search",    # rejected
    tier="low",           # rejected
)

pydantic.ValidationError: 2 validation errors for SessionInput
category
  Extra inputs are not permitted [type=extra_forbidden]
tier
  Extra inputs are not permitted [type=extra_forbidden]

Expected Behavior

session_input and other eval models should accept additional fields. The ADK's own EvalConfig already uses extra="allow" — eval case models should follow the same pattern, or at minimum SessionInput should allow extra fields since it's the natural place for user-defined metadata.

Workaround

Currently we patch all EvalBaseModel subclasses at runtime:

from google.adk.evaluation import eval_case, eval_set

for mod in (eval_case, eval_set):
    for name in dir(mod):
        cls = getattr(mod, name)
        if isinstance(cls, type) and hasattr(cls, "model_config"):
            if cls.model_config.get("extra") == "forbid":
                cls.model_config["extra"] = "ignore"
                cls.__pydantic_complete__ = False

    for name in dir(mod):
        cls = getattr(mod, name)
        if isinstance(cls, type) and hasattr(cls, "model_rebuild"):
            cls.model_rebuild(force=True)

Affected Classes (as of 2.1.0)

All 11 classes inheriting from EvalBaseModel:

• eval_case.AppDetails
• eval_case.ConversationScenario
• eval_case.EvalBaseModel
• eval_case.EvalCase
• eval_case.IntermediateData
• eval_case.Invocation
• eval_case.InvocationEvent
• eval_case.InvocationEvents
• eval_case.Rubric
• eval_case.SessionInput
• eval_set.EvalCase

Suggested Fix

Change extra="forbid" to extra="ignore" (or extra="allow") in EvalBaseModel.model_config:

class EvalBaseModel(pydantic.BaseModel):
    model_config = pydantic.ConfigDict(
        alias_generator=alias_generators.to_camel,
        populate_by_name=True,
        extra="ignore",  # was "forbid"
        arbitrary_types_allowed=True,
    )

Environment

• google-adk: 1.34.0 (also verified on main/2.1.0)
• Python: 3.12

[surajksharma07]

@tottenjordan this makes sense — extra="forbid" on EvalBaseModel is stricter than it needs to be for user-facing eval models, and the inconsistency with BaseCriterion using extra="allow" is a good catch.

As a quick workaround before #5922 lands, try patching each affected class at import time by setting cls.model_config["extra"] = "ignore" and calling cls.model_rebuild(force=True).

Worth noting that each subclass holds its own copy of model_config, so the patch needs to be applied per class rather than just on EvalBaseModel.

Could you give it a thorough run against your eval pipeline and confirm it holds up before we move forward with the PR review?