Support messages with images in prepare_multimodal_messages

[albertvillanova]

Support messages with images in prepare_multimodal_messages.

This PR enhances the handling of multimodal messages by ensuring that existing image payloads are preserved and only unfilled placeholders are populated, preventing accidental overwrites. Additionally, the test suite is expanded to cover this behavior, and prompt processing is streamlined in the trainer.

See related comment in:

• Support multimodal tool responses in environment_factory for VLM training #5323 (comment)

CC: @sergiopaniego

Changes

Multimodal message handling improvements:

• Updated prepare_multimodal_messages to preserve existing image payloads in image blocks and only fill placeholders without an "image" key.

Testing enhancements:

• Added a new test, test_prepared_image_blocks_without_new_images, to verify that existing image payloads are not overwritten when no new images are provided.

Trainer integration:

• Simplified prompt normalization for VLMs by directly using prepare_multimodal_messages, ensuring consistent handling of multimodal content during tokenization.

---

Note

Low Risk
Small, localized change to multimodal message preparation and a trainer call-site refactor; main risk is subtle behavioral differences in placeholder counting for edge-case message formats.

Overview
prepare_multimodal_messages now preserves existing image blocks that already carry an "image" payload, and only counts/fills unfilled {"type": "image"} placeholders from the images argument (avoiding accidental overwrites and placeholder-count mismatches).

Adds a regression test ensuring prepared image payloads remain intact when images is omitted, and simplifies GRPO trainer VLM prompt normalization by delegating to prepare_multimodal_messages instead of ad-hoc string wrapping.

^{Reviewed by Cursor Bugbot for commit 24bcdd1. Bugbot is set up for automated code reviews on this repo. Configure here.}

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 414deb9. Configure here.}

[sergiopaniego]

Thanks for the update @albertvillanova!
I think that once the merge conflicts are resolved and the problem raised by Cursor in grpo_trainer.py is solved, we're good to go 😄

[albertvillanova]

This PR requires (see #5474 (comment)):

• Fix prepare_multimodal_messages not normalizing empty string content for assistant/tool roles #5496

[qgallouedec]

I think it looks good!

[qgallouedec]

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Chef's kiss.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".