Skip to content

fix(qwen): fix CFG failing when passing neg prompt embeds with none mask #13379

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sayakpaul merged 12 commits into from
Apr 17, 2026
Merged

fix(qwen): fix CFG failing when passing neg prompt embeds with none mask #13379

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
dbe4f95
fix(qwen): fix CFG failing when passing neg prompt embeds with none mask
Sunhill666 Apr 1, 2026
d7814c3
Merge branch 'huggingface:main' into fix/qwen-image-cfg-mask
Sunhill666 Apr 13, 2026
f600a36
fix(qwen): safely handle missing embeds masks in edit and inpaint pip…
Sunhill666 Apr 13, 2026
5b2a1d2
test(qwen): add tests for true cfg scale without neg prompt mask
Sunhill666 Apr 13, 2026
c774885
Merge branch 'main' into fix/qwen-image-cfg-mask
sayakpaul Apr 16, 2026
908c304
Merge branch 'main' into fix/qwen-image-cfg-mask
Sunhill666 Apr 16, 2026
eb86a2e
fix(qwen): correct comments for copied functions in controlnet and in…
Sunhill666 Apr 16, 2026
42587ae
fix(qwen): add warnings for missing prompt and negative prompt masks …
Sunhill666 Apr 16, 2026
7fa33ee
test(qwen): use torch_device and clarify dummy inputs in cfg mask tests
Sunhill666 Apr 16, 2026
17938f5
fix(qwen): address Claude PR review feedback
Sunhill666 Apr 17, 2026
30e10dd
fix(qwen): fix warning message based on reviewer suggestion
Sunhill666 Apr 17, 2026
4f736f7
Merge branch 'main' into fix/qwen-image-cfg-mask
Sunhill666 Apr 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -584,9 +584,7 @@ def __call__(

device = self._execution_device

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None
Copy link
Copy Markdown
Member

@sayakpaul sayakpaul Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we handle negative_prompt_embeds_mask? 👀 Is this block affected?

diffusers/src/diffusers/pipelines/qwenimage/pipeline_qwenimage.py

Lines 261 to 267 in 71a6fd9

if prompt_embeds_mask is not None:
prompt_embeds_mask = prompt_embeds_mask[:, :max_sequence_length]
prompt_embeds_mask = prompt_embeds_mask.repeat(1, num_images_per_prompt, 1)
prompt_embeds_mask = prompt_embeds_mask.view(batch_size * num_images_per_prompt, seq_len)
if prompt_embeds_mask.all():
prompt_embeds_mask = None

Copy link
Copy Markdown
Contributor Author

@Sunhill666 Sunhill666 Apr 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If negative_prompt_embeds_mask isn't provided (is None), that block is safely skipped, and encode_prompt just returns None for the mask. The transformer model later receives this None mask and handles it natively by treating all tokens as valid.

Interestingly, while working on this, I found that some other variants (like edit and inpaint) were actually missing this exact if prompt_embeds_mask is not None: check and were crashing on .repeat(). I've added the same check to those pipelines in this PR as well so they all handle None masks gracefully now.


if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_controlnet.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -700,9 +700,7 @@ def __call__(

device = self._execution_device

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_controlnet_inpaint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -739,9 +739,7 @@ def __call__(

device = self._execution_device

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None
do_true_cfg = true_cfg_scale > 1 and has_neg_prompt
prompt_embeds, prompt_embeds_mask = self.encode_prompt(
prompt=prompt,
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_edit.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -705,9 +705,7 @@ def __call__(
image = self.image_processor.preprocess(image, calculated_height, calculated_width)
image = image.unsqueeze(2)

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_edit_inpaint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -878,9 +878,7 @@ def __call__(
)
image = image.to(dtype=torch.float32)

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_edit_plus.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -693,9 +693,7 @@ def __call__(
condition_images.append(self.image_processor.resize(img, condition_height, condition_width))
vae_images.append(self.image_processor.preprocess(img, vae_height, vae_width).unsqueeze(2))

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_img2img.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -677,9 +677,7 @@ def __call__(

device = self._execution_device

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_inpaint.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -822,9 +822,7 @@ def __call__(

device = self._execution_device

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
4 changes: 1 addition & 3 deletions src/diffusers/pipelines/qwenimage/pipeline_qwenimage_layered.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -697,9 +697,7 @@ def __call__(
else:
batch_size = prompt_embeds.shape[0]

has_neg_prompt = negative_prompt is not None or (
negative_prompt_embeds is not None and negative_prompt_embeds_mask is not None
)
has_neg_prompt = negative_prompt is not None or negative_prompt_embeds is not None

if true_cfg_scale > 1 and not has_neg_prompt:
logger.warning(
Expand Down
Loading