fix torchao quantizer for new torchao versions by vkuzo · Pull Request #12901 · huggingface/diffusers

vkuzo · 2025-12-29T17:14:53Z

Summary:

torchao==0.16.0 (not yet released) has some bc-breaking changes, this PR fixes the diffusers repo with those changes. Specifics on the changes:

UInt4Tensor is removed: Remove unused UInt4Tensor pytorch/ao#3536
old float8 tensors v1 are removed: deprecate v1 of Float8WeightOnlyConfig|Float8DynamicActivationFloat8WeightConfig pytorch/ao#3510

In this PR:

move the logger variable up (not sure why it was in the middle of the file before) to get better error messages
gate the old torchao objects by torchao version

Test Plan:

import diffusers objects with new versions of torchao works:

> python -c "import torchao; print(torchao.__version__); from diffusers import StableDiffusionPipeline"
0.16.0.dev20251229+cu129

Reviewers:

Subscribers:

Tasks:

Tags:

What does this PR do?

Fixes # (issue)

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you read the contributor guideline?
Did you read our philosophy doc (important for complex PRs)?
Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
Did you make sure to update the documentation with your changes? Here are the
documentation guidelines, and
here are tips on formatting docstrings.
Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Summary: `torchao==0.16.0` (not yet released) has some bc-breaking changes, this PR fixes the diffusers repo with those changes. Specifics on the changes: 1. `UInt4Tensor` is removed: pytorch/ao#3536 2. old float8 tensors v1 are removed: pytorch/ao#3510 In this PR: 1. move the logger variable up (not sure why it was in the middle of the file before) to get better error messages 2. gate the old torchao objects by torchao version Test Plan: import diffusers objects with new versions of torchao works: ```bash > python -c "import torchao; print(torchao.__version__); from diffusers import StableDiffusionPipeline" 0.16.0.dev20251229+cu129 ``` Reviewers: Subscribers: Tasks: Tags:

vkuzo · 2025-12-29T17:16:07Z

cc @sayakpaul, would you know who can help review this?

sayakpaul

Thanks!

No other changes needed to ensure 0.16 release is smooth?

sayakpaul · 2025-12-30T03:59:51Z

@bot /style

github-actions · 2025-12-30T04:00:13Z

Style bot fixed some files and pushed the changes.

HuggingFaceDocBuilderDev · 2025-12-30T04:10:16Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

DN6 · 2025-12-30T13:43:51Z

@vkuzo Just a question. With the changes introduced in 0.16.0, loading quants serialized with something like uint4wo from older torchao versions would not work anymore right?

import torch
from diffusers import FluxTransformer2DModel, TorchAoConfig

# assuming this was saved using torchao<=0.15.0
transformer = FluxTransformer2DModel.from_pretrained(
    "hf-internal-testing/tiny-flux-transformer",
    quantization_config=TorchAoConfig("uint4wo"),
    torch_dtype=torch.bfloat16,
)
save_path = "<path to checkpoint>"
transformer.save_pretrained(save_path, safe_serialization=False)
del transformer

# this will break with 0.16.0 since Uint4Tensor is no longer in allowed safe globals?
FluxTransformer2DModel.from_pretrained(save_path)

vkuzo · 2025-12-30T15:44:13Z

With the changes introduced in 0.16.0, loading quants serialized with something like uint4wo from older torchao versions would not work anymore right?

Yes, that's correct! For test models, can we just regenerate them using the new v2 configs? I'm actually not sure which specific config is the new version of uint4wo but when @jerryzh168 is back from holiday he can chime in. If there are user models that need to be migrated and cannot be regenerated, we can help with a one-off migration script if needed.

vkuzo · 2025-12-30T15:46:01Z

No other changes needed to ensure 0.16 release is smooth?

The branch cut did not happen yet, I just made this fix for things we know about so far to unblock some internal work. We can follow up in a separate PR if there are any more bc-breaking changes before the branch cut.

By the way, we are adding perf+accuracy benchmarks with diffusers to torchao here: pytorch/ao#3502

sayakpaul · 2025-12-30T15:56:42Z

That looks great, just reviewed and commented! Thanks so much for working on it.

* fix torchao quantizer for new torchao versions Summary: `torchao==0.16.0` (not yet released) has some bc-breaking changes, this PR fixes the diffusers repo with those changes. Specifics on the changes: 1. `UInt4Tensor` is removed: pytorch/ao#3536 2. old float8 tensors v1 are removed: pytorch/ao#3510 In this PR: 1. move the logger variable up (not sure why it was in the middle of the file before) to get better error messages 2. gate the old torchao objects by torchao version Test Plan: import diffusers objects with new versions of torchao works: ```bash > python -c "import torchao; print(torchao.__version__); from diffusers import StableDiffusionPipeline" 0.16.0.dev20251229+cu129 ``` Reviewers: Subscribers: Tasks: Tags: * Apply style fixes --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

…ixes huggingface#13104) Add a test to prevent regression of the NameError that occurred when `logger` was used inside `_update_torch_safe_globals()` before being defined at module level. The fix (moving `logger` before the function) was included in PR huggingface#12901, but this test ensures it cannot regress.

sayakpaul approved these changes Dec 30, 2025

View reviewed changes

Apply style fixes

8513570

sayakpaul merged commit 1cdb872 into huggingface:main Dec 30, 2025
10 of 11 checks passed

dhruvildarji mentioned this pull request Feb 23, 2026

tests: add regression test for logger NameError in torchao_quantizer #13175

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix torchao quantizer for new torchao versions#12901

fix torchao quantizer for new torchao versions#12901
sayakpaul merged 2 commits intohuggingface:mainfrom
vkuzo:20251229_fix_torchao_float8_import

vkuzo commented Dec 29, 2025 •

edited

Loading

Uh oh!

vkuzo commented Dec 29, 2025

Uh oh!

sayakpaul left a comment

Uh oh!

sayakpaul commented Dec 30, 2025

Uh oh!

github-actions bot commented Dec 30, 2025 •

edited

Loading

Uh oh!

HuggingFaceDocBuilderDev commented Dec 30, 2025

Uh oh!

Uh oh!

DN6 commented Dec 30, 2025 •

edited

Loading

Uh oh!

vkuzo commented Dec 30, 2025

Uh oh!

vkuzo commented Dec 30, 2025

Uh oh!

sayakpaul commented Dec 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

vkuzo commented Dec 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Before submitting

Who can review?

Uh oh!

vkuzo commented Dec 29, 2025

Uh oh!

sayakpaul left a comment

Choose a reason for hiding this comment

Uh oh!

sayakpaul commented Dec 30, 2025

Uh oh!

github-actions bot commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Dec 30, 2025

Uh oh!

Uh oh!

DN6 commented Dec 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vkuzo commented Dec 30, 2025

Uh oh!

vkuzo commented Dec 30, 2025

Uh oh!

sayakpaul commented Dec 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vkuzo commented Dec 29, 2025 •

edited

Loading

github-actions bot commented Dec 30, 2025 •

edited

Loading

DN6 commented Dec 30, 2025 •

edited

Loading