Repository navigation
AutoRound: user-passed quantization_config silently overwrites checkpoint's quant settings #15120
Description
Describe the bug
DiffusersAutoQuantizer.merge_quantization_configs (in src/diffusers/quantizers/auto.py) is supposed to let users override runtime-only fields like backend on an AutoRoundConfig while keeping the checkpoint's own quantization settings. Instead, it loops over every field in the user-supplied AutoRoundConfig.__dict__ and overrides any field that differs from the checkpoint's value, not just backend.
The documented call is AutoRoundConfig(backend="auto"). That object still carries the dataclass defaults bits=4, group_size=128, sym=True. So loading a checkpoint that was quantized with, say, group_size=32 silently rewrites it to group_size=128 before convert_hf_model reads it — producing a shape mismatch at best, wrong (dequantized-as-the-wrong-grouping) outputs at worst.
Reproduction
from diffusers.quantizers.auto import DiffusersAutoQuantizer
from diffusers.quantizers.quantization_config import AutoRoundConfig
# Simulates the checkpoint's own config.json quantization_config
checkpoint_config = AutoRoundConfig(bits=4, group_size=32, sym=False, backend="torch")
# The documented way to override just the runtime backend
args_config = AutoRoundConfig(backend="auto")
merged = DiffusersAutoQuantizer.merge_quantization_configs(checkpoint_config, args_config)
print(merged.backend) # "auto" - expected
print(merged.group_size) # 128 <- should be 32, the checkpoint's own value
print(merged.sym) # True <- should be False, the checkpoint's own valueSystem Info
diffusers@main, commitaaf57d0- Python 3.12, Linux
Activity
Fix AutoRound checkpoint config being overwritten by user args
Fix NVIDIAModelOptConfig.to_dict() crashing on forward_loop callable
Fix from_single_file forcing pre_quantized=True on quantize-on-load b…
Fix quantizer configs being mutated in place across loads
Validate Nunchaku AWQ group_size against what the kernel supports
Fix broken __eq__/__repr__/hashing on quant configs, and to_json_stri…
Fix PipelineQuantizationConfig validation always raising
Fix bnb logging/error-message bugs and unsafe SCB key lookup
Fix AutoRound checkpoint config being overwritten by user args
Fix NVIDIAModelOptConfig.to_dict() crashing on forward_loop callable
Fix from_single_file forcing pre_quantized=True on quantize-on-load b…
Fix quantizer configs being mutated in place across loads
Validate Nunchaku AWQ group_size against what the kernel supports
Fix broken __eq__/__repr__/hashing on quant configs, and to_json_stri…
Fix PipelineQuantizationConfig validation always raising
Fix bnb logging/error-message bugs and unsafe SCB key lookup
[-]Several correctness bugs in src/diffusers/quantizers/[/-][+]AutoRound: user-passed quantization_config silently overwrites checkpoint's quant settings[/+]