I check the x-flux code, the single block lora should load after qkv attention，not norm,they are just has the same shape

@raulmosa could you give this a look?

I've checked it and it's right, it shouldn't be norm. Looks good to me @sayakpaul .
Nice catch @zhaowendao30 , thanks! =)

@zhaowendao30 thanks for your contribnutions!

Could you also do a side-by-side comparison of your changes applied and without your changes in the outputs? That would be very much appreciated.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I've checked it and it's right, it shouldn't be norm. Looks good to me @sayakpaul . Nice catch @zhaowendao30 , thanks! =)

No thanks =)

@zhaowendao30 thanks for your contribnutions!

Could you also do a side-by-side comparison of your changes applied and without your changes in the outputs? That would be very much appreciated.

OK, the single block i just trained index 1,2,3,4, and the first image is load in qkv, the second is load in norm, the last is no lora

@zhaowendao30 thanks, but please prefer not using human subjects in the public forums.

I will run the tests today and update the slices as needed because of the change.

@zhaowendao30 thanks, but please prefer not using human subjects in the public forums.

I will run the tests today and update the slices as needed because of the change.

OK, I've uploaded it again. However, this is a LoRA about people, and the panda looks a bit strange. =）

Just ran

pytest tests/lora/ -k "test_flux_xlabs"

test_flux_xlabs is passing and test_flux_xlabs_load_lora_with_single_blocks is failing because of hardware change, which is expected. I will change the slices in https://github.com/huggingface/diffusers/pull/9845/files.

@yiyixuxu a gentle ping.