The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

let us know when it is ready for a review!
cc @asomoza

@yiyixuxu @asomoza I think it's ready for review :)

THIS NEEDS TO BE IN THE DOCS.

@stevhliu any ideas about the location?

Super cool! 🤩

We can add it to the "Specific Pipeline Examples" section and then build out the Flux doc there as discussed here.

@yiyixuxu I'm not sure what's the issue, when I run make fixup it doesnt make any changes / flag any issues 🤔

@linoytsaban It's the doc-builder check, try doc-builder style src/diffusers docs/source --max_len 119 on its own or make style, it's not covered with make fixup

thanks @hlky!

@linoytsaban let's add some docs (#10056 (comment)) and communicate?

效果不太好，并没有达到官方示例的效果

@Thekey756 do you have an example comparison to the original?

@Thekey756 I think to achieve the original effect you're referring to we need to also take in consideration - #10025

Example using prompts and attention masking for improved prompt adherence:

pipe_prior_output = pipe_prior_redux([image], prompt=["anime illustration"], 
                                     prompt_embeds_scale=[1.],
                                     pooled_prompt_embeds_scale=[1.]
                                    )

cond_size = 729
hidden_size = 4096
max_sequence_length = 512
full_attention_size = max_sequence_length + hidden_size + cond_size
attention_mask = torch.zeros(
    (full_attention_size, full_attention_size), device="cuda", dtype=torch.bfloat16
)
reference_scale: float = 0.04 # example
bias = torch.log(
    torch.tensor(reference_scale, dtype=torch.bfloat16, device="cuda").clamp(min=1e-5, max=1)
)
attention_mask[:, max_sequence_length : max_sequence_length + cond_size] = bias
joint_attention_kwargs=dict(attention_mask=attention_mask)

pipe(
    guidance_scale=2.5,
    height=1024,
    width=1024,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0),
    joint_attention_kwargs=joint_attention_kwargs,
    **pipe_prior_output,
).images[0]

Add the following to the flux redux prior pipeline:

* prompt inputs

* image interpolation

inference example:

image = Image.open("Self-portrait-oil-canvas-Thorn-Necklace-Hummingbird-Frida.jpg").convert("RGB")
image2 = Image.open("Mona_Lisa.jpg").convert("RGB")

pipe_prior_output = pipe_prior_redux([image,image2], prompt=["self portrait by frida khalo", "mona lisa"], 
                                     prompt_embeds_scale=[.9, .75],
                                     pooled_prompt_embeds_scale=[.6,1.25]
                                    )
pipe(
    guidance_scale=2.5,
    height=1024,
    width=1024,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0),
    **pipe_prior_output,
).images[0]

Can you give an example for your Fast FLUX.1 Redux in the space of huggingface? Thanks! And what is the masking scale?

how should I set prompt_embeds_scale, pooled_prompt_embeds_scale, and guidance_scale in general? @linoytsaban