HuggingFaceM4/Idefics3-8B-Llama3 crash fix#3267
Conversation
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
|
following is the crash in XPU 2025-06-16T01:25:16.438091Z INFO text_generation_launcher: image_id 0 start_idx 0 end_idx 2314, length 2314 [W616 01:23:57.772114900 OperatorEntry.cpp:154] Warning: Warning only once for all operators, other operators may also be overridden. |
drbh
left a comment
drbh
left a comment
There was a problem hiding this comment.
LGTM in general, but maybe the scaling logic can be reduced a bit?
possibly something with less branching like:
let max_longest_edge_for_image_resize = config.get_max_longest_edge_for_image_resize();
let max_image_size = config.get_max_image_size();
let (height, width) = {
let h = height as f32;
let w = width as f32;
// First resize to max_longest_edge (always scale to this size)
let scale1 = max_longest_edge / h.max(w);
let (h, w) = (h * scale1, w * scale1);
// Ensure we dont exceed max_size (only scale down)
let scale2 = (max_size / h.max(w)).min(1.0);
((h * scale2) as usize, (w * scale2) as usize)
};
Signed-off-by: Wang, Yi A <yi.a.wang@intel.com>
|
update, thanks for the review @drbh |
5 participants
server:
text-generation-launcher --model-id=HuggingFaceM4/Idefics3-8B-Llama3 -p 8080
client
What is in the picture?\n\n","parameters":{"max_new_tokens":100, "seed": 42, "do_sample":true}}'
curl -N 0.0.0.0:8080/generate_stream
-X POST
-d '{"inputs":"
-H 'Content-Type: application/json'
crash happens because slot allocated in rust is not large enough.
gaudi and XPU has the same issue, assume cuda has same issue too.
the image process logic is copied from https://github.com/huggingface/transformers/blob/main/src/transformers/models/idefics3/image_processing_idefics3.py#L118