[Inference] Add Text-Generation support for Replicate provider #1813
Conversation
lucataco
requested review from
SBrandeis,
hanouticelina and
julien-c
as code owners
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (Array.isArray(value)) { | ||
| for (const item of value) { | ||
| const text = extractTextFromReplicateResponse(item); | ||
| if (typeof text === "string" && text.length > 0) { | ||
| return text; |
There was a problem hiding this comment.
The new extractTextFromReplicateResponse walks arrays and returns the first non‑empty string it finds. Replicate’s LLM responses are often arrays of chat messages where the first element is the user (or system) prompt and the assistant’s generated text appears later in the array. In such cases textGeneration will now echo the prompt instead of the model’s reply. Consider iterating from the end of the array or preferring elements with role === "assistant" so the final generated text is returned.
Useful? React with 👍 / 👎.
|
Hi @lucataco ! Those are the APIs used to power the chat widget on model pages and our chat API at router.huggingface.co for conversational models (such as GPT-5). If you support such APIs, adding support for them in the JS client should be very easy, see this for example: |
2 participants
This PR adds support for the
Text Generationtask type for Replicate models.Example: