Add GenAi Models to AI Tutor Tester - #60278
Merged
Merged
Conversation
Erin007
changed the title
Contributor
Author
ebeastlake
reviewed
Aug 9, 2024
| for (let i = 0; i < data.length; i++) { | ||
| if (selectedEndpoint === 'llm-guard') { | ||
| getLLMGuardToxicity(data[i]); | ||
| } else if (genAIEndpointIds.includes(selectedEndpoint)) { |
Contributor
There was a problem hiding this comment.
If you use a concurrency method like Promise.allSettled, you can do something like:
const getAIResponses = async () => {
setResponsesPending(true);
const responsePromises = data.map(async (row) => {
if (selectedEndpoint === 'llm-guard') {
return getLLMGuardToxicity(row);
} else if (genAIEndpointIds.includes(selectedEndpoint)) {
return getGenAIResponses(row);
} else {
return askAITutor(row);
}
});
const results = await Promise.allSettled(responsePromises);
setResponsesPending(false);
};
I think it makes how you're handling the asynchronicity a little more explicit 🤷♀️
It also allows you to easily filter for successes and failures, if desired.
// Count successful responses
const successfulResponses = results.filter(result => result.status === 'fulfilled').length;
See: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/allSettled
ebeastlake
reviewed
Aug 10, 2024
| status: 'ok', | ||
| timestamp: new Date().getTime(), | ||
| }; | ||
| const temperature = row.temperature ? row.temperature : 0.8; |
Contributor
There was a problem hiding this comment.
Nit: Could we make this a constant and add a comment that it's the default temperature used in levelbuilder?
ebeastlake
reviewed
Aug 10, 2024
| aiCustomizations, | ||
| aichatContext | ||
| ); | ||
| row.aiResponse = genAIResponse.messages[1].chatMessageText; |
Contributor
There was a problem hiding this comment.
Should we add optional chaining here?
ebeastlake
reviewed
Aug 10, 2024
| id: 'gen-ai-mistral-pirate-7b', | ||
| name: 'Mistral Pirate + Webpurify', | ||
| }, | ||
| ]; |
Contributor
There was a problem hiding this comment.
Non-blocking nit: I'd consider making this a little more "constant-y" (is that a word? 🙃), a la...
// Non-Gen AI endpoint constants
const AI_TUTOR_ENDPOINT = 'ai-tutor';
const LLM_GUARD_ENDPOINT = 'llm-guard';
// Gen AI endpoint constants
const GEN_AI_MISTRAL_BASE = 'gen-ai-mistral-7b-inst-v01';
const GEN_AI_ARITHMO = 'gen-ai-arithmo2-mistral-7b';
const GEN_AI_BIOMISTRAL = 'gen-ai-biomistral-7b';
const GEN_AI_KAREN_CREATIVE = 'gen-ai-karen-creative-mistral-7b';
const GEN_AI_PIRATE = 'gen-ai-mistral-pirate-7b';
const endpoints = [
{
id: AI_TUTOR_ENDPOINT,
name: 'AI Tutor + Webpurify',
},
{
id: LLM_GUARD_ENDPOINT,
name: 'LLM Guard',
},
];
const genAIEndpoints = [
{
id: GEN_AI_MISTRAL_BASE,
name: 'Mistral Base + Webpurify',
},
{
id: GEN_AI_ARITHMO,
name: 'Mistral Arithmo + Webpurify',
},
{
id: GEN_AI_BIOMISTRAL,
name: 'Mistral Biomistral + Webpurify',
},
{
id: GEN_AI_KAREN_CREATIVE,
name: 'Mistral Karen + Webpurify',
},
{
id: GEN_AI_PIRATE,
name: 'Mistral Pirate + Webpurify',
},
];
...
Contributor
|
Nice work! 🎉 Just a couple small comments. |
ebeastlake
approved these changes
Aug 13, 2024
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the GenAI Models as endpoint options for the AI Tutor Tester.
Supported columns now update based on the endpoint selected.
Links
CT-964 partial
Similar to #60224
Testing story
I played around with a few combinations of uploaded columns + the GenAi models with small datasets and things seem to be working. Given that this is in an internal tool, I'm going to be relying on @samantha-code to let me know if things aren't working as expected for her 😁
Here's an example with the Pirate 🏴☠️ model:
tester_genai_models.mov
Deployment strategy
You need to be a levelbuilder with ai tutor access permission to access the Tester. Additionally, you need to be in the gen ai pilot to be able to hit the GenAi endpoints so this feels well protected from the public; no concerns about when/how it's deployed.
Follow-up work
There are more details in the linked Jira ticket. The next tasks are to get call LLMGuard in Python rather then the EC2 instance and then then get LLMGuard working with the GenAi models.
PR Checklist: