Skip to content

Commit 5387535

Browse files
author
Haiping Chen
committed
Unify the LLM Provider Settings SciSharp#234 
1 parent ab913b0 commit 5387535

39 files changed

Lines changed: 251 additions & 153 deletions

File tree

‎docs/agent/intro.md‎

Lines changed: 12 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -16,14 +16,19 @@ Suppose we need to write a Pizza restaurant order AI Bot. First, specify a name
1616
BotSharp uses the latest large language model in natural language understanding, can interact with OpenAI's ChatGPT, and also supports the most widely used open source large language model [LLaMA](https://ai.meta.com/blog/large-language-model-llama-meta-ai/) and its fine-tuning model. In this example, we use [Azure OpenAI](https://azure.microsoft.com/en-us/products/ai-services/openai-service) as the LLM engine.
1717

1818
```json
19-
"AzureOpenAi": {
20-
"ApiKey": "",
21-
"Endpoint": "",
22-
"DeploymentModel": {
23-
"ChatCompletionModel": "",
24-
"TextCompletionModel": ""
19+
"LlmProviders": [
20+
{
21+
"Provider": "azure-openai",
22+
"Models": [{
23+
"Name": "gpt-35-turbo",
24+
"ApiKey": "",
25+
"Endpoint": "https://gpt-35-turbo.openai.azure.com/",
26+
"Type": "chat",
27+
"PromptCost": 0.0015,
28+
"CompletionCost": 0.002
29+
}]
2530
}
26-
}
31+
]
2732
```
2833

2934
If you use the installation package to run, please ensure that the [BotSharp.Plugin.AzureOpenAI](https://www.nuget.org/packages/BotSharp.Plugin.AzureOpenAI) plugin package is installed.

‎docs/conf.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -64,9 +64,9 @@
6464
# built documents.
6565
#
6666
# The short X.Y version.
67-
version = '0.20'
67+
version = '0.21'
6868
# The full version, including alpha/beta/rc tags.
69-
release = '0.20.0'
69+
release = '0.21.0'
7070

7171
# The language for content autogenerated by Sphinx. Refer to documentation
7272
# for a list of supported languages.

‎docs/quick-start/installation.md‎

Lines changed: 23 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,29 @@ PS D:\> dotnet build
1919
`BotSharp` can work with serveral LLM providers. Update `appsettings.json` in your project. Below config is tasking Azure OpenAI as the LLM backend
2020

2121
```json
22-
"AzureOpenAi": {
23-
"ApiKey": "",
24-
"Endpoint": "https://xxx.openai.azure.com/",
25-
"DeploymentModel": {
26-
"ChatCompletionModel": "",
27-
"TextCompletionModel": ""
28-
}
29-
}
22+
"LlmProviders": [
23+
{
24+
"Provider": "azure-openai",
25+
"Models": [
26+
{
27+
"Name": "gpt-35-turbo",
28+
"ApiKey": "",
29+
"Endpoint": "https://gpt-35-turbo.openai.azure.com/",
30+
"Type": "chat",
31+
"PromptCost": 0.0015,
32+
"CompletionCost": 0.002
33+
},
34+
{
35+
"Name": "gpt-35-turbo-instruct",
36+
"ApiKey": "",
37+
"Endpoint": "https://gpt-35-turbo-instruct.openai.azure.com/",
38+
"Type": "text",
39+
"PromptCost": 0.0015,
40+
"CompletionCost": 0.002
41+
}
42+
]
43+
}
44+
]
3045
```
3146

3247
### Run backend web project

‎docs/quick-start/overview.md‎

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -21,15 +21,4 @@ Even with this simple question, you can see conversational experience are hard t
2121

2222
Your code would have to handle all these different types of requests ro carry out the same logic: looking up some forecast information for a feature. For this reason, a traditional computer interface would tend to force users to input a well-known, standard request at the detriment of the user experience, because it's just easier.
2323

24-
However, BotSharp lets you easily achieve a conversational user experience by handling the natural language understanding (NLU) for you. When you use BotSharp, you can create agents that can understand the meaning of natural language and the nuances and trainslate that to structured meaning your software can understand.
25-
26-
Features
27-
-------------
28-
29-
* Built-in multi-Agents management, easy to build Bot as a Service platform.
30-
* Integrate with multiple LLMs like ChatGPT and LLaMA.
31-
* Using plug-in design, it is easy to expand functions.
32-
* Working with multiple Vector Stores for senmatic search.
33-
* Supporting different UI providers like [Chatbot UI](https://github.com/SciSharp/chatbot-ui) and [HuggingChat UI](https://github.com/huggingface/chat-ui).
34-
* Integrated with popular social platforms like Facebook Messenger, Slack and Telegram.
35-
* Providing REST APIs to work with your own UI.
24+
However, BotSharp lets you easily achieve a conversational user experience by handling the natural language understanding (NLU) for you. When you use BotSharp, you can create agents that can understand the meaning of natural language and the nuances and trainslate that to structured meaning your software can understand.

‎src/Infrastructure/BotSharp.Abstraction/Agents/Models/Agent.cs‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,11 @@ public class Agent
1111
public DateTime CreatedDateTime { get; set; }
1212
public DateTime UpdatedDateTime { get; set; }
1313

14+
/// <summary>
15+
/// Default LLM settings
16+
/// </summary>
17+
public AgentLlmConfig? LlmConfig { get; set; }
18+
1419
/// <summary>
1520
/// Instruction
1621
/// </summary>

‎src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentLlmConfig.cs‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
namespace BotSharp.Abstraction.Agents.Models;
2+
3+
public class AgentLlmConfig
4+
{
5+
/// <summary>
6+
/// Completion Provider
7+
/// </summary>
8+
[JsonPropertyName("provider")]
9+
public string? Provider { get; set; }
10+
11+
/// <summary>
12+
/// Model name
13+
/// </summary>
14+
[JsonPropertyName("model")]
15+
public string? Model { get; set; }
16+
}

‎src/Infrastructure/BotSharp.Abstraction/Conversations/Models/TokenStatsModel.cs‎

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,10 @@ namespace BotSharp.Abstraction.Conversations.Models;
22

33
public class TokenStatsModel
44
{
5+
public string Provider { get; set; }
56
public string Model { get; set; }
67
public string Prompt { get; set; }
78
public int PromptCount { get; set; }
89
public int CompletionCount { get; set; }
9-
10-
/// <summary>
11-
/// Prompt cost per 1K token
12-
/// </summary>
13-
public float PromptCost { get; set; }
14-
15-
/// <summary>
16-
/// Completion cost per 1K token
17-
/// </summary>
18-
public float CompletionCost { get; set; }
10+
public AgentLlmConfig LlmConfig { get; set; }
1911
}

‎src/Infrastructure/BotSharp.Abstraction/Evaluations/Settings/EvaluatorSetting.cs‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,5 @@ namespace BotSharp.Abstraction.Evaluations.Settings;
22

33
public class EvaluatorSetting
44
{
5-
public string EvaluatorId { get; set; }
6-
public string Provider { get; set; }
7-
public string Model { get; set; }
5+
public string AgentId { get; set; }
86
}

‎src/Infrastructure/BotSharp.Abstraction/MLTasks/ILlmProviderSettingService.cs‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
using BotSharp.Abstraction.MLTasks.Settings;
2+
3+
namespace BotSharp.Abstraction.MLTasks;
4+
5+
public interface ILlmProviderSettingService
6+
{
7+
LlmModelSetting GetSetting(string provider, string model);
8+
}

‎src/Infrastructure/BotSharp.Abstraction/MLTasks/Settings/ChatCompletionSetting.cs‎

Lines changed: 0 additions & 7 deletions
This file was deleted.

0 commit comments

Comments
 (0)