Skip to content

Commit f5eaa71

Browse files
author
Haiping Chen
committed
Add featuer of VisibilityExpression
1 parent 3dd203f commit f5eaa71

9 files changed

Lines changed: 128 additions & 33 deletions

File tree

‎docs/llm/function.md‎

Lines changed: 29 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,32 @@
22

33
A **calling function** is a function that is passed as an argument to another function and is executed after a specific event or action occurs. In the context of **large language models (LLMs)**, calling functions can be used to hook into various stages of an LLM application. They are useful for tasks such as logging, monitoring, streaming, and more. For example, in the **BotSharp** framework, calling functions can be used to log information, monitor the progress of an LLM application, or perform other tasks. The BotSharp provides a `callbacks` argument that allows developers to interactive with external systems.
44

5-
The use of calling functions in LLM applications provides flexibility and extensibility. Developers can customize the behavior of their applications by defining callback handlers that implement specific methods. These handlers can be used for tasks like logging, error handling, or interacting with external systems. The function will be triggered by LLM based on the conversation context.
5+
The use of calling functions in LLM applications provides flexibility and extensibility. Developers can customize the behavior of their applications by defining callback handlers that implement specific methods. These handlers can be used for tasks like logging, error handling, or interacting with external systems. The function will be triggered by LLM based on the conversation context.
6+
7+
## Hide Function
8+
9+
In order to more flexibly control whether the Agent is allowed to use a certain function, there is a Visibility Expression property in the function definition that can be used to control display or hiding. When we input prompt into LLM, although we can use state variables in the system instruction file to control the rendering content, LLM will still take the definition of the function into consideration. If the related functions are not hidden at the same time, LLM will still be It is possible to call related functions, bringing unexpected results. Because we need to control system instruction and function definition at the same time to make them consistent.
10+
11+
```json
12+
{
13+
"name": "make_payment",
14+
"description": "call this function to make payment",
15+
"visibility_expression": "{% if states.order_number != empty %}visible{% endif %}",
16+
"parameters": {
17+
"type": "object",
18+
"properties": {
19+
"order_number": {
20+
"type": "string",
21+
"description": "order number."
22+
},
23+
"total_amount": {
24+
"type": "string",
25+
"description": "total amount."
26+
}
27+
},
28+
"required": ["order_number", "total_amount"]
29+
}
30+
}
31+
```
32+
33+
The above is an example. The system will parse the liquid template of Visibility Expression `{% if states.order_number != empty %}visible{% endif %}`. When "visible" is returned, the system will allow the Agent to use this function. In liquid In expressions, we can use `states.name` to reference the state value in the conversation.

‎src/Infrastructure/BotSharp.Abstraction/Agents/IAgentService.cs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
using BotSharp.Abstraction.Functions.Models;
12
using BotSharp.Abstraction.Plugins.Models;
23
using BotSharp.Abstraction.Repositories.Filters;
34

@@ -23,6 +24,8 @@ public interface IAgentService
2324

2425
string RenderedTemplate(Agent agent, string templateName);
2526

27+
bool RenderFunction(Agent agent, FunctionDef def);
28+
2629
/// <summary>
2730
/// Get agent detail without trigger any hook.
2831
/// </summary>

‎src/Infrastructure/BotSharp.Abstraction/Functions/Models/FunctionDef.cs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,9 @@ public class FunctionDef
55
public string Name { get; set; }
66
public string Description { get; set; }
77

8+
[JsonPropertyName("visibility_expression")]
9+
public string? VisibilityExpression { get; set; }
10+
811
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
912
public string? Impact { get; set; }
1013

‎src/Infrastructure/BotSharp.Abstraction/Loggers/IContentGeneratingHook.cs‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,4 +28,13 @@ public interface IContentGeneratingHook
2828
/// </summary>
2929
/// <returns></returns>
3030
Task AfterGenerated(RoleDialogModel message, TokenStatsModel tokenStats) => Task.CompletedTask;
31+
32+
/// <summary>
33+
/// Rdndering template
34+
/// </summary>
35+
/// <param name="agent"></param>
36+
/// <param name="name"></param>
37+
/// <param name="content"></param>
38+
/// <returns></returns>
39+
Task OnRenderingTemplate(Agent agent, string name, string content) => Task.CompletedTask;
3140
}

‎src/Infrastructure/BotSharp.Core/Agents/Services/AgentService.LoadAgent.cs‎

Lines changed: 0 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -90,32 +90,6 @@ public async Task<Agent> LoadAgent(string id)
9090
return agent;
9191
}
9292

93-
public string RenderedTemplate(Agent agent, string templateName)
94-
{
95-
// render liquid template
96-
var render = _services.GetRequiredService<ITemplateRender>();
97-
var template = agent.Templates.First(x => x.Name == templateName).Content;
98-
// update states
99-
var conv = _services.GetRequiredService<IConversationService>();
100-
foreach (var t in conv.States.GetStates())
101-
{
102-
agent.TemplateDict[t.Key] = t.Value;
103-
}
104-
return render.Render(template, agent.TemplateDict);
105-
}
106-
107-
public string RenderedInstruction(Agent agent)
108-
{
109-
var render = _services.GetRequiredService<ITemplateRender>();
110-
// update states
111-
var conv = _services.GetRequiredService<IConversationService>();
112-
foreach (var t in conv.States.GetStates())
113-
{
114-
agent.TemplateDict[t.Key] = t.Value;
115-
}
116-
return render.Render(agent.Instruction, agent.TemplateDict);
117-
}
118-
11993
private void PopulateState(Dictionary<string, object> dict)
12094
{
12195
var conv = _services.GetRequiredService<IConversationService>();

‎src/Infrastructure/BotSharp.Core/Agents/Services/AgentService.Rendering.cs‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
using BotSharp.Abstraction.Loggers;
2+
using BotSharp.Abstraction.Templating;
3+
4+
namespace BotSharp.Core.Agents.Services;
5+
6+
public partial class AgentService
7+
{
8+
public string RenderedInstruction(Agent agent)
9+
{
10+
var render = _services.GetRequiredService<ITemplateRender>();
11+
// update states
12+
var conv = _services.GetRequiredService<IConversationService>();
13+
foreach (var t in conv.States.GetStates())
14+
{
15+
agent.TemplateDict[t.Key] = t.Value;
16+
}
17+
return render.Render(agent.Instruction, agent.TemplateDict);
18+
}
19+
20+
public bool RenderFunction(Agent agent, FunctionDef def)
21+
{
22+
if (!string.IsNullOrEmpty(def.VisibilityExpression))
23+
{
24+
var render = _services.GetRequiredService<ITemplateRender>();
25+
var result = render.Render(def.VisibilityExpression, new Dictionary<string, object>
26+
{
27+
{ "states", agent.TemplateDict }
28+
});
29+
return result == "visible";
30+
}
31+
32+
return true;
33+
}
34+
35+
public string RenderedTemplate(Agent agent, string templateName)
36+
{
37+
// render liquid template
38+
var render = _services.GetRequiredService<ITemplateRender>();
39+
var template = agent.Templates.First(x => x.Name == templateName).Content;
40+
// update states
41+
var conv = _services.GetRequiredService<IConversationService>();
42+
foreach (var t in conv.States.GetStates())
43+
{
44+
agent.TemplateDict[t.Key] = t.Value;
45+
}
46+
47+
var content = render.Render(template, agent.TemplateDict);
48+
49+
HookEmitter.Emit<IContentGeneratingHook>(_services, async hook =>
50+
await hook.OnRenderingTemplate(agent, templateName, content)
51+
).Wait();
52+
53+
return content;
54+
}
55+
}

‎src/Infrastructure/BotSharp.Core/Agents/Services/AgentService.cs‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,3 @@
1-
using BotSharp.Abstraction.Repositories;
21
using System.IO;
32

43
namespace BotSharp.Core.Agents.Services;

‎src/Plugins/BotSharp.Plugin.AzureOpenAI/Providers/ChatCompletionProvider.cs‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -219,12 +219,15 @@ public async Task<bool> GetChatCompletionsStreamingAsync(Agent agent, List<RoleD
219219

220220
foreach (var function in agent.Functions)
221221
{
222-
chatCompletionsOptions.Functions.Add(new FunctionDefinition
222+
if (agentService.RenderFunction(agent, function))
223223
{
224-
Name = function.Name,
225-
Description = function.Description,
226-
Parameters = BinaryData.FromObjectAsJson(function.Parameters)
227-
});
224+
chatCompletionsOptions.Functions.Add(new FunctionDefinition
225+
{
226+
Name = function.Name,
227+
Description = function.Description,
228+
Parameters = BinaryData.FromObjectAsJson(function.Parameters)
229+
});
230+
}
228231
}
229232

230233
foreach (var message in conversations)

‎src/Plugins/BotSharp.Plugin.ChatHub/Hooks/StreamingLogHook.cs‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,27 @@ public override async Task OnPostbackMessageReceived(RoleDialogModel message, Po
7272
await _chatHub.Clients.User(_user.Id).SendAsync("OnConversationContentLogGenerated", BuildContentLog(input));
7373
}
7474

75+
public async Task OnRenderingTemplate(Agent agent, string name, string content)
76+
{
77+
if (!_convSettings.ShowVerboseLog) return;
78+
79+
var conversationId = _state.GetConversationId();
80+
81+
var log = $"{agent.Name} is using template {name}";
82+
var message = new RoleDialogModel(AgentRole.System, log)
83+
{
84+
MessageId = _routingCtx.MessageId
85+
};
86+
87+
var input = new ContentLogInputModel(conversationId, message)
88+
{
89+
Name = agent.Name,
90+
Source = ContentLogSource.HardRule,
91+
Log = log
92+
};
93+
await _chatHub.Clients.User(_user.Id).SendAsync("OnConversationContentLogGenerated", BuildContentLog(input));
94+
}
95+
7596
public async Task BeforeGenerating(Agent agent, List<RoleDialogModel> conversations)
7697
{
7798
if (!_convSettings.ShowVerboseLog) return;

0 commit comments

Comments
 (0)