Repository navigation
v0.4.12: guardrails, mistral models, privacy policy updates - #1608
Conversation
…dex on snapshot id (#1603)
* updated privacy policy for google * update terms, privacy, and emails to incl address and update verbiage
* Adding guardrails block * ack PR comments * cleanup checkbox in dark mode * cleanup * fix supabase tools
… API, add drag-and-drop functionality to deployed chat (#1606) * fix(inference): fix inference billing when stream is true via API * add drag-and-drop to deployed chat
* feat(mistal): added mistral as a provider, updated model prices * remove the ability for a block to reference its own outluts * fixed order of responses for guardrails block
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Greptile Overview
Summary
This release bundles 6 separate PRs into v0.4.12, introducing guardrails for content validation, Mistral AI provider support, performance improvements, and UX enhancements.
Major Changes:
- Guardrails Feature (#1605): Added comprehensive content validation system with JSON validation, regex pattern matching, RAG-based hallucination detection using knowledge bases, and PII detection/masking using Microsoft Presidio
- Mistral AI Provider (#1607): Integrated Mistral AI as a new LLM provider with streaming support, tool calling, and updated pricing for all models across providers
- Inference Billing Fix (#1606): Resolved critical billing bug where streaming API executions weren't properly tracked; now defers logging completion until after token counting
- Drag-and-Drop Files (#1606): Enhanced deployed chat interface with drag-and-drop file upload capability
- Performance Optimization (#1603): Removed unused database indexes on
workflow_edges(source/target blocks) and added index onstate_snapshot_idfor faster execution log queries - Blog Path Redirect (#1604): Added permanent redirect from
/building/*to/blog/*for backward compatibility - Privacy Policy Update (#1602): Updated privacy policy for additional Google OAuth scopes
Key Implementation Details:
- Guardrails block uses Python subprocess for PII detection via Presidio, with proper error handling and JSON parsing
- Hallucination detection queries knowledge bases via internal API and uses LLM to score confidence (0-10 scale)
- Mistral provider uses OpenAI SDK with custom base URL for API compatibility
- Streaming billing fix adds
skipLoggingCompleteflag and_streamingMetadatato defer cost calculation
Confidence Score: 5/5
- This PR is safe to merge with high confidence
- All changes are well-implemented with no critical issues found. The PR bundles multiple tested PRs (#1603-1607) that were already reviewed individually. Code follows established patterns, includes proper error handling, and the billing fix addresses a real production issue. Database migrations are minimal and safe (index changes only).
- No files require special attention
Important Files Changed
File Analysis
| Filename | Score | Overview |
|---|---|---|
| packages/db/migrations/0098_thick_prima.sql | 5/5 | Removed unused source/target block indexes and added index on state_snapshot_id for performance optimization |
| apps/sim/blocks/blocks/guardrails.ts | 5/5 | Added comprehensive guardrails block with JSON, regex, hallucination, and PII validation types |
| apps/sim/lib/guardrails/validate_pii.py | 5/5 | Implemented PII detection using Presidio library with block/mask modes and multi-language support |
| apps/sim/lib/guardrails/validate_hallucination.ts | 5/5 | Implemented RAG-based hallucination detection with knowledge base retrieval and LLM confidence scoring |
| apps/sim/providers/mistral/index.ts | 5/5 | Added Mistral AI provider with OpenAI-compatible API client, streaming support, and tool calling |
| apps/sim/lib/workflows/streaming.ts | 5/5 | Fixed streaming workflow billing by deferring logging completion until after token counting |
| apps/sim/app/api/workflows/[id]/execute/route.ts | 5/5 | Added skipLoggingComplete option to support deferred billing in streaming workflows |
Sequence Diagram
sequenceDiagram
participant Client
participant API
participant Executor
participant Guardrails
participant KB as Knowledge Base
participant LLM
participant Presidio as PII Detection
participant Billing
Client->>API: POST /api/workflows/[id]/execute
API->>Executor: executeWorkflow(stream: true)
alt Guardrails Block
Executor->>Guardrails: Validate content
alt Hallucination Check
Guardrails->>KB: Query knowledge base (RAG)
KB-->>Guardrails: Return chunks
Guardrails->>LLM: Score confidence (0-10)
LLM-->>Guardrails: Score + reasoning
else PII Detection
Guardrails->>Presidio: Analyze text
Presidio-->>Guardrails: Detected entities
alt Block Mode
Guardrails-->>Executor: Validation failed
else Mask Mode
Guardrails-->>Executor: Masked text
end
end
end
Executor->>LLM: Execute workflow (stream)
LLM-->>Executor: Stream response
Executor-->>API: Stream chunks
Note over Executor,Billing: Streaming Fix: Defer logging completion
Executor->>Executor: Build trace spans with costs
Executor->>Billing: Complete logging session
API-->>Client: Return response
59 files reviewed, no comments
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| export function validateRegex(inputStr: string, pattern: string): ValidationResult { | ||
| try { | ||
| const regex = new RegExp(pattern) |
Check failure
Code scanning / CodeQL
Regular expression injection
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
To fix this vulnerability, you need to escape any potentially malicious regular expression meta-characters from user-provided input before using it to construct a regular expression object.
- The recommended approach is to escape the pattern using a sanitization function such as
_.escapeRegExpfrom the lodash library. - This should occur within the
validateRegexfunction, before callingnew RegExp(pattern). - To avoid changing existing functionality for cases where you do want a full regex, you may want to distinguish between pattern types (regex vs. literal), but based on the evidence, defaulting to escaping is safer.
- You will need to:
- Import lodash.
- Use
_.escapeRegExp(pattern)to sanitize the pattern before constructing theRegExp.
Changes:
- apps/sim/lib/guardrails/validate_regex.ts
- Add lodash
escapeRegExpimport. - Escape
patternbefore using it inRegExp.
- Add lodash
| @@ -1,3 +1,4 @@ | ||
| import _ from 'lodash'; | ||
| /** | ||
| * Validate if input matches regex pattern | ||
| */ | ||
| @@ -8,7 +9,9 @@ | ||
|
|
||
| export function validateRegex(inputStr: string, pattern: string): ValidationResult { | ||
| try { | ||
| const regex = new RegExp(pattern) | ||
| // Sanitize the pattern using lodash to prevent regex injection | ||
| const safePattern = _.escapeRegExp(pattern); | ||
| const regex = new RegExp(safePattern) | ||
| const match = regex.test(inputStr) | ||
|
|
||
| if (match) { |
| @@ -115,7 +115,8 @@ | ||
| "three": "0.177.0", | ||
| "uuid": "^11.1.0", | ||
| "xlsx": "0.18.5", | ||
| "zod": "^3.24.2" | ||
| "zod": "^3.24.2", | ||
| "lodash": "^4.17.21" | ||
| }, | ||
| "devDependencies": { | ||
| "@testing-library/jest-dom": "^6.6.3", |
| Package | Version | Security advisories |
| lodash (npm) | 4.17.21 | None |
…1611) * fix(vulns): fix SSRF vulnerabilities * cleanup * cleanup * regen docs * remove unused deps * fix failing tests * cleanup * update deps * regen bun lock
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<script[\s\S]*?<\/script\s*>/gi, '') |
Check failure
Code scanning / CodeQL
Incomplete multi-character sanitization
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
The best, safest, and most robust way to address this problem is to use a well-tested library designed for sanitizing HTML, such as sanitize-html. This ensures proper removal of all <script>, <style>, and any other potentially dangerous tags or attributes, and will handle irregular or fragmented HTML that regex-based approaches may miss.
What to change:
- Import
sanitize-htmlat the top of the file. - Replace the entire
stripHtmlfunction with a call tosanitizeHtml(with no allowed tags or attributes, to strip everything to plain text).
Code regions to edit:
- Add
sanitize-htmlimport at the top of the file. - Replace the
stripHtmlfunction definition.
What is needed:
- Import statement for
sanitize-html. - Definition of
stripHtml(now usingsanitizeHtml). - No other code needs to change.
| @@ -2,6 +2,7 @@ | ||
| import { createLogger } from '@/lib/logs/console/logger' | ||
| import { executeTool } from '@/tools' | ||
| import type { TableRow } from '@/tools/types' | ||
| import sanitizeHtml from 'sanitize-html' | ||
|
|
||
| interface MakeApiRequestParams { | ||
| url: string | ||
| @@ -51,18 +52,8 @@ | ||
|
|
||
| const stripHtml = (html: string): string => { | ||
| try { | ||
| let text = html | ||
| let previous: string | ||
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<script[\s\S]*?<\/script\s*>/gi, '') | ||
| text = text.replace(/<style[\s\S]*?<\/style\s*>/gi, '') | ||
| text = text.replace(/<[^>]*>/g, ' ') | ||
| text = text.replace(/[<>]/g, ' ') | ||
| } while (text !== previous) | ||
|
|
||
| return text.replace(/\s+/g, ' ').trim() | ||
| // Remove all HTML tags and scripts/styles using sanitize-html (no tags allowed) | ||
| return sanitizeHtml(html, { allowedTags: [], allowedAttributes: {} }).replace(/\s+/g, ' ').trim() | ||
| } catch { | ||
| return html | ||
| } |
| @@ -115,7 +115,8 @@ | ||
| "three": "0.177.0", | ||
| "uuid": "^11.1.0", | ||
| "xlsx": "0.18.5", | ||
| "zod": "^3.24.2" | ||
| "zod": "^3.24.2", | ||
| "sanitize-html": "^2.17.0" | ||
| }, | ||
| "devDependencies": { | ||
| "@testing-library/jest-dom": "^6.6.3", |
| Package | Version | Security advisories |
| sanitize-html (npm) | 2.17.0 | None |
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<script[\s\S]*?<\/script\s*>/gi, '') |
Check failure
Code scanning / CodeQL
Bad HTML filtering regexp
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
To properly address the issue, replace the custom stripHtml function with one that uses a well-tested HTML parser/sanitizer library. Since we can only change the shown code in apps/sim/lib/copilot/tools/server/other/make-api-request.ts, and new dependencies are allowed if well-known, we should use a stable and popular HTML parsing/sanitization library, such as sanitize-html or html-to-text for robust conversion to plain text.
The most reliable and minimal change, without altering non-related logic, is to replace the implementation of stripHtml so it uses html-to-text's convert function to turn HTML into plain text, preserving semantic integrity and making XSS attacks infeasible. We also need to import this library at the top.
Required changes:
- Import
html-to-textat the top. - Modify the
stripHtmlfunction so that it usesconvert(html)to convert HTML to safe plain text. - No other logic or structure needs to be changed.
| @@ -2,6 +2,7 @@ | ||
| import { createLogger } from '@/lib/logs/console/logger' | ||
| import { executeTool } from '@/tools' | ||
| import type { TableRow } from '@/tools/types' | ||
| import { convert as htmlToText } from 'html-to-text' | ||
|
|
||
| interface MakeApiRequestParams { | ||
| url: string | ||
| @@ -51,18 +52,12 @@ | ||
|
|
||
| const stripHtml = (html: string): string => { | ||
| try { | ||
| let text = html | ||
| let previous: string | ||
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<script[\s\S]*?<\/script\s*>/gi, '') | ||
| text = text.replace(/<style[\s\S]*?<\/style\s*>/gi, '') | ||
| text = text.replace(/<[^>]*>/g, ' ') | ||
| text = text.replace(/[<>]/g, ' ') | ||
| } while (text !== previous) | ||
|
|
||
| return text.replace(/\s+/g, ' ').trim() | ||
| return htmlToText(html, { | ||
| wordwrap: false, | ||
| selectors: [ | ||
| { selector: 'a', options: { ignoreHref: true } } | ||
| ] | ||
| }).trim() | ||
| } catch { | ||
| return html | ||
| } |
| do { | ||
| previous = text | ||
| text = text.replace(/<script[\s\S]*?<\/script\s*>/gi, '') | ||
| text = text.replace(/<style[\s\S]*?<\/style\s*>/gi, '') |
Check failure
Code scanning / CodeQL
Incomplete multi-character sanitization
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
The most robust way to fix this problem is to use a well-established sanitization library such as sanitize-html to remove unwanted HTML tags (like <style> and <script>) and their contents safely. This avoids the pitfalls of regular expression-based HTML stripping.
- Add the import for
sanitize-htmlat the top of the file. - Refactor the
stripHtmlfunction to callsanitizeHtmlwith a configuration that removes all tags or, at minimum, excludesstyleandscripttags and their contents. - This approach fully replaces the current regex loop, making it safer and simpler.
Required changes:
- Import
sanitize-htmlinapps/sim/lib/copilot/tools/server/other/make-api-request.ts. - Replace the body of the
stripHtmlfunction (lines 53–69) with a call tosanitizeHtmlconfigured to remove all tags. - No need for the
do...whileloop or manual regex replacements insidestripHtml.
| @@ -2,6 +2,7 @@ | ||
| import { createLogger } from '@/lib/logs/console/logger' | ||
| import { executeTool } from '@/tools' | ||
| import type { TableRow } from '@/tools/types' | ||
| import sanitizeHtml from 'sanitize-html'; | ||
|
|
||
| interface MakeApiRequestParams { | ||
| url: string | ||
| @@ -51,20 +52,12 @@ | ||
|
|
||
| const stripHtml = (html: string): string => { | ||
| try { | ||
| let text = html | ||
| let previous: string | ||
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<script[\s\S]*?<\/script\s*>/gi, '') | ||
| text = text.replace(/<style[\s\S]*?<\/style\s*>/gi, '') | ||
| text = text.replace(/<[^>]*>/g, ' ') | ||
| text = text.replace(/[<>]/g, ' ') | ||
| } while (text !== previous) | ||
|
|
||
| return text.replace(/\s+/g, ' ').trim() | ||
| // Remove all HTML tags and their contents where appropriate | ||
| // Only preserve plain text content | ||
| const cleaned = sanitizeHtml(html, { allowedTags: [], allowedAttributes: {} }); | ||
| return cleaned.replace(/\s+/g, ' ').trim(); | ||
| } catch { | ||
| return html | ||
| return html; | ||
| } | ||
| } | ||
| let normalized = toStringSafe(data) |
| @@ -115,7 +115,8 @@ | ||
| "three": "0.177.0", | ||
| "uuid": "^11.1.0", | ||
| "xlsx": "0.18.5", | ||
| "zod": "^3.24.2" | ||
| "zod": "^3.24.2", | ||
| "sanitize-html": "^2.17.0" | ||
| }, | ||
| "devDependencies": { | ||
| "@testing-library/jest-dom": "^6.6.3", |
| Package | Version | Security advisories |
| sanitize-html (npm) | 2.17.0 | None |
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<[^>]*>/g, '') |
Check failure
Code scanning / CodeQL
Incomplete multi-character sanitization
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
The best fix is to replace the custom stripHtmlTags logic with an established HTML sanitization library, such as sanitize-html, which is specifically designed for robust and reliable removal of HTML tags and content. This avoids subtle mistakes with regular expressions and ensures edge cases are handled securely. Update stripHtmlTags in apps/sim/tools/confluence/utils.ts to use sanitize-html, and add a required import at the top of the file. The rest of the code using stripHtmlTags doesn't require changes, provided the new function's semantics are the same (i.e., returning text without tags). If you are limited to the code visible, add the import and replace the function body.
| @@ -1,3 +1,5 @@ | ||
| import sanitizeHtml from "sanitize-html"; | ||
|
|
||
| export async function getConfluenceCloudId(domain: string, accessToken: string): Promise<string> { | ||
| const response = await fetch('https://api.atlassian.com/oauth/token/accessible-resources', { | ||
| method: 'GET', | ||
| @@ -44,16 +46,8 @@ | ||
| } | ||
|
|
||
| function stripHtmlTags(html: string): string { | ||
| let text = html | ||
| let previous: string | ||
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<[^>]*>/g, '') | ||
| text = text.replace(/[<>]/g, '') | ||
| } while (text !== previous) | ||
|
|
||
| return text.trim() | ||
| // Use sanitize-html to robustly remove HTML tags and content | ||
| return sanitizeHtml(html, { allowedTags: [], allowedAttributes: {} }).trim(); | ||
| } | ||
|
|
||
| export function transformPageData(data: any) { |
| @@ -115,7 +115,8 @@ | ||
| "three": "0.177.0", | ||
| "uuid": "^11.1.0", | ||
| "xlsx": "0.18.5", | ||
| "zod": "^3.24.2" | ||
| "zod": "^3.24.2", | ||
| "sanitize-html": "^2.17.0" | ||
| }, | ||
| "devDependencies": { | ||
| "@testing-library/jest-dom": "^6.6.3", |
| Package | Version | Security advisories |
| sanitize-html (npm) | 2.17.0 | None |
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<[^>]*>/g, '') |
Check failure
Code scanning / CodeQL
Incomplete multi-character sanitization
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI about 1 year ago
The best way to fix this problem is to avoid regex-based HTML stripping and use a well-known, robust sanitization library such as sanitize-html or striptags. Both are designed to defend against evasion patterns that regex alone tends to miss. Since the job of this function (as its name implies) is just to extract plain text from HTML, striptags is optimal, since it removes all tags and leaves only the text content.
To implement the fix, update the stripHtmlTags function in apps/sim/tools/sharepoint/utils.ts to use the striptags package. Add import striptags from 'striptags' at the top. Replace the current implementation of stripHtmlTags with a single call to striptags(html).trim().
This will ensure that the sanitization is thorough, tested, and robust against all incomplete multi-character matches or evasion tricks.
| @@ -1,19 +1,10 @@ | ||
| import { createLogger } from '@/lib/logs/console/logger' | ||
| import type { CanvasLayout } from '@/tools/sharepoint/types' | ||
|
|
||
| import striptags from 'striptags' | ||
| const logger = createLogger('SharepointUtils') | ||
|
|
||
| function stripHtmlTags(html: string): string { | ||
| let text = html | ||
| let previous: string | ||
|
|
||
| do { | ||
| previous = text | ||
| text = text.replace(/<[^>]*>/g, '') | ||
| text = text.replace(/[<>]/g, '') | ||
| } while (text !== previous) | ||
|
|
||
| return text.trim() | ||
| return striptags(html).trim() | ||
| } | ||
|
|
||
| export function extractTextFromCanvasLayout(canvasLayout: CanvasLayout | null | undefined): string { |
| @@ -115,7 +115,8 @@ | ||
| "three": "0.177.0", | ||
| "uuid": "^11.1.0", | ||
| "xlsx": "0.18.5", | ||
| "zod": "^3.24.2" | ||
| "zod": "^3.24.2", | ||
| "striptags": "^3.2.0" | ||
| }, | ||
| "devDependencies": { | ||
| "@testing-library/jest-dom": "^6.6.3", |
| Package | Version | Security advisories |
| striptags (npm) | 3.2.0 | None |
Uh oh!
There was an error while loading. Please reload this page.