Skip to content

[Docs][Site][UI] - Search is now MCP based with UI updates.#909

Merged
madster456 merged 26 commits intodevfrom
docs-updated_search
Oct 22, 2025
Merged

[Docs][Site][UI] - Search is now MCP based with UI updates.#909
madster456 merged 26 commits intodevfrom
docs-updated_search

Conversation

@madster456
Copy link
Copy Markdown
Collaborator

@madster456 madster456 commented Sep 22, 2025

Enhances the search functionality to now use the MCP server.

Now handles API endpoints and webhooks.

Now looks at what platform the user has selected, and searches based on that. User can choose to filter differently if needed.

image

High-level PR Summary

This PR enhances the search functionality for Stack Auth documentation by integrating it with their MCP (Model Control Plane) server. The implementation replaces the previous client-side search algorithm with a server-side approach that leverages the MCP's search capabilities. The changes affect three main files: the MCP handler that now includes a new search_docs tool, the search route handler that now forwards queries to the MCP server instead of performing local searches, and the search dialog UI that has been updated to better handle platform-specific filtering and API documentation. The new implementation provides more relevant search results and automatically filters based on the user's current platform context, while allowing users to customize their search filters as needed.

⏱️ Estimated Review Time: 30-90 minutes

💡 Review Order Suggestion
Order File Path
1 docs/src/app/api/internal/[transport]/route.ts
2 docs/src/app/api/search/route.ts
3 docs/src/components/layout/custom-search-dialog.tsx

Important

Enhances search by integrating with MCP server, updating UI for platform-specific filtering, and improving result presentation.

  • Search Functionality:
    • Integrates search with MCP server in route.ts files, replacing client-side search.
    • Supports API and platform-specific results, filtering out admin API endpoints.
    • Sorts results by platform priority.
  • UI Updates:
    • Updates custom-search-dialog.tsx to support platform-specific filtering and API results.
    • Adds platform badges and icons for different result types.
    • Auto-detects platform from URL and adjusts search filters accordingly.
  • Miscellaneous:
    • Adds new helper functions for platform extraction and MCP server communication.
    • Improves error handling and logging for search operations.

This description was created by Ellipsis for 9941d02. You can customize this summary. It will automatically update as commits are pushed.


Review by RecurseML

🔍 Review performed on 7a0bf86..28264f7

  Severity     Location     Issue     Delete  
Medium docs/src/app/api/internal/[transport]/route.ts:125 API parameter 'query' uses camelCase instead of required snake_case
Medium docs/src/app/api/internal/[transport]/route.ts:126 API parameter 'limit' uses camelCase instead of required snake_case
Medium docs/src/app/api/internal/[transport]/route.ts:174 Async operation not wrapped with runAsynchronously
Medium docs/src/app/api/search/route.ts:23 REST API parameters not using snake_case
Medium docs/src/app/api/search/route.ts:130 Async function call not wrapped in runAsynchronously
Medium docs/src/app/api/search/route.ts:11 Async function definition without proper runAsynchronously usage
✅ Files analyzed, no issues (1)

docs/src/components/layout/custom-search-dialog.tsx

Need help? Join our Discord

Summary by CodeRabbit

  • New Features

    • Search now includes API docs as an "API" result type with relevance snippets, API-specific icons and group titles.
    • Platform-aware filtering auto-detects platform on open and adds “API only” and “Platform + API” views.
  • Improvements

    • Search is powered by a centralized streamed service for more consistent results, better error handling, and fallbacks.
    • Results are ordered with platform-priority, show clearer counts/footers, and present more resilient, unified result formatting.

@vercel
Copy link
Copy Markdown

vercel bot commented Sep 22, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
stack-backend Ready Ready Preview Comment Oct 22, 2025 6:01am
stack-dashboard Ready Ready Preview Comment Oct 22, 2025 6:01am
stack-demo Ready Ready Preview Comment Oct 22, 2025 6:01am
stack-docs Ready Ready Preview Comment Oct 22, 2025 6:01am

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Sep 22, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Adds an internal MCP tool search_docs, updates the public /api/search endpoint to call MCP and consume streamed results (adds api result type and platform-priority sorting), and extends the search dialog UI to include and prioritize API results alongside platform docs.

Changes

Cohort / File(s) Summary
Internal MCP tool: docs search
docs/src/app/api/internal/[transport]/route.ts
Adds server.tool('search_docs', ...) handler that accepts { search_query, result_limit }, performs multi-page search over docs (excludes admin API pages), computes composite relevance from title/description/TOC/content, attempts file reads for sanitized snippets with graceful fallbacks, streams/returns ranked results (title, description, url, type, score, snippet).
Public search API using MCP
docs/src/app/api/search/route.ts
Replaces local filesystem/MDX search with MCP call/stream parsing (SSE-style), maps MCP items to SearchResult (adds type: 'api'), removes local scoring, filters admin URLs, applies platform-priority tie-breaker sorting, and returns MCP-driven JSON with error handling for MCP anomalies.
Search dialog UI updates
docs/src/components/layout/custom-search-dialog.tsx
Extends SearchResult handling for api type, differentiates API vs docs base URLs, generates platform-aware API titles/icons, auto-detects current platform on open, supports platform+API filtering and prioritization, and updates footer/labels/result counts to reflect combined API results.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant U as User
  participant UI as Search Dialog (client)
  participant API as /api/search (Next.js)
  participant MCP as MCP Server
  participant Tool as search_docs Tool

  U->>UI: open dialog / type query
  UI->>API: GET /api/search?q=...
  API->>MCP: call `search_docs` (SSE/stream)
  MCP->>Tool: invoke `search_docs(search_query, result_limit)`
  Tool-->>MCP: stream ranked results + snippets
  MCP-->>API: stream MCP results
  API->>API: filter admin URLs, apply platform-priority sorting
  API-->>UI: JSON results (type: page/heading/text/api)
  UI->>UI: group, iconify, apply platform+API filters
  UI-->>U: render results
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

I nibble through indexes, sniff out each line,
I fetch snippets and scores, and sort them just fine.
Platform or API, I hop where you're led—
A searching rabbit, returning results ahead. 🐇✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)
Check name Status Explanation
Title Check ✅ Passed The title succinctly captures the main change by indicating that search functionality is now MCP-based and includes UI updates.
Description Check ✅ Passed The description begins with the required guidelines comment and provides a clear overview of changes, including backend search integration, UI improvements, platform filtering logic, and error handling, supplemented by a screenshot for context.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch docs-updated_search

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR refactors the documentation search functionality from a local file-system based approach to an MCP (Model Context Protocol) server-based architecture. The changes span three key files:

  1. MCP Server Enhancement (docs/src/app/api/internal/[transport]/route.ts): Adds a new search_docs tool to the MCP handler that provides comprehensive documentation search with sophisticated ranking algorithms. The search scores results based on title matches, description content, table of contents, and full content matching. It includes intelligent filtering to exclude admin API endpoints and implements snippet extraction around search matches.

  2. UI Search Dialog Updates (docs/src/components/layout/custom-search-dialog.tsx): Enhances the search interface to support multiple content types including API endpoints and webhooks. The component now auto-detects the user's current platform context from the URL and provides platform-aware filtering. It includes specialized icons for different content types (webhooks, API endpoints) and implements intelligent result prioritization that shows platform-specific content first while maintaining access to API documentation.

  3. Search Route Simplification (docs/src/app/api/search/route.ts): Completely refactors the search endpoint to delegate to the MCP server instead of performing local file operations. This removes complex local scoring algorithms, file system dependencies, and text extraction logic in favor of a cleaner client-server architecture.

The changes align with the broader Stack Auth architecture by centralizing search logic in a dedicated MCP server, which enables more sophisticated search capabilities including semantic search, better relevance scoring, and unified indexing of both documentation and API endpoints. The platform-aware filtering ensures users see contextually relevant results while maintaining discoverability of related content.

Confidence score: 3/5

  • This PR introduces significant architectural changes that could cause issues if the MCP server is unavailable or misconfigured
  • Score reflects concerns about hardcoded localhost URLs, potential SSE parsing failures, and silent error handling that could mask problems
  • Pay close attention to the search route implementation and MCP server communication logic

3 files reviewed, 1 comment

Edit Code Review Bot Settings | Greptile

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (10)
docs/src/components/layout/custom-search-dialog.tsx (5)

191-191: Fix browser typing for setTimeout ref (avoid NodeJS.Timeout).

Typing useRef<NodeJS.Timeout>() in a client component can cause TS friction in the browser (setTimeout returns a number). Prefer ReturnType.

-  const searchTimeoutRef = useRef<NodeJS.Timeout>();
+  const searchTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);

122-137: Harden PlatformBadge against unknown platforms (prevent undefined styles/text).

When platform is not in PLATFORM_COLORS/NAMES, color/name become undefined resulting in invalid CSS. Add safe fallbacks.

-  const color = PLATFORM_COLORS[platform as keyof typeof PLATFORM_COLORS];
-  const name = PLATFORM_NAMES[platform as keyof typeof PLATFORM_NAMES];
+  const fallbackColor = '#64748B'; // slate-500
+  const color = PLATFORM_COLORS[platform as keyof typeof PLATFORM_COLORS] ?? fallbackColor;
+  const name =
+    PLATFORM_NAMES[platform as keyof typeof PLATFORM_NAMES] ??
+    (platform ? platform.charAt(0).toUpperCase() + platform.slice(1) : 'Unknown');

43-48: Avoid defaulting to 'api' when platform is undetectable.

Defaulting to 'api' risks misclassification and odd grouping. Prefer 'all' (or 'unknown') and handle it in grouping.

 function extractPlatformFromUrl(url: string): string {
   if (url.startsWith('/api/')) return 'api';
   const match = url.match(/\/docs\/([^\/]+)/);
-  const platform = match?.[1] || 'api';
-  return platform;
+  const platform = match?.[1];
+  return platform ?? 'all';
 }

Optionally, in groupResultsByPage, when platform === 'all', fall back to the page URL sans hash as the baseUrl.


316-323: Use Next.js router for navigation on Enter (avoid full page reload).

Use router.push instead of assigning window.location.href for smoother SPA navigation.

-import { usePathname } from 'next/navigation';
+import { usePathname, useRouter } from 'next/navigation';
...
 export function CustomSearchDialog({ open, onOpenChange }: CustomSearchDialogProps) {
+  const router = useRouter();
...
       case 'Enter': {
         if (!dropdownOpen) {
           e.preventDefault();
           const selectedResult = flatResults[selectedIndex];
-          window.location.href = selectedResult.url;
+          router.push(selectedResult.url);
           onOpenChange(false);
         }
         break;
       }

Also applies to: 5-6, 180-190


271-281: Avoid in-place sort mutating derived data (safer immutability).

Sorting filteredResults in place can surprise future readers. Copy before sorting.

-    filteredResults = filteredResults.sort((a, b) => {
+    filteredResults = [...filteredResults].sort((a, b) => {
docs/src/app/api/internal/[transport]/route.ts (2)

121-245: Good: Useful MCP tool with ranking + snippets. Consider perf and result shape improvements.

Sequentially reading every content file on each query will not scale. Consider caching/pre-indexing, async pooling, or limiting file reads once score > 0 from title/desc and early-continue after pushing.

If helpful, I can sketch a lightweight in-memory inverted index builder for dev and a build-time index for prod.


350-371: Expose search_docs in capabilities for proper tool introspection.

The tool is registered, but not advertised in the capabilities payload. Add it for clients that enumerate tools.

       tools: {
         listAvailableDocs: {
           description:
             "Use this tool to learn about what Stack Auth is, available documentation, and see if you can use it for what you're working on. It returns a list of all available Stack Auth Documentation pages.",
         },
         getDocById: {
           description:
             "Use this tool to retrieve a specific Stack Auth Documentation page by its ID. It gives you the full content of the page so you can know exactly how to use specific Stack Auth APIs. Whenever using Stack Auth, you should always check the documentation first to have the most up-to-date information. When you write code using Stack Auth documentation you should reference the content you used in your comments.",
           parameters: {
             type: "object",
             properties: {
               id: {
                 type: "string",
                 description: "The ID of the documentation page to retrieve.",
               },
             },
             required: ["id"],
           },
         },
+        searchDocs: {
+          description:
+            "Search through all Stack Auth documentation including API docs, guides, and examples. Returns ranked results with snippets and relevance scores.",
+          parameters: {
+            type: "object",
+            properties: {
+              query: { type: "string", description: "The search query to find relevant documentation" },
+              limit: { type: "number", description: "Maximum number of results to return (default: 10)" },
+            },
+            required: ["query"],
+          },
+        },
       },
docs/src/app/api/search/route.ts (3)

107-116: Include '/docs/javascript/' in platform priority mapping.

UI supports both 'js' and 'javascript'. Add the alias to keep ordering consistent.

   if (url.includes('/docs/js/')) return 70;
+  if (url.includes('/docs/javascript/')) return 70;

135-141: Optional: stable sort to preserve MCP relevance on ties.

Use a tiebreaker preserving original order to avoid reordering equal-priority items.

-    const sortedResults = filteredResults.sort((a, b) => {
-      return getPlatformPriority(b.url) - getPlatformPriority(a.url);
-    });
+    const sortedResults = filteredResults
+      .map((r, i) => ({ r, i }))
+      .sort((x, y) => {
+        const d = getPlatformPriority(y.r.url) - getPlatformPriority(x.r.url);
+        return d !== 0 ? d : x.i - y.i; // stable
+      })
+      .map(({ r }) => r);

90-97: Type fidelity: preserve MCP ‘type’ when possible.

All non-api results are forced to 'page', losing heading/text nuance. If MCP returns richer types later, pass them through.

-          type: type === 'api' ? 'api' : 'page',
+          type: (type === 'api' || type === 'heading' || type === 'text') ? (type as any) : 'page',
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7a0bf86 and 28264f7.

📒 Files selected for processing (3)
  • docs/src/app/api/internal/[transport]/route.ts (1 hunks)
  • docs/src/app/api/search/route.ts (2 hunks)
  • docs/src/components/layout/custom-search-dialog.tsx (13 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Prefer ES6 Map over Record when representing key–value collections

Files:

  • docs/src/app/api/internal/[transport]/route.ts
  • docs/src/app/api/search/route.ts
  • docs/src/components/layout/custom-search-dialog.tsx
🧬 Code graph analysis (1)
docs/src/app/api/internal/[transport]/route.ts (1)
docs/scripts/generate-docs.js (1)
  • page (201-201)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: all-good
  • GitHub Check: setup-tests
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: docker
  • GitHub Check: build (22.x)
  • GitHub Check: docker
  • GitHub Check: restart-dev-and-test
  • GitHub Check: build (22.x)
  • GitHub Check: Security Check

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (6)
docs/src/app/api/internal/[transport]/route.ts (2)

137-147: Expose search_docs in capabilities and align tool names.

Capabilities list omits search_docs and uses camelCase names that don’t match registered tools; clients may not discover tools.

Add/update capabilities (outside this hunk):

   capabilities: {
     tools: {
-      listAvailableDocs: {
+      list_available_docs: {
         description:
           "Use this tool to learn about what Stack Auth is, available documentation, and see if you can use it for what you're working on. It returns a list of all available Stack Auth Documentation pages.",
       },
+      search_docs: {
+        description:
+          "Search through all Stack Auth documentation including API docs, guides, and examples. Returns ranked results with snippets and relevance scores.",
+        parameters: {
+          type: "object",
+          properties: {
+            search_query: { type: "string", description: "The search query to find relevant documentation" },
+            result_limit: { type: "number", description: "Maximum number of results to return (default: 50)" }
+          },
+          required: ["search_query"]
+        }
+      },
-      getDocById: {
+      get_docs_by_id: {
         description:
           "Use this tool to retrieve a specific Stack Auth Documentation page by its ID. It gives you the full content of the page so you can know exactly how to use specific Stack Auth APIs. Whenever using Stack Auth, you should always check the documentation first to have the most up-to-date information. When you write code using Stack Auth documentation you should reference the content you used in your comments.",
         parameters: {
           type: "object",
           properties: {
             id: {
               type: "string",
               description: "The ID of the documentation page to retrieve.",
             },
           },
           required: ["id"],
         },
       },
-      getStackAuthSetupInstructions: {
+      get_stack_auth_setup_instructions: {
         description:
           "Use this tool when the user wants to set up Stack Auth in a new project. It provides step-by-step instructions for installing and configuring Stack Auth authentication, including environment setup, file scaffolding, and verification steps.",
         parameters: {
           type: "object",
           properties: {},
           required: [],
         },
       },
     },
   },

187-244: Consider parallelizing reads with bounded concurrency.

Loop does N sequential reads; for many pages this is slow. Use a pool (e.g., 8) with Promise.allSettled.

If helpful, I can provide a small in-file concurrency helper.

docs/src/app/api/search/route.ts (4)

68-96: Preserve MCP relevance by parsing Score and carrying it through.

You currently drop score and later sort only by platform. Parse “Score:” and include it.

-      let snippet = '';
+      let snippet = '';
+      let score: number | undefined = undefined;
...
-        } else if (line.startsWith('Snippet: ')) {
+        } else if (line.startsWith('Snippet: ')) {
           snippet = line.substring(9);
+        } else if (line.startsWith('Score: ')) {
+          const n = Number(line.substring(7).trim());
+          if (Number.isFinite(n)) score = n;
         }
...
-      if (title && url) {
+      if (title && url) {
         results.push({
           id: `${url}-${type}`,
-          type: type === 'api' ? 'api' : 'page',
-          content: snippet || description || title,
+          type: type === 'api' ? 'api' : 'page',
+          content: snippet || description || title,
+          // @ts-expect-error: add optional field; update type as per comment below
+          score,
           url: url,
         });
       }

Also update type (outside this hunk):

 type SearchResult = {
   id: string,
   type: 'page' | 'heading' | 'text' | 'api',
   content: string,
   url: string,
+  score?: number,
 };

138-140: Use score as primary sort, platform as tiebreaker.

-    const sortedResults = filteredResults.sort((a, b) => {
-      return getPlatformPriority(b.url) - getPlatformPriority(a.url);
-    });
+    const sortedResults = filteredResults.sort((a, b) => {
+      const scoreDiff = (b.score ?? 0) - (a.score ?? 0);
+      if (scoreDiff !== 0) return scoreDiff;
+      return getPlatformPriority(b.url) - getPlatformPriority(a.url);
+    });

135-136: Tighten admin filter to avoid false positives.

Match '/api/admin/' prefix only.

-    const filteredResults = results.filter(result => !result.url.startsWith('/api/admin'));
+    const filteredResults = results.filter(result => !result.url.startsWith('/api/admin/'));

58-66: Fragile text parsing from MCP; prefer JSON payload.

Switch search_docs to return structured JSON in content[0].text to avoid parsing drift.

I can provide coordinated patches in both files to emit/consume JSON if you’d like.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7909549 and ac06c4a.

📒 Files selected for processing (2)
  • docs/src/app/api/internal/[transport]/route.ts (1 hunks)
  • docs/src/app/api/search/route.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Prefer ES6 Map over Record when representing key–value collections

Files:

  • docs/src/app/api/internal/[transport]/route.ts
  • docs/src/app/api/search/route.ts
🧬 Code graph analysis (1)
docs/src/app/api/search/route.ts (1)
docs/src/app/llms.mdx/[[...slug]]/route.ts (1)
  • GET (8-30)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: docker
  • GitHub Check: build (22.x)
  • GitHub Check: build (22.x)
  • GitHub Check: all-good
  • GitHub Check: setup-tests
  • GitHub Check: restart-dev-and-test
  • GitHub Check: docker
  • GitHub Check: Security Check
🔇 Additional comments (6)
docs/src/app/api/internal/[transport]/route.ts (3)

141-147: Good: parameters use snake_case.

Matches our REST/MCP param naming rule.


187-244: Wrap async I/O per project rule (runAsynchronously).

Project rule requires wrapping async ops; several readFile awaits aren’t wrapped.

#!/bin/bash
# Find 'runAsynchronously' helper and sites that await readFile without it.
rg -nP "runAsynchronously\s*\(" -C1
rg -nP "await\s+readFile\(" docs/src/app/api/internal/\[transport\]/route.ts -n -C1

181-185: Guard against missing TOC to prevent runtime crash.

-          // TOC/heading matching
-          for (const tocItem of page.data.toc) {
+          // TOC/heading matching
+          const toc = Array.isArray((page as any).data?.toc) ? (page as any).data.toc : [];
+          for (const tocItem of toc) {
             if (typeof tocItem.title === 'string' && tocItem.title.toLowerCase().includes(queryLower)) {
               score += 30;
             }
           }
docs/src/app/api/search/route.ts (3)

34-48: SSE parsing reads only the first data frame; consume all and use last valid JSON.

-    const lines = text.split('\n');
-    let jsonData = null;
-
-    for (const line of lines) {
-      if (line.startsWith('data: ')) {
-        try {
-          jsonData = JSON.parse(line.substring(6));
-          break;
-        } catch (e) {
-          // Continue looking for valid JSON data
-        }
-      }
-    }
+    const lines = text.split('\n');
+    let jsonData: any = null;
+    for (const line of lines) {
+      if (!line.startsWith('data: ')) continue;
+      const payload = line.substring(6).trim();
+      if (!payload || payload === '[DONE]') continue;
+      try {
+        jsonData = JSON.parse(payload);
+      } catch {
+        continue;
+      }
+    }

128-133: Wrap async per project rule (runAsynchronously).

If enforced, wrap the MCP call.

#!/bin/bash
# Locate runAsynchronously and check usage here.
rg -nP "\brunAsynchronously\b" -C1
rg -nP "const results\s*=\s*await\s+callMcpServer\(" docs/src/app/api/search/route.ts -n -C1

13-28: Don’t hard-code MCP URL; add timeout and same-host fallback.

Use internal route or env base and protect with AbortController.

-    const response = await fetch('https://mcp.stack-auth.com/api/internal/mcp', {
+    const controller = new AbortController();
+    const timeoutId = setTimeout(() => controller.abort(), 10_000);
+    const mcpUrl = process.env.INTERNAL_MCP_BASE_URL
+      ? `${process.env.INTERNAL_MCP_BASE_URL}/api/internal/mcp`
+      : '/api/internal/mcp';
+    const response = await fetch(mcpUrl, {
       method: 'POST',
       headers: {
         'Content-Type': 'application/json',
         'Accept': 'application/json, text/event-stream',
       },
       body: JSON.stringify({
         jsonrpc: '2.0',
         method: 'tools/call',
         params: {
           name: 'search_docs',
           arguments: { search_query, result_limit: 20 },
         },
         id: Date.now(),
-      }),
+      }),
+      signal: controller.signal,
     });
+    clearTimeout(timeoutId);

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
docs/src/app/api/internal/[transport]/route.ts (3)

402-431: Expose search_docs in MCP capabilities (discoverability).

Without this, some clients won’t list/invoke the tool.

       tools: {
         listAvailableDocs: {
           description:
             "Use this tool to learn about what Stack Auth is, available documentation, and see if you can use it for what you're working on. It returns a list of all available Stack Auth Documentation pages.",
         },
+        searchDocs: {
+          description:
+            "Search through all Stack Auth documentation including API docs, guides, and examples. Returns ranked results with snippets and relevance scores.",
+          parameters: {
+            type: "object",
+            properties: {
+              search_query: {
+                type: "string",
+                description: "The search query to find relevant documentation",
+              },
+              result_limit: {
+                type: "integer",
+                minimum: 1,
+                maximum: 100,
+                default: 50,
+                description: "Maximum number of results to return (default: 50)",
+              },
+            },
+            required: ["search_query"],
+          },
+        },
         getDocById: {

280-359: Repeat path hardening for get_docs_by_id file reads (primary and alt paths).

Apply the same resolve-and-validate pattern as in search to all reads here.

Example for primary path:

-        const filePath = `content/${page.file.path}`;
+        const contentRoot = path.resolve("content");
+        const filePath = path.resolve(path.join("content", page.file.path));
+        if (!filePath.startsWith(contentRoot + path.sep)) {
+          return {
+            content: [{ type: "text", text: `Title: ${page.data.title}\nDescription: ${page.data.description}\nError: Invalid file path` }],
+            isError: true,
+          };
+        }

For alt paths, resolve and validate each altPath similarly before reading.


26-29: Validate/canonicalize OpenAPI spec paths before reading (prevent path traversal)

Resolve the spec path and check it is inside an explicit allowlist of directories (e.g., content/, content/api/, public/openapi/, openapi/) before calling readFile; reject any resolved path outside those roots.

  • Affected: docs/src/app/api/internal/[transport]/route.ts — lines 26–29 (currently: const specPath = specFile; await readFile(specPath,...))
  • Also harden: docs/lib/get-llm-text.ts — lines 40–42 (join(process.cwd(), documentPath) + readFileSync)
  • Audit: docs/src/app/openapi/[...path]/route.ts — line 22 (readFile(fullPath,...))

Use path.resolve + allowlist check (or path.relative to detect traversal). If specs legitimately live outside content/, explicitly include those directories in the allowlist.

🧹 Nitpick comments (3)
docs/src/app/api/internal/[transport]/route.ts (3)

155-244: Avoid N× awaited disk reads on the hot path; use limited concurrency or pre‑index/cache.

Current per-page await readFile can bottleneck search.

  • Option A: Pre-index content once (startup) and reuse in-memory text for searches.
  • Option B: Use a small concurrency pool (e.g., 8) for file reads; push tasks and await with a limiter.
  • Option C: Cache normalized textContent in a Map keyed by file path+mtime.

15-20: Make EnhancedAPIPage parsing robust to multiline props.

The current regex fails when props span lines.

-  const componentMatch = content.match(/<EnhancedAPIPage\s+([^>]+)>/);
+  const componentMatch = content.match(/<EnhancedAPIPage\b([\s\S]*?)>/);
   if (componentMatch) {
     const props = componentMatch[1];
     const documentMatch = props.match(/document=\{"([^"]+)"\}/);
-    const operationsMatch = props.match(/operations=\{(\[[^\]]+\])\}/);
+    const operationsMatch = props.match(/operations=\{(\[[\s\S]*?\])\}/);

435-438: Gate verbose logs by environment.

Avoid noisy logs in production.

-    verboseLogs: true,
+    verboseLogs: process.env.NODE_ENV !== "production",
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ac06c4a and 435925a.

📒 Files selected for processing (1)
  • docs/src/app/api/internal/[transport]/route.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Prefer ES6 Map over Record when representing key–value collections

Files:

  • docs/src/app/api/internal/[transport]/route.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
  • GitHub Check: restart-dev-and-test
  • GitHub Check: all-good
  • GitHub Check: check_prisma_migrations (22.x)
  • GitHub Check: docker
  • GitHub Check: setup-tests
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: docker
  • GitHub Check: build (22.x)
  • GitHub Check: build (22.x)
  • GitHub Check: Security Check
🔇 Additional comments (4)
docs/src/app/api/internal/[transport]/route.ts (4)

141-145: Set Zod defaults/bounds and rely on schema default (avoid handler default).

Keeps API consistent, prevents extreme limits, and removes duplication.

-        search_query: z.string().describe("The search query to find relevant documentation"),
-        result_limit: z.number().optional().describe("Maximum number of results to return (default: 50)")
+        search_query: z.string().min(1).describe("The search query to find relevant documentation"),
+        result_limit: z.number().int().min(1).max(100).default(50)
+          .describe("Maximum number of results to return (default: 50)")
       },
-      async ({ search_query, result_limit = 50 }) => {
+      async ({ search_query, result_limit }) => {

181-185: Guard against missing TOC to prevent runtime errors.

page.data.toc can be undefined.

-          // TOC/heading matching
-          for (const tocItem of page.data.toc) {
+          // TOC/heading matching
+          const toc = Array.isArray((page as any).data?.toc) ? (page as any).data.toc : [];
+          for (const tocItem of toc) {
             if (typeof tocItem.title === 'string' && tocItem.title.toLowerCase().includes(queryLower)) {
               score += 30;
             }
           }

189-191: Harden file path resolution to prevent path traversal.

Validate paths under content/ before reading.

-            const filePath = `content/${page.file.path}`;
-            const content = await readFile(filePath, "utf-8");
+            const contentRoot = path.resolve("content");
+            const resolvedPath = path.resolve(path.join("content", page.file.path));
+            if (!resolvedPath.startsWith(contentRoot + path.sep)) {
+              continue;
+            }
+            const content = await readFile(resolvedPath, "utf-8");

Add import near the top of the file:

 import { readFile } from "node:fs/promises";
+import path from "node:path";

187-191: Do not wrap these readFile calls with runAsynchronously — they are awaited and required by the handler.
runAsynchronously exists in the repo for fire‑and‑forget/background tasks, but the readFile calls in this route are awaited and their results are used to build the response; wrapping them would change behavior.
Locations: docs/src/app/api/internal/[transport]/route.ts — extractOpenApiDetails (lines ~25–29), content matching (170–192), primary path (270–284), altPaths (310–322), setup instructions (368–378).

Likely an incorrect or invalid review comment.

@cursor
Copy link
Copy Markdown

cursor bot commented Sep 25, 2025

You have run out of free Bugbot PR reviews for this billing cycle. This will reset on October 10.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (5)
docs/src/app/api/internal/[transport]/route.ts (5)

141-142: Address parameter naming convention.

Past reviews flagged that search_query and result_limit should follow the project's REST API naming convention. Since this has been previously raised, please refer to those comments for details.


145-149: Privacy concern with logging raw search queries.

A previous review flagged that capturing raw search_query values could raise privacy/compliance concerns. Consider logging metadata (e.g., query_length) instead of the actual query text.


181-185: Guard against undefined TOC.

A previous review flagged that page.data.toc might be undefined, causing a runtime error. This issue remains unaddressed.


189-191: Path traversal vulnerability in file reading.

Previous reviews flagged two critical security/reliability concerns here:

  1. Path traversal: page.file.path is used directly without validation
  2. Missing runAsynchronously wrapper for the async file read

Both issues remain unaddressed.


154-244: Sequential file reads may impact performance.

A previous review noted that file reads are performed sequentially in the loop. For many pages, consider parallelizing with Promise.all() to improve performance.

🧹 Nitpick comments (1)
docs/src/app/api/internal/[transport]/route.ts (1)

142-142: Add Zod validation bounds for result_limit.

The schema describes a default of 50 but doesn't enforce it via .default(50) or validate reasonable bounds. Consider adding .int().min(1).max(100).default(50) to prevent invalid values and make the default explicit in the schema.

Apply this diff:

-        result_limit: z.number().optional().describe("Maximum number of results to return (default: 50)")
+        result_limit: z.number().int().min(1).max(100).default(50).describe("Maximum number of results to return (default: 50)")

Then remove the handler default to rely on the schema:

-      async ({ search_query, result_limit = 50 }) => {
+      async ({ search_query, result_limit }) => {
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 26b9cf5 and e583748.

📒 Files selected for processing (1)
  • docs/src/app/api/internal/[transport]/route.ts (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.{ts,tsx,js,jsx}

📄 CodeRabbit inference engine (AGENTS.md)

Prefer ES6 Map over Record when representing key–value collections

Files:

  • docs/src/app/api/internal/[transport]/route.ts
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (10)
  • GitHub Check: Vercel Agent Review
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: setup-tests
  • GitHub Check: restart-dev-and-test
  • GitHub Check: docker
  • GitHub Check: build (22.x)
  • GitHub Check: build (22.x)
  • GitHub Check: all-good
  • GitHub Check: docker
  • GitHub Check: Security Check
🔇 Additional comments (1)
docs/src/app/api/internal/[transport]/route.ts (1)

246-260: LGTM!

The result sorting, limiting, and formatting logic is clear and correct. The implementation properly sorts by score descending and limits to the requested count.

@madster456 madster456 enabled auto-merge (squash) October 22, 2025 06:23
@madster456 madster456 disabled auto-merge October 22, 2025 06:24
@madster456 madster456 merged commit 1bc28c0 into dev Oct 22, 2025
18 of 21 checks passed
@madster456 madster456 deleted the docs-updated_search branch October 22, 2025 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants