Feature/add google search console #669
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughThis pull request integrates Google Search Console into the platform via OAuth2 authentication. It adds backend services for OAuth token management and search analytics data retrieval, frontend components for configuration and visualization, database schema extensions for token storage, and comprehensive API endpoints connecting the two layers. Changes
Sequence Diagram(s)sequenceDiagram
participant User as User/Browser
participant FrontendApp as Frontend App
participant BackendAPI as Backend API
participant GoogleOAuth as Google OAuth
participant SearchConsole as Google Search Console
participant Database as Database
rect rgb(240, 248, 255)
note over User,Database: OAuth2 Connection Flow
User->>FrontendApp: Click "Connect Search Console"
FrontendApp->>BackendAPI: GET /site/{siteId}/search-console/oauth/url
BackendAPI->>GoogleOAuth: Generate authorization URL (offline access)
GoogleOAuth-->>BackendAPI: Return authUrl with state
BackendAPI-->>FrontendApp: Return { authUrl }
FrontendApp->>GoogleOAuth: Redirect to Google consent screen
GoogleOAuth->>User: Show consent prompt
User->>GoogleOAuth: Grant permissions
GoogleOAuth->>BackendAPI: Redirect to callback with code + state
BackendAPI->>GoogleOAuth: Exchange code for tokens
GoogleOAuth-->>BackendAPI: Return access_token + refresh_token + expiry
BackendAPI->>Database: Store tokens & expiry in sites table
Database-->>BackendAPI: OK
BackendAPI->>FrontendApp: Redirect to /success
FrontendApp->>User: Show success message
end
rect rgb(240, 250, 240)
note over User,Database: Data Retrieval Flow
User->>FrontendApp: Navigate to Search Console dashboard
FrontendApp->>BackendAPI: GET /api/search-console/{site}?startDate=X&endDate=Y
BackendAPI->>Database: Get site tokens (access/refresh)
Database-->>BackendAPI: Return tokens + expiry
BackendAPI->>GoogleOAuth: Check if token expired (within 5 min window)
alt Token expired
GoogleOAuth->>GoogleOAuth: Use refresh_token to get new access_token
GoogleOAuth->>Database: Update tokens + expiry
end
BackendAPI->>SearchConsole: Fetch parallel queries (analytics by query, page, date, device, country, etc.)
SearchConsole-->>BackendAPI: Return raw analytics data
BackendAPI->>BackendAPI: Aggregate & process (totals, CTR, rankings, time series)
BackendAPI-->>FrontendApp: Return SearchConsoleData { clicks, impressions, topQueries, topPages, timeSeries, ... }
FrontendApp->>FrontendApp: Transform to chart-ready datasets
FrontendApp->>User: Render multi-tab dashboard with visualizations
end
rect rgb(255, 245, 238)
note over FrontendApp,Database: Site Discovery & Addition Flow
User->>FrontendApp: Click "Add Sites"
FrontendApp->>BackendAPI: GET /site/{siteId}/search-console/sites
BackendAPI->>GoogleOAuth: Obtain valid access_token for siteId
BackendAPI->>SearchConsole: List all accessible properties
SearchConsole-->>BackendAPI: Return list of Google properties
BackendAPI->>Database: Check which domains already exist in org
Database-->>BackendAPI: Return existing domains
BackendAPI-->>FrontendApp: Return { new sites, existing sites }
FrontendApp->>User: Show modal with available + added sites, checkboxes for selection
User->>FrontendApp: Select sites + click "Add"
FrontendApp->>BackendAPI: POST /site/{siteId}/search-console/add-sites
BackendAPI->>GoogleOAuth: Get valid token
BackendAPI->>SearchConsole: Verify selected sites exist
BackendAPI->>Database: INSERT new domains into sites table
Database-->>BackendAPI: OK
BackendAPI-->>FrontendApp: Return { added, existing, errors }
FrontendApp->>User: Show success count
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Possibly related PRs
Poem
Pre-merge checks and finishing touches✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 17
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (5)
client/src/app/components/AddSite.tsx (1)
104-112: Remove duplicate open prop.The
Dialogcomponent has theopenprop specified twice (lines 105 and 112). This will cause React to either throw a warning or use the last value, making the controlled state unpredictable.Apply this diff to remove the duplicate:
<Dialog open={open} onOpenChange={(isOpen) => { setOpen(isOpen); if (isOpen) { resetForm(); } }} - open={open} >server/src/index.ts (4)
201-217: OAuth callback must be public; add route to PUBLIC_ROUTES.Without this, Google’s redirect will hit auth gate and fail.
"/api/auth/callback/google", "/api/auth/callback/github", + "/api/search-console/oauth/callback", "/api/stripe/webhook",
218-253: Expose Search Console analytics endpoint as public-eligible.To allow public sites to fetch Search Console data without auth, include the prefix in
ANALYTICS_ROUTES.const ANALYTICS_ROUTES = [ + "/api/search-console/", "/api/live-user-count/", "/api/overview/", ... ];
154-167: CORS is permissive with credentials; restrict origins.
origin: (_origin, callback) => callback(null, true)reflects any origin whilecredentials: trueis set. This weakens session cookie protections.Use an allowlist (e.g., from config/env) and strict match:
-server.register(cors, { - origin: (_origin, callback) => { - callback(null, true); - }, +const ALLOWED_ORIGINS = (process.env.ALLOWED_ORIGINS || "") + .split(",") + .map((s) => s.trim()) + .filter(Boolean); +server.register(cors, { + origin: (origin, callback) => { + if (!origin || ALLOWED_ORIGINS.includes(origin)) return callback(null, true); + return callback(new Error("Not allowed by CORS"), false); + }, methods: ["GET", "POST", "PUT", "DELETE", "OPTIONS", "PATCH"], allowedHeaders: ["Content-Type", "Authorization", "X-Requested-With"], credentials: true, });If you already load allowed domains elsewhere, wire that source instead. Based on coding guidelines.
255-290: Auth bypass match usesincludes— tighten to avoid accidental bypass.
processedUrl.includes(route)may match unrelated paths (e.g.,/api/authenticate) and skip auth.- if (PUBLIC_ROUTES.some((route) => processedUrl.includes(route))) { + if (PUBLIC_ROUTES.some((route) => processedUrl === route || processedUrl.startsWith(route))) { return; }Similarly consider normalizing trailing slashes.
🧹 Nitpick comments (15)
server/drizzle/meta/_journal.json (1)
27-27: Add trailing newline for formatting consistency.The file is missing a trailing newline. While this is an auto-generated file managed by Drizzle ORM, check whether your project's formatting rules (
.editorconfig,.prettierrc, or linters) enforce trailing newlines.If required by project standards, ensure Drizzle ORM respects your formatter config. Otherwise, you can add a trailing newline to the file:
] } +GOOGLE_SEARCH_CONSOLE_SETUP.md (1)
62-62: Consider "revoke" instead of "remove" for clarity.For the disconnect option, consider using "revoke" instead of "remove" as it's more precise terminology for OAuth access revocation.
-- ✅ **Disconnect option** (remove access anytime) +- ✅ **Disconnect option** (revoke access anytime)client/src/app/[site]/search-console/README.md (1)
88-94: Add language identifier to fenced code block.The fenced code block for the file structure should have a language identifier to satisfy markdown linting rules.
As per coding guidelines:
Apply this diff:
-``` +```text search-console/ ├── page.tsx # Main page component ├── components/ │ └── ConnectionStatus.tsx # Connection status component └── README.md # This documentation</blockquote></details> <details> <summary>server/src/api/sites/updateSearchConsoleApiKey.ts (1)</summary><blockquote> `10-12`: **Consider adding API key format validation.** The schema validates that the API key is a non-empty string but doesn't validate its format. If Google Search Console API keys have a specific format (e.g., length, character set), consider adding validation. Example: ```diff const updateSearchConsoleApiKeySchema = z.object({ - apiKey: z.string().min(1, "API key is required"), + apiKey: z.string() + .min(1, "API key is required") + .regex(/^[A-Za-z0-9_-]+$/, "Invalid API key format"), });Adjust the regex based on the actual Google Search Console API key format requirements.
server/src/services/SEARCH_CONSOLE_IMPLEMENTATION.md (1)
40-45: Add language identifier to environment variables code block.The code block should specify a language for proper syntax highlighting and to satisfy markdown linting rules.
Apply this diff:
Add these to your `.env` file: -``` +```bash GOOGLE_CLIENT_ID=your_client_id_here GOOGLE_CLIENT_SECRET=your_client_secret_here BASE_URL=http://localhost:3001 # or your production URL</blockquote></details> <details> <summary>server/src/api/searchConsole/getSearchConsoleSites.ts (1)</summary><blockquote> `46-48`: **Prefer structured logging over `console.log`.** Switch these statements to the shared logger (e.g., `request.log` or `logger`) so they inherit log levels, formatting, and redaction policies instead of writing directly to stdout. This keeps operational telemetry consistent across endpoints. </blockquote></details> <details> <summary>client/src/api/analytics/searchConsole/useGetSearchConsoleSites.ts (1)</summary><blockquote> `21-40`: **Trim verbose client-side logging.** These `console.log` calls will ship to users’ browsers and can leak implementation details or clutter diagnostics. Consider removing them or guarding behind a debug flag before release. </blockquote></details> <details> <summary>client/src/app/[site]/search-console/oauth/error/page.tsx (1)</summary><blockquote> `12-33`: **Narrow the route param type** `useParams()` returns `string | string[] | undefined`, so `siteId` can become an array if this route ever turns into a catch-all segment. Normalizing to a string (or throwing when array) keeps navigation predictable and eliminates the union in downstream code. ([nextjs.org](https://nextjs.org/docs/app/api-reference/functions/use-params?utm_source=openai)) ```diff - const params = useParams(); - const siteId = params.site; + const params = useParams(); + const rawSite = params.site; + const siteId = Array.isArray(rawSite) ? rawSite[0] : rawSite ?? ""; + if (!siteId) { + // optionally log or route away + return null; + }server/public/script-full.js (2)
62-73: Make analytics host extraction robust for both/script.jsand/script-full.js.Current
src.split("/script.js")[0]fails if the tag loadsscript-full.js. Use a regex on the full href.- const analyticsHost = src.split("/script.js")[0]; + const url = new URL(src, window.location.href); + // Strip trailing /script.js or /script-full.js (with optional query/hash) + const analyticsHost = url.href.replace(/\/script(?:-full)?\.js(?:[?#].*)?$/i, "");
421-466: Client error tracking: good origin guard; consider sampling to avoid noise.You already filter cross-origin stacks. Add optional sampling (e.g., 10–20%) to reduce backend load during error storms.
- this.track("error", error.name || "Error", errorProperties); + // Optional: sample errors at 20% to reduce volume + const SAMPLE_RATE = 0.2; + if (Math.random() < SAMPLE_RATE) { + this.track("error", error.name || "Error", errorProperties); + }server/src/index.ts (1)
292-296: Serve/api/script-full.jslike other assets or remove from allowlist.You added it to PUBLIC_ROUTES but didn’t register a handler. Either serve it or drop it from the list.
+server.get("/api/script-full.js", async (_, reply) => reply.sendFile("script-full.js"));server/src/services/googleOAuthService.ts (2)
6-16: Typeoauth2Clientexplicitly.Use the official type for better IntelliSense and safety.
-import { google } from 'googleapis'; +import { google } from 'googleapis'; +import type { OAuth2Client } from 'google-auth-library'; ... - private oauth2Client: any; + private oauth2Client: OAuth2Client;
98-110: Refresh flow: handle deprecations and persist rotated refresh token.
refreshAccessToken()is deprecated; preferthis.oauth2Client.refreshAccessToken()only if your version supports it, otherwisethis.oauth2Client.refreshToken(...). If Google rotates refresh tokens, persistcredentials.refresh_tokenwhen present.- const { credentials } = await this.oauth2Client.refreshAccessToken(); + const { credentials } = await this.oauth2Client.refreshAccessToken(); await db .update(sites) .set({ searchConsoleAccessToken: credentials.access_token, searchConsoleTokenExpiry: credentials.expiry_date ? new Date(credentials.expiry_date) : null, + ...(credentials.refresh_token ? { searchConsoleRefreshToken: credentials.refresh_token } : {}), })server/src/api/searchConsole/oauth.ts (1)
69-88: Use a dedicatedFRONTEND_URLenv and avoid hardcoded port swapping; remove risky default siteId.Relying on
BASE_URL.replace('3001','3002')is brittle. Also avoid defaulting to site1on errors.- const frontendUrl = process.env.BASE_URL?.replace('3001', '3002') || 'http://localhost:3002'; + const frontendUrl = process.env.FRONTEND_URL || 'http://localhost:3002'; ... - const siteId = request.query.state || '1'; // Default to site 1 if state is not available - const frontendUrl = process.env.BASE_URL?.replace('3001', '3002') || 'http://localhost:3002'; - return reply.redirect(`${frontendUrl}/${siteId}/search-console/oauth/error?message=${encodeURIComponent('Failed to complete OAuth flow')}`); + const siteId = request.query.state ?? ''; + const frontendUrl = process.env.FRONTEND_URL || 'http://localhost:3002'; + const path = siteId ? `/${siteId}/search-console/oauth/error` : `/search-console/oauth/error`; + return reply.redirect(`${frontendUrl}${path}?message=${encodeURIComponent('Failed to complete OAuth flow')}`);Pair this with the signed
statechange suggested in the service.client/src/app/[site]/search-console/page.tsx (1)
98-104: AvoidMath.random()in render — causes SSR/client mismatch.Derive colors deterministically from the country string.
- const countryData = searchConsoleData.countryBreakdown.map(item => ({ - id: item.country, - label: item.country, - value: item.clicks, - color: `hsl(${Math.random() * 360}, 70%, 50%)` - })); + const hueFor = (s: string) => { + let h = 0; + for (let i = 0; i < s.length; i++) h = (h * 31 + s.charCodeAt(i)) | 0; + return (h >>> 0) % 360; + }; + const countryData = searchConsoleData.countryBreakdown.map((item) => ({ + id: item.country, + label: item.country, + value: item.clicks, + color: `hsl(${hueFor(item.country)}, 70%, 50%)`, + }));
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (2)
client/pnpm-lock.yamlis excluded by!**/pnpm-lock.yamlserver/package-lock.jsonis excluded by!**/package-lock.json
📒 Files selected for processing (45)
GOOGLE_SEARCH_CONSOLE_SETUP.md(1 hunks)client/package.json(2 hunks)client/src/api/analytics/searchConsole/useAddSitesFromSearchConsole.ts(1 hunks)client/src/api/analytics/searchConsole/useGetSearchConsoleApiKey.ts(1 hunks)client/src/api/analytics/searchConsole/useGetSearchConsoleData.ts(1 hunks)client/src/api/analytics/searchConsole/useGetSearchConsoleSites.ts(1 hunks)client/src/api/analytics/searchConsole/useSearchConsoleOAuth.ts(1 hunks)client/src/api/analytics/searchConsole/useUpdateSearchConsoleApiKey.ts(1 hunks)client/src/app/[site]/components/Sidebar/Sidebar.tsx(2 hunks)client/src/app/[site]/search-console/README.md(1 hunks)client/src/app/[site]/search-console/components/AddSitesModal.tsx(1 hunks)client/src/app/[site]/search-console/components/ConnectionStatus.tsx(1 hunks)client/src/app/[site]/search-console/components/SearchConsoleApiKeyModal.tsx(1 hunks)client/src/app/[site]/search-console/components/ShowSitesModal.tsx(1 hunks)client/src/app/[site]/search-console/oauth/error/page.tsx(1 hunks)client/src/app/[site]/search-console/oauth/success/page.tsx(1 hunks)client/src/app/[site]/search-console/page.tsx(1 hunks)client/src/app/components/AddSite.tsx(3 hunks)client/src/lib/store.ts(1 hunks)server/drizzle.config.ts(1 hunks)server/drizzle/0000_wealthy_vulture.sql(1 hunks)server/drizzle/0001_far_shinko_yamashiro.sql(1 hunks)server/drizzle/0002_aspiring_cable.sql(1 hunks)server/drizzle/meta/0000_snapshot.json(1 hunks)server/drizzle/meta/0001_snapshot.json(1 hunks)server/drizzle/meta/0002_snapshot.json(1 hunks)server/drizzle/meta/_journal.json(1 hunks)server/drizzle/relations.ts(1 hunks)server/drizzle/schema.ts(1 hunks)server/package.json(1 hunks)server/public/script-full.js(4 hunks)server/public/script.js(1 hunks)server/src/api/analytics/searchConsole/getSearchConsoleData.ts(1 hunks)server/src/api/searchConsole/addSitesFromSearchConsole.ts(1 hunks)server/src/api/searchConsole/getSearchConsoleSites.ts(1 hunks)server/src/api/searchConsole/oauth.ts(1 hunks)server/src/api/sites/addSite.ts(1 hunks)server/src/api/sites/getSearchConsoleApiKey.ts(1 hunks)server/src/api/sites/updateSearchConsoleApiKey.ts(1 hunks)server/src/db/postgres/schema.ts(1 hunks)server/src/index.ts(4 hunks)server/src/lib/siteConfig.ts(4 hunks)server/src/services/SEARCH_CONSOLE_IMPLEMENTATION.md(1 hunks)server/src/services/googleOAuthService.ts(1 hunks)server/src/services/searchConsoleService.ts(1 hunks)
🧰 Additional context used
📓 Path-based instructions (3)
{client,server}/**/*.{ts,tsx}
📄 CodeRabbit inference engine (CLAUDE.md)
{client,server}/**/*.{ts,tsx}: Use TypeScript with strict typing throughout both client and server
Use try/catch blocks with specific error types for error handling
Use camelCase for variables and functions, PascalCase for components and types
Group imports by external, then internal, and sort alphabetically within groups
Files:
server/src/api/searchConsole/getSearchConsoleSites.tsclient/src/app/[site]/search-console/components/ConnectionStatus.tsxserver/src/api/sites/updateSearchConsoleApiKey.tsclient/src/app/components/AddSite.tsxserver/src/lib/siteConfig.tsclient/src/api/analytics/searchConsole/useGetSearchConsoleSites.tsclient/src/api/analytics/searchConsole/useAddSitesFromSearchConsole.tsserver/src/db/postgres/schema.tsserver/src/api/sites/getSearchConsoleApiKey.tsclient/src/app/[site]/search-console/components/SearchConsoleApiKeyModal.tsxclient/src/app/[site]/search-console/oauth/success/page.tsxclient/src/app/[site]/search-console/components/AddSitesModal.tsxserver/src/api/analytics/searchConsole/getSearchConsoleData.tsserver/src/api/searchConsole/addSitesFromSearchConsole.tsclient/src/lib/store.tsserver/src/api/searchConsole/oauth.tsserver/src/services/googleOAuthService.tsclient/src/app/[site]/search-console/components/ShowSitesModal.tsxclient/src/api/analytics/searchConsole/useGetSearchConsoleApiKey.tsserver/src/api/sites/addSite.tsclient/src/api/analytics/searchConsole/useUpdateSearchConsoleApiKey.tsclient/src/app/[site]/components/Sidebar/Sidebar.tsxserver/drizzle/schema.tsclient/src/api/analytics/searchConsole/useSearchConsoleOAuth.tsclient/src/api/analytics/searchConsole/useGetSearchConsoleData.tsserver/src/services/searchConsoleService.tsclient/src/app/[site]/search-console/page.tsxserver/drizzle/relations.tsserver/drizzle.config.tsserver/src/index.tsclient/src/app/[site]/search-console/oauth/error/page.tsx
server/**/*
📄 CodeRabbit inference engine (CLAUDE.md)
Backend: Use Fastify, Drizzle ORM (Postgres), ClickHouse, and Zod
Files:
server/src/api/searchConsole/getSearchConsoleSites.tsserver/src/services/SEARCH_CONSOLE_IMPLEMENTATION.mdserver/package.jsonserver/src/api/sites/updateSearchConsoleApiKey.tsserver/src/lib/siteConfig.tsserver/src/db/postgres/schema.tsserver/src/api/sites/getSearchConsoleApiKey.tsserver/src/api/analytics/searchConsole/getSearchConsoleData.tsserver/drizzle/0001_far_shinko_yamashiro.sqlserver/src/api/searchConsole/addSitesFromSearchConsole.tsserver/src/api/searchConsole/oauth.tsserver/src/services/googleOAuthService.tsserver/drizzle/0002_aspiring_cable.sqlserver/public/script.jsserver/src/api/sites/addSite.tsserver/drizzle/meta/0002_snapshot.jsonserver/public/script-full.jsserver/drizzle/schema.tsserver/drizzle/0000_wealthy_vulture.sqlserver/drizzle/meta/0001_snapshot.jsonserver/src/services/searchConsoleService.tsserver/drizzle/relations.tsserver/drizzle.config.tsserver/drizzle/meta/_journal.jsonserver/src/index.tsserver/drizzle/meta/0000_snapshot.json
client/**/*
📄 CodeRabbit inference engine (CLAUDE.md)
Frontend: Use Next.js, Tailwind CSS, Shadcn UI, Tanstack Query, Zustand, Luxon, Nivo, and react-hook-form
Files:
client/src/app/[site]/search-console/components/ConnectionStatus.tsxclient/src/app/components/AddSite.tsxclient/package.jsonclient/src/api/analytics/searchConsole/useGetSearchConsoleSites.tsclient/src/api/analytics/searchConsole/useAddSitesFromSearchConsole.tsclient/src/app/[site]/search-console/components/SearchConsoleApiKeyModal.tsxclient/src/app/[site]/search-console/oauth/success/page.tsxclient/src/app/[site]/search-console/components/AddSitesModal.tsxclient/src/lib/store.tsclient/src/app/[site]/search-console/README.mdclient/src/app/[site]/search-console/components/ShowSitesModal.tsxclient/src/api/analytics/searchConsole/useGetSearchConsoleApiKey.tsclient/src/api/analytics/searchConsole/useUpdateSearchConsoleApiKey.tsclient/src/app/[site]/components/Sidebar/Sidebar.tsxclient/src/api/analytics/searchConsole/useSearchConsoleOAuth.tsclient/src/api/analytics/searchConsole/useGetSearchConsoleData.tsclient/src/app/[site]/search-console/page.tsxclient/src/app/[site]/search-console/oauth/error/page.tsx
🧠 Learnings (4)
📚 Learning: 2025-08-03T17:30:25.559Z
Learnt from: CR
Repo: rybbit-io/rybbit PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-08-03T17:30:25.559Z
Learning: Applies to server/**/* : Backend: Use Fastify, Drizzle ORM (Postgres), ClickHouse, and Zod
Applied to files:
server/package.jsonserver/src/api/sites/updateSearchConsoleApiKey.tsserver/drizzle/schema.tsserver/drizzle/relations.ts
📚 Learning: 2025-08-03T17:30:25.559Z
Learnt from: CR
Repo: rybbit-io/rybbit PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-08-03T17:30:25.559Z
Learning: Applies to client/**/*.{tsx} : Client: Use React functional components with minimal useEffect and inline functions
Applied to files:
client/src/app/[site]/search-console/components/ConnectionStatus.tsxclient/src/app/[site]/search-console/components/SearchConsoleApiKeyModal.tsxclient/src/app/[site]/search-console/oauth/success/page.tsxclient/src/app/[site]/search-console/components/AddSitesModal.tsxclient/src/app/[site]/search-console/oauth/error/page.tsx
📚 Learning: 2025-08-03T17:30:25.559Z
Learnt from: CR
Repo: rybbit-io/rybbit PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-08-03T17:30:25.559Z
Learning: Applies to client/**/* : Frontend: Use Next.js, Tailwind CSS, Shadcn UI, Tanstack Query, Zustand, Luxon, Nivo, and react-hook-form
Applied to files:
client/src/app/[site]/components/Sidebar/Sidebar.tsxclient/src/app/[site]/search-console/page.tsxclient/src/app/[site]/search-console/oauth/error/page.tsx
📚 Learning: 2025-09-13T04:45:39.436Z
Learnt from: nktnet1
Repo: rybbit-io/rybbit PR: 270
File: server/src/db/clickhouse/clickhouse.ts:7-7
Timestamp: 2025-09-13T04:45:39.436Z
Learning: In the rybbit codebase, environment variables for ClickHouse configuration (CLICKHOUSE_HOST, CLICKHOUSE_DB, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD) are used directly from process.env without default values to maintain consistency across the configuration pattern.
Applied to files:
server/drizzle.config.ts
🧬 Code graph analysis (25)
server/src/api/searchConsole/getSearchConsoleSites.ts (4)
server/src/lib/auth-utils.ts (1)
getUserHasAdminAccessToSite(123-126)server/src/db/postgres/postgres.ts (1)
db(20-20)server/src/db/postgres/schema.ts (1)
sites(51-85)server/src/services/searchConsoleService.ts (1)
searchConsoleService(919-919)
client/src/app/[site]/search-console/components/ConnectionStatus.tsx (1)
server/src/services/googleOAuthService.ts (1)
isConnected(139-146)
server/src/api/sites/updateSearchConsoleApiKey.ts (4)
server/src/lib/auth-utils.ts (1)
getUserHasAdminAccessToSite(123-126)server/src/db/postgres/postgres.ts (1)
db(20-20)server/src/db/postgres/schema.ts (1)
sites(51-85)server/src/lib/siteConfig.ts (1)
siteConfig(271-271)
server/src/lib/siteConfig.ts (2)
server/drizzle/schema.ts (1)
sites(49-73)server/src/db/postgres/schema.ts (1)
sites(51-85)
client/src/api/analytics/searchConsole/useGetSearchConsoleSites.ts (1)
client/src/lib/const.ts (1)
BACKEND_URL(1-4)
client/src/api/analytics/searchConsole/useAddSitesFromSearchConsole.ts (1)
client/src/lib/const.ts (1)
BACKEND_URL(1-4)
server/src/api/sites/getSearchConsoleApiKey.ts (3)
server/src/lib/auth-utils.ts (1)
getUserHasAdminAccessToSite(123-126)server/src/db/postgres/postgres.ts (1)
db(20-20)server/src/db/postgres/schema.ts (1)
sites(51-85)
client/src/app/[site]/search-console/components/SearchConsoleApiKeyModal.tsx (1)
client/src/api/analytics/searchConsole/useUpdateSearchConsoleApiKey.ts (1)
useUpdateSearchConsoleApiKey(10-43)
client/src/app/[site]/search-console/components/AddSitesModal.tsx (1)
server/src/db/postgres/schema.ts (1)
sites(51-85)
server/src/api/analytics/searchConsole/getSearchConsoleData.ts (4)
client/src/api/analytics/searchConsole/useGetSearchConsoleData.ts (1)
SearchConsoleData(9-128)server/src/services/searchConsoleService.ts (2)
getSearchConsoleData(426-574)searchConsoleService(919-919)shared/src/params.ts (1)
FilterParams(10-10)server/src/lib/auth-utils.ts (1)
getUserHasAccessToSitePublic(113-116)
server/src/api/searchConsole/addSitesFromSearchConsole.ts (4)
server/src/lib/auth-utils.ts (1)
getUserHasAdminAccessToSite(123-126)server/src/db/postgres/postgres.ts (1)
db(20-20)server/src/db/postgres/schema.ts (1)
sites(51-85)server/src/services/searchConsoleService.ts (1)
searchConsoleService(919-919)
client/src/lib/store.ts (1)
shared/src/filters.ts (1)
FilterParameter(3-27)
server/src/api/searchConsole/oauth.ts (2)
server/src/lib/auth-utils.ts (1)
getUserHasAdminAccessToSite(123-126)server/src/services/googleOAuthService.ts (2)
googleOAuthService(149-149)isConnected(139-146)
server/src/services/googleOAuthService.ts (2)
server/src/db/postgres/postgres.ts (1)
db(20-20)server/src/db/postgres/schema.ts (1)
sites(51-85)
client/src/app/[site]/search-console/components/ShowSitesModal.tsx (5)
client/src/lib/store.ts (2)
useStore(149-276)resetStore(278-286)client/src/lib/auth.ts (1)
authClient(8-15)client/src/lib/utils.ts (2)
isValidDomain(87-90)normalizeDomain(61-79)server/src/api/sites/addSite.ts (1)
addSite(8-117)server/src/db/postgres/schema.ts (1)
sites(51-85)
client/src/api/analytics/searchConsole/useGetSearchConsoleApiKey.ts (1)
client/src/api/utils.ts (1)
authedFetch(56-90)
client/src/api/analytics/searchConsole/useUpdateSearchConsoleApiKey.ts (1)
client/src/api/utils.ts (1)
authedFetch(56-90)
server/public/script-full.js (1)
server/public/web-vitals.iife.js (22)
i(1-1)i(1-1)o(1-1)o(1-1)o(1-1)e(1-1)e(1-1)e(1-1)l(1-1)m(1-1)t(1-1)t(1-1)t(1-1)n(1-1)f(1-1)_(1-1)C(1-1)I(1-1)w(1-1)k(1-1)B(1-1)q(1-1)
server/drizzle/schema.ts (1)
server/src/db/postgres/schema.ts (11)
user(18-39)organization(146-160)sites(51-85)goals(235-260)telemetry(263-270)uptimeAlerts(411-435)uptimeMonitors(273-375)uptimeAlertHistory(438-464)uptimeMonitorStatus(378-408)notificationChannels(536-588)uptimeIncidents(477-533)
client/src/api/analytics/searchConsole/useSearchConsoleOAuth.ts (1)
client/src/api/utils.ts (1)
authedFetch(56-90)
client/src/api/analytics/searchConsole/useGetSearchConsoleData.ts (2)
client/src/lib/store.ts (3)
useStore(149-276)getFilteredFilters(424-427)SEARCH_CONSOLE_PAGE_FILTERS(103-119)client/src/api/utils.ts (2)
getQueryParams(35-54)authedFetch(56-90)
server/src/services/searchConsoleService.ts (3)
client/src/api/analytics/searchConsole/useGetSearchConsoleData.ts (1)
SearchConsoleData(9-128)server/src/db/postgres/postgres.ts (2)
db(20-20)sql(23-23)server/src/db/postgres/schema.ts (1)
sites(51-85)
client/src/app/[site]/search-console/page.tsx (1)
client/src/api/analytics/searchConsole/useGetSearchConsoleData.ts (1)
useGetSearchConsoleData(130-154)
server/drizzle/relations.ts (1)
server/drizzle/schema.ts (15)
invitation(6-25)user(165-185)organization(35-47)sites(49-73)member(75-92)session(94-112)account(114-134)funnels(136-154)uptimeMonitors(256-282)notificationChannels(309-334)uptimeIncidents(336-374)goals(187-200)uptimeAlerts(211-225)uptimeAlertHistory(227-245)uptimeMonitorStatus(284-307)
server/src/index.ts (7)
server/src/api/analytics/searchConsole/getSearchConsoleData.ts (1)
getSearchConsoleData(28-92)server/src/services/searchConsoleService.ts (1)
getSearchConsoleData(426-574)server/src/api/sites/getSearchConsoleApiKey.ts (1)
getSearchConsoleApiKey(7-58)server/src/api/sites/updateSearchConsoleApiKey.ts (1)
updateSearchConsoleApiKey(14-73)server/src/api/searchConsole/oauth.ts (4)
generateOAuthUrl(6-43)handleOAuthCallback(46-89)disconnectSearchConsole(92-129)getConnectionStatus(132-169)server/src/api/searchConsole/getSearchConsoleSites.ts (1)
getSearchConsoleSites(5-62)server/src/api/searchConsole/addSitesFromSearchConsole.ts (1)
addSitesFromSearchConsole(5-60)
🪛 LanguageTool
server/src/services/SEARCH_CONSOLE_IMPLEMENTATION.md
[style] ~8-~8: ‘with success’ might be wordy. Consider a shorter alternative.
Context: ...nd Integration**: OAuth2 flow in the UI with success/error pages ✅ Service Layer: Search...
(EN_WORDINESS_PREMIUM_WITH_SUCCESS)
GOOGLE_SEARCH_CONSOLE_SETUP.md
[style] ~62-~62: Consider using a different verb to strengthen your wording.
Context: ...e connected) - ✅ Disconnect option (remove access anytime) - ✅ Error handling ...
(REMOVE_REVOKE)
🪛 markdownlint-cli2 (0.18.1)
server/src/services/SEARCH_CONSOLE_IMPLEMENTATION.md
41-41: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
client/src/app/[site]/search-console/README.md
88-88: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (18)
server/drizzle/meta/_journal.json (1)
1-27: No issues found; journal entries correctly reference and track GSC OAuth migration files.The three migration files exist and are properly indexed in the journal:
0000_wealthy_vulture.sql: Initial schema (12.5KB with all core tables)0001_far_shinko_yamashiro.sql: Addssearch_console_api_keycolumn0002_aspiring_cable.sql: Addssearch_console_access_token,search_console_refresh_token, andsearch_console_token_expirycolumns for OAuth2 token managementThe schema changes appropriately support Google Search Console OAuth2 integration. The auto-generated journal file is correctly structured with matching entries for each migration.
server/public/script.js (1)
1-1: No changes detected inserver/public/script.js— review comment is incorrect.Git confirms the working tree is clean with no changes to the source files (
server/src/analytics-script/*) or the minified output (server/public/script.js). Both files share identical timestamps (Nov 3 05:46), indicating no rebuild occurred in this PR.The review comment references changes to the minified script, but
git diff HEAD -- server/public/script.jsreturns no diff. The source files are also unchanged (git diff HEAD -- server/src/analytics-script/returns empty). This file is not part of the PR changes, so the review comment's concerns do not apply.Likely an incorrect or invalid review comment.
client/src/app/[site]/components/Sidebar/Sidebar.tsx (2)
19-19: LGTM!The
Searchicon import is correctly added and appropriate for the Search Console feature.
97-102: Search Console route verified and exists.The sidebar implementation is correct. The corresponding page route at
client/src/app/[site]/search-console/page.tsxexists, along with OAuth callback pages.client/package.json (1)
6-6: Verify removal of --turbopack flag.The --turbopack flag has been removed from the dev script. According to Next.js 15 documentation, Turbopack is stable in dev mode, but this change may impact development performance.
Please confirm this removal is intentional and not causing any development workflow issues.
server/drizzle.config.ts (1)
12-12: LGTM with minor edge case note.The change to explicitly parse the port as a number is good practice. However, be aware that
Number("0")returns0(truthy), so if someone setsPOSTGRES_PORT=0, it would use port 0 instead of falling back to 5432. This is an unlikely edge case in practice.client/src/api/analytics/searchConsole/useGetSearchConsoleApiKey.ts (1)
1-21: LGTM!The hook is well-structured and follows React Query best practices:
- Proper query key namespacing with site-specific cache
- Type-safe response handling
- Appropriate enabled condition to prevent unnecessary requests
- Clean integration with the authedFetch utility
server/drizzle/0001_far_shinko_yamashiro.sql (1)
1-11: Verify migration complexity is necessary.This migration drops and recreates constraints and an index on
uptime_monitor_statustable just to add a column to thesitestable. This seems unnecessarily complex and risky.Please verify:
- Why are the
uptime_monitor_statusconstraints being dropped and recreated?- Is this a Drizzle ORM artifact or is there a dependency between these operations?
- Could this be split into separate, simpler migrations?
If the constraint manipulation is required by Drizzle's migration system, consider adding a comment in the migration explaining why to help future maintainers.
client/src/app/components/AddSite.tsx (1)
27-27: LGTM - prefilledDomain feature.The addition of the
prefilledDomainprop and its integration into the form reset logic is well-implemented. This allows the component to be initialized with a pre-populated domain value when needed.Also applies to: 79-79
GOOGLE_SEARCH_CONSOLE_SETUP.md (1)
1-89: Excellent documentation!The setup guide is comprehensive, well-structured, and provides clear step-by-step instructions. The troubleshooting section is particularly helpful.
server/src/api/sites/addSite.ts (1)
107-107: LGTM!The addition of
searchConsoleApiKeyto the site config cache correctly propagates the field from the database. This aligns with the schema changes and enables the Search Console integration feature.client/src/lib/store.ts (1)
103-119: Consider including UTM parameters.The
SEARCH_CONSOLE_PAGE_FILTERSconstant is well-structured and consistent with existing filter patterns. However, it excludes UTM parameters (utm_source,utm_medium,utm_campaign,utm_term,utm_content) that are present inSESSION_PAGE_FILTERS.If UTM tracking is relevant for Search Console analytics (e.g., for correlating organic search performance with campaign data), consider including these parameters:
export const SEARCH_CONSOLE_PAGE_FILTERS: FilterParameter[] = [ "hostname", "browser", "browser_version", "operating_system", "operating_system_version", "language", "country", "region", "city", "device_type", "referrer", "event_name", "channel", "entry_page", "exit_page", + "utm_source", + "utm_medium", + "utm_campaign", + "utm_term", + "utm_content", ];Otherwise, the current filter set is appropriate for basic Search Console data filtering.
client/src/app/[site]/search-console/oauth/success/page.tsx (1)
21-34: LGTM!The success UI provides clear user feedback with appropriate styling and messaging. The redirect notification helps users understand what's happening.
server/src/services/SEARCH_CONSOLE_IMPLEMENTATION.md (1)
40-45: BASE_URL is already properly documented in.env.example.The verification confirms that
BASE_URLis documented in the project's.env.examplefile with clear context:# Domain and URL Configuration DOMAIN_NAME=demo.rybbit.io BASE_URL="https://${DOMAIN_NAME}"This satisfies the configuration documentation requirement. The variable is placed in the standard location (
.env.example) with appropriate comments explaining its purpose in the domain and URL configuration section.server/public/script-full.js (1)
823-842: Opt‑out stub includeserror()method — good.Public API surface remains consistent even when disabled.
client/src/app/[site]/search-console/page.tsx (1)
21-24: Client connect flow assumes server redirect — OK if server is updated.After changing the server to redirect from
/api/site/:siteId/search-console/oauth/url, this is correct. If you keep JSON, switch tofetchthenwindow.location.href = authUrl.client/src/api/analytics/searchConsole/useGetSearchConsoleData.ts (1)
130-154: Hook looks good; query key includes time and filters.No issues spotted; types align with server payload.
server/drizzle/meta/0001_snapshot.json (1)
797-885: ****The review comment misidentifies the snapshot state. The file
server/drizzle/meta/0001_snapshot.jsonis the baseline schema prior to OAuth token support. The OAuth columns (search_console_access_token,search_console_refresh_token,search_console_token_expiry) are properly introduced in the subsequent migrationserver/drizzle/0002_aspiring_cable.sqland reflected inserver/drizzle/meta/0002_snapshot.json. The schema inserver/src/db/postgres/schema.tsand usage inserver/src/services/googleOAuthService.tsare aligned with the latest snapshot state. No migration or schema mismatches exist.Likely an incorrect or invalid review comment.
| "date-fns": "^4.1.0", | ||
| "framer-motion": "12.11.0", | ||
| "geojson": "0.5.0", | ||
| "googleapis": "^155.0.0", |
There was a problem hiding this comment.
Move googleapis to server dependencies - security and architecture issue.
The googleapis library is a Node.js server-side library that should NOT be included in client dependencies. Including it in the client package will:
- Bloat the frontend bundle with unnecessary server-side code
- Potentially expose OAuth credentials if any client code attempts to use it
- Violate the architectural separation between client and server
All Google Search Console API interactions should happen exclusively on the server via the existing server-side services.
Apply this diff to remove the client-side dependency:
- "googleapis": "^155.0.0",Then verify that googleapis is properly listed in server/package.json dependencies instead.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "googleapis": "^155.0.0", |
🤖 Prompt for AI Agents
In client/package.json around line 56, remove the "googleapis" entry from the
client dependencies and ensure no client code imports it; then add "googleapis"
to server/package.json dependencies (not devDependencies) so all Google Search
Console API calls run server-side; after making these edits run the appropriate
package manager install (npm/yarn/pnpm) in both client and server to update
lockfiles and confirm the client bundle no longer contains googleapis and
server/package.json lists the dependency.
| mutationFn: async (siteId: number) => { | ||
| const response = await authedFetch<{ | ||
| success: boolean; | ||
| message: string; | ||
| }>(`/site/${siteId}/search-console/disconnect`, { | ||
| method: "POST", | ||
| }); |
There was a problem hiding this comment.
Fix the disconnect request to actually send a POST
authedFetch treats its second argument as query params, so this call is issuing a GET with ?method=POST instead of hitting the POST endpoint. The disconnect route will never run. Pass the HTTP method through the Axios config (third argument) so the request is truly POST.
- const response = await authedFetch<{
- success: boolean;
- message: string;
- }>(`/site/${siteId}/search-console/disconnect`, {
- method: "POST",
- });
+ const response = await authedFetch<{
+ success: boolean;
+ message: string;
+ }>(
+ `/site/${siteId}/search-console/disconnect`,
+ undefined,
+ { method: "POST" }
+ );📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| mutationFn: async (siteId: number) => { | |
| const response = await authedFetch<{ | |
| success: boolean; | |
| message: string; | |
| }>(`/site/${siteId}/search-console/disconnect`, { | |
| method: "POST", | |
| }); | |
| mutationFn: async (siteId: number) => { | |
| const response = await authedFetch<{ | |
| success: boolean; | |
| message: string; | |
| }>( | |
| `/site/${siteId}/search-console/disconnect`, | |
| undefined, | |
| { method: "POST" } | |
| ); |
🤖 Prompt for AI Agents
In client/src/api/analytics/searchConsole/useSearchConsoleOAuth.ts around lines
45 to 51, the call to authedFetch is passing { method: "POST" } as the second
argument which authedFetch treats as query params, causing a GET with
?method=POST; change the call to pass no query params (or an appropriate params
object) as the second argument and move the { method: "POST" } into the third
argument (Axios config) so the request is sent as an actual POST to
/site/{siteId}/search-console/disconnect.
| mutationFn: async ({ siteId, apiKey }: UpdateSearchConsoleApiKeyParams) => { | ||
| const response = await authedFetch<{ | ||
| success: boolean; | ||
| data: { searchConsoleApiKey: string }; | ||
| }>(`/site/${siteId}/search-console-api-key`, { | ||
| method: "POST", | ||
| headers: { | ||
| "Content-Type": "application/json", | ||
| }, | ||
| body: JSON.stringify({ apiKey }), | ||
| }); | ||
| return response.data; | ||
| }, |
There was a problem hiding this comment.
Fix incorrect authedFetch usage.
The authedFetch call is passing method, headers, and body in the second parameter (params), but these should be in the third parameter (config). The second parameter is for query string parameters, not request configuration.
Looking at the authedFetch signature from client/src/api/utils.ts:
authedFetch<T>(url: string, params?: Record<string, any>, config: AxiosRequestConfig = {})This will cause a runtime error because the function will try to serialize method, headers, and body as query parameters.
Apply this diff to fix the usage:
mutationFn: async ({ siteId, apiKey }: UpdateSearchConsoleApiKeyParams) => {
const response = await authedFetch<{
success: boolean;
data: { searchConsoleApiKey: string };
- }>(`/site/${siteId}/search-console-api-key`, {
+ }>(`/site/${siteId}/search-console-api-key`, undefined, {
method: "POST",
headers: {
"Content-Type": "application/json",
},
- body: JSON.stringify({ apiKey }),
+ data: { apiKey },
});
return response.data;
},Note: Also changed body to data as that's the correct Axios configuration property name.
🤖 Prompt for AI Agents
In client/src/api/analytics/searchConsole/useUpdateSearchConsoleApiKey.ts around
lines 14 to 26, the authedFetch call incorrectly passes request config (method,
headers, body) as the second parameter (query params) and uses body instead of
Axios's data; change the call to pass query params (or undefined) as the second
argument and move method, headers, and data into the third config argument,
replacing body with data and preserving Content-Type and method POST so the
request is sent correctly.
| useEffect(() => { | ||
| const message = searchParams.get("message"); | ||
| if (message) { | ||
| setErrorMessage(decodeURIComponent(message)); | ||
| } | ||
| }, [searchParams]); |
There was a problem hiding this comment.
Guard against malformed query strings
decodeURIComponent will throw a URIError for malformed or partially encoded input. Because message comes straight from user-controlled search params, a crafted value like %E0%A4%A will crash this page before the error UI renders. Wrap the decode in a try/catch (or use a safe helper) and fall back to the original string so the page can still load. (developer.mozilla.org)
- const message = searchParams.get("message");
- if (message) {
- setErrorMessage(decodeURIComponent(message));
- }
+ const message = searchParams.get("message");
+ if (message) {
+ try {
+ setErrorMessage(decodeURIComponent(message));
+ } catch {
+ setErrorMessage(message);
+ }
+ }🤖 Prompt for AI Agents
In client/src/app/[site]/search-console/oauth/error/page.tsx around lines 15 to
20, decoding the message query param with decodeURIComponent can throw a
URIError for malformed input; wrap the decode in a try/catch (or call a small
safeDecode helper) so that if decodeURIComponent throws you fall back to the
original message string (or an empty string) before calling setErrorMessage,
ensuring the page won't crash on crafted/malformed query strings.
| export default function OAuthSuccessPage() { | ||
| const router = useRouter(); | ||
| const params = useParams(); | ||
| const siteId = params.site; | ||
|
|
||
| useEffect(() => { | ||
| // Redirect back to search console page after 3 seconds | ||
| const timer = setTimeout(() => { | ||
| router.push(`/${siteId}/search-console`); | ||
| }, 3000); | ||
|
|
||
| return () => clearTimeout(timer); | ||
| }, [router, siteId]); |
There was a problem hiding this comment.
Add null check for siteId parameter.
The component uses siteId from params without verifying it exists, which could cause issues if the route is accessed incorrectly.
Apply this diff to add defensive null checking:
export default function OAuthSuccessPage() {
const router = useRouter();
const params = useParams();
const siteId = params.site;
useEffect(() => {
+ if (!siteId) {
+ router.push('/');
+ return;
+ }
+
// Redirect back to search console page after 3 seconds
const timer = setTimeout(() => {
router.push(`/${siteId}/search-console`);
}, 3000);
return () => clearTimeout(timer);
}, [router, siteId]);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export default function OAuthSuccessPage() { | |
| const router = useRouter(); | |
| const params = useParams(); | |
| const siteId = params.site; | |
| useEffect(() => { | |
| // Redirect back to search console page after 3 seconds | |
| const timer = setTimeout(() => { | |
| router.push(`/${siteId}/search-console`); | |
| }, 3000); | |
| return () => clearTimeout(timer); | |
| }, [router, siteId]); | |
| export default function OAuthSuccessPage() { | |
| const router = useRouter(); | |
| const params = useParams(); | |
| const siteId = params.site; | |
| useEffect(() => { | |
| if (!siteId) { | |
| router.push('/'); | |
| return; | |
| } | |
| // Redirect back to search console page after 3 seconds | |
| const timer = setTimeout(() => { | |
| router.push(`/${siteId}/search-console`); | |
| }, 3000); | |
| return () => clearTimeout(timer); | |
| }, [router, siteId]); |
🤖 Prompt for AI Agents
In client/src/app/[site]/search-console/oauth/success/page.tsx around lines 7 to
19, add a defensive null/undefined check for params.site (siteId) before using
it in the effect; if siteId is falsy, skip scheduling the redirect (or
immediately navigate to a safe fallback like '/' or the generic search-console
listing), and ensure the cleanup still clears the timer only when one was
created. Update the dependency usage accordingly so router.push is only called
with a valid siteId.
| import { FastifyRequest, FastifyReply } from "fastify"; | ||
| import { db } from "../../db/postgres/postgres.js"; | ||
| import { sites } from "../../db/postgres/schema.js"; | ||
| import { eq } from "drizzle-orm"; | ||
| import { z } from "zod"; | ||
| import { siteConfig } from "../../lib/siteConfig.js"; | ||
| import { getUserHasAdminAccessToSite } from "../../lib/auth-utils.js"; | ||
|
|
||
| // Schema for updating Search Console API key | ||
| const updateSearchConsoleApiKeySchema = z.object({ | ||
| apiKey: z.string().min(1, "API key is required"), | ||
| }); | ||
|
|
||
| export async function updateSearchConsoleApiKey( | ||
| request: FastifyRequest, | ||
| reply: FastifyReply | ||
| ) { | ||
| try { | ||
| if (!request.user) { | ||
| return reply.status(401).send({ success: false, error: "Unauthorized" }); | ||
| } | ||
|
|
||
| const { siteId } = request.params as { siteId: string }; | ||
| const parsedSiteId = parseInt(siteId, 10); | ||
|
|
||
| if (isNaN(parsedSiteId)) { | ||
| return reply.status(400).send({ success: false, error: "Invalid site ID" }); | ||
| } | ||
|
|
||
| const userHasAdminAccessToSite = await getUserHasAdminAccessToSite( | ||
| request, | ||
| String(parsedSiteId) | ||
| ); | ||
| if (!userHasAdminAccessToSite) { | ||
| return reply.status(403).send({ error: "Forbidden" }); | ||
| } | ||
|
|
||
| // Validate request body | ||
| const validationResult = updateSearchConsoleApiKeySchema.safeParse(request.body); | ||
| if (!validationResult.success) { | ||
| return reply.status(400).send({ | ||
| success: false, | ||
| error: "Invalid request body", | ||
| details: validationResult.error.flatten(), | ||
| }); | ||
| } | ||
|
|
||
| const { apiKey } = validationResult.data; | ||
|
|
||
| // Update the site | ||
| const updatedSite = await db | ||
| .update(sites) | ||
| .set({ searchConsoleApiKey: apiKey }) | ||
| .where(eq(sites.siteId, parsedSiteId)) | ||
| .returning({ | ||
| searchConsoleApiKey: sites.searchConsoleApiKey, | ||
| }); | ||
|
|
||
| // Update the site config cache | ||
| siteConfig.updateSiteSearchConsoleApiKey(parsedSiteId, apiKey); | ||
|
|
||
| return reply.send({ | ||
| success: true, | ||
| data: updatedSite[0], | ||
| }); | ||
| } catch (error) { | ||
| console.error("Error updating Search Console API key:", error); | ||
| return reply.status(500).send({ | ||
| success: false, | ||
| error: "Failed to update Search Console API key", | ||
| }); | ||
| } | ||
| } |
There was a problem hiding this comment.
Clarify searchConsoleApiKey usage given OAuth2 implementation.
This endpoint updates searchConsoleApiKey, but according to the schema comment in server/src/db/postgres/schema.ts (line 67), this field is "deprecated - use OAuth2". The PR introduces a full OAuth2 flow with tokens stored in searchConsoleAccessToken and searchConsoleRefreshToken fields.
Please clarify:
- Is the API key approach still supported alongside OAuth2?
- If so, update the schema comment to remove "deprecated"
- If not, consider removing this endpoint or clearly document it's for legacy/migration purposes only
Additionally, the implementation should specify which authentication method (API key vs OAuth2) takes precedence when both are present.
🤖 Prompt for AI Agents
In server/src/api/sites/updateSearchConsoleApiKey.ts around lines 1-73, the
endpoint updates the deprecated searchConsoleApiKey while the project now uses
OAuth2 tokens (searchConsoleAccessToken/searchConsoleRefreshToken); clarify
intended behavior and implement one of the following fixes: (A) If API key
support remains, remove "deprecated" from the DB schema comment, add explicit
docs on precedence (define whether API key or OAuth2 wins), and update this
endpoint to check for existing OAuth2 tokens and either log/warn or override per
the defined precedence; (B) If API key is no longer supported, delete or
deprecate this endpoint (return 410 or a clear message), remove write paths to
searchConsoleApiKey, and add migration notes to move existing API keys into
OAuth2 tokens if needed; also update README/schema comments and any tests to
reflect the chosen approach.
| searchConsoleApiKey: text("search_console_api_key"), // Google Search Console API key (deprecated - use OAuth2) | ||
| searchConsoleAccessToken: text("search_console_access_token"), // Google OAuth2 access token for Search Console | ||
| searchConsoleRefreshToken: text("search_console_refresh_token"), // Google OAuth2 refresh token for Search Console | ||
| searchConsoleTokenExpiry: timestamp("search_console_token_expiry"), // Token expiry timestamp |
There was a problem hiding this comment.
Encrypt stored OAuth tokens
Saving searchConsoleAccessToken and searchConsoleRefreshToken as plain text leaves bearer credentials unprotected if the database or backups are accessed. Industry guidance for Google OAuth explicitly calls for encrypting access and refresh tokens at rest (e.g., KMS-backed encrypted columns or a secrets store) so leaked data can’t be replayed. Please persist these values encrypted (or move them to a managed secret store) before shipping. (developers.google.com)
| generateAuthUrl(siteId: number): string { | ||
| const scopes = [ | ||
| 'https://www.googleapis.com/auth/webmasters', | ||
| 'https://www.googleapis.com/auth/webmasters.readonly' | ||
| ]; | ||
|
|
||
| return this.oauth2Client.generateAuthUrl({ | ||
| access_type: 'offline', | ||
| scope: scopes, | ||
| prompt: 'consent', | ||
| state: siteId.toString(), // Pass siteId in state parameter | ||
| }); | ||
| } |
There was a problem hiding this comment.
Sign and randomize OAuth state to prevent CSRF and siteId spoofing.
Using plain siteId as state is predictable and lets an attacker bind their Google account to another site. Generate a signed, time-bound state and verify it on callback.
+import crypto from 'crypto';
+const STATE_SECRET = process.env.OAUTH_STATE_SECRET || process.env.SESSION_SECRET || 'change-me';
+function buildState(siteId: number) {
+ const nonce = crypto.randomBytes(16).toString('hex');
+ const ts = Date.now();
+ const payload = `${siteId}.${nonce}.${ts}`;
+ const sig = crypto.createHmac('sha256', STATE_SECRET).update(payload).digest('hex');
+ return Buffer.from(JSON.stringify({ siteId, nonce, ts, sig })).toString('base64url');
+}
+function parseAndVerifyState(state: string): number | null {
+ try {
+ const { siteId, nonce, ts, sig } = JSON.parse(Buffer.from(state, 'base64url').toString('utf8'));
+ const payload = `${siteId}.${nonce}.${ts}`;
+ const expected = crypto.createHmac('sha256', STATE_SECRET).update(payload).digest('hex');
+ if (sig !== expected) return null;
+ if (Date.now() - Number(ts) > 10 * 60 * 1000) return null; // 10 min TTL
+ return Number(siteId);
+ } catch { return null; }
+}
...
- return this.oauth2Client.generateAuthUrl({
+ return this.oauth2Client.generateAuthUrl({
access_type: 'offline',
scope: scopes,
prompt: 'consent',
- state: siteId.toString(), // Pass siteId in state parameter
+ state: buildState(siteId),
});And in handleCallback:
- const siteId = parseInt(state, 10);
- if (isNaN(siteId)) {
+ const siteId = parseAndVerifyState(state);
+ if (!siteId) {
return { success: false, error: 'Invalid site ID' };
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| generateAuthUrl(siteId: number): string { | |
| const scopes = [ | |
| 'https://www.googleapis.com/auth/webmasters', | |
| 'https://www.googleapis.com/auth/webmasters.readonly' | |
| ]; | |
| return this.oauth2Client.generateAuthUrl({ | |
| access_type: 'offline', | |
| scope: scopes, | |
| prompt: 'consent', | |
| state: siteId.toString(), // Pass siteId in state parameter | |
| }); | |
| } | |
| generateAuthUrl(siteId: number): string { | |
| const scopes = [ | |
| 'https://www.googleapis.com/auth/webmasters', | |
| 'https://www.googleapis.com/auth/webmasters.readonly' | |
| ]; | |
| return this.oauth2Client.generateAuthUrl({ | |
| access_type: 'offline', | |
| scope: scopes, | |
| prompt: 'consent', | |
| state: buildState(siteId), | |
| }); | |
| } |
| # Google Search Console Implementation Status | ||
|
|
||
| ## Current Status | ||
|
|
||
| ✅ **Database Schema**: Added OAuth2 token fields to the sites table | ||
| ✅ **OAuth2 Authentication**: Full OAuth2 flow implemented for Google Search Console | ||
| ✅ **API Endpoints**: OAuth2 endpoints for authentication and data fetching | ||
| ✅ **Frontend Integration**: OAuth2 flow in the UI with success/error pages | ||
| ✅ **Service Layer**: SearchConsoleService with real Google Search Console API integration | ||
| ✅ **Real Data**: Fetches actual data from Google Search Console API | ||
|
|
||
| ## What's Working | ||
|
|
||
| 1. **OAuth2 Authentication**: Users can connect their Google account securely | ||
| 2. **Real Data Fetching**: Fetches actual search performance data from Google Search Console | ||
| 3. **Token Management**: Automatic token refresh and expiry handling | ||
| 4. **Connection Status**: Real-time connection status checking | ||
| 5. **Error Handling**: Proper error messages and OAuth error pages | ||
|
|
||
| ## Setup Instructions | ||
|
|
||
| ### 1. Google Cloud Console Setup | ||
| 1. Go to [Google Cloud Console](https://console.cloud.google.com/) | ||
| 2. Create a new project or select an existing one | ||
| 3. Enable the Search Console API: | ||
| - Go to "APIs & Services" > "Library" | ||
| - Search for "Search Console API" | ||
| - Click "Enable" | ||
|
|
||
| ### 2. OAuth2 Credentials | ||
| 1. Go to "APIs & Services" > "Credentials" | ||
| 2. Click "Create Credentials" > "OAuth 2.0 Client IDs" | ||
| 3. Choose "Web application" | ||
| 4. Add authorized redirect URIs: | ||
| - `http://localhost:3001/api/search-console/oauth/callback` (for development) | ||
| - `https://yourdomain.com/api/search-console/oauth/callback` (for production) | ||
| 5. Copy the Client ID and Client Secret | ||
|
|
||
| ### 3. Environment Variables | ||
| Add these to your `.env` file: | ||
| ``` | ||
| GOOGLE_CLIENT_ID=your_client_id_here | ||
| GOOGLE_CLIENT_SECRET=your_client_secret_here | ||
| BASE_URL=http://localhost:3001 # or your production URL | ||
| ``` | ||
|
|
||
| ## API Endpoints | ||
|
|
||
| ### OAuth2 Flow | ||
| - `GET /api/site/:siteId/search-console/oauth/url` - Generate OAuth2 authorization URL | ||
| - `GET /api/search-console/oauth/callback` - Handle OAuth2 callback | ||
| - `GET /api/site/:siteId/search-console/connection-status` - Check connection status | ||
| - `POST /api/site/:siteId/search-console/disconnect` - Disconnect Search Console | ||
|
|
||
| ### Data Fetching | ||
| - `GET /api/search-console/:site` - Get search console data (uses OAuth2 tokens) | ||
|
|
||
| ## How It Works | ||
|
|
||
| 1. **User clicks "Connect"** → Generates OAuth2 authorization URL | ||
| 2. **User authorizes** → Google redirects to callback with authorization code | ||
| 3. **Server exchanges code** → Gets access token and refresh token | ||
| 4. **Tokens stored securely** → In database with expiry tracking | ||
| 5. **Data fetching** → Uses access token to call Google Search Console API | ||
| 6. **Token refresh** → Automatically refreshes expired tokens | ||
|
|
||
| ## Features | ||
|
|
||
| - ✅ **Secure OAuth2 Flow**: No API keys needed, uses Google's secure authentication | ||
| - ✅ **Real Data**: Fetches actual search performance data from Google | ||
| - ✅ **Token Management**: Automatic refresh of expired tokens | ||
| - ✅ **Error Handling**: Proper error pages and user feedback | ||
| - ✅ **Connection Status**: Real-time checking of connection status | ||
| - ✅ **Disconnect**: Users can disconnect their account anytime | ||
|
|
||
| ## Testing | ||
|
|
||
| 1. Set up Google Cloud Console credentials | ||
| 2. Add environment variables | ||
| 3. Click "Connect" on the Search Console page | ||
| 4. Complete OAuth2 flow | ||
| 5. View real search performance data | ||
|
|
||
| The implementation now fetches real data from Google Search Console API using OAuth2 authentication! |
There was a problem hiding this comment.
Documentation inconsistency between README files.
This implementation document indicates OAuth2 is fully complete (line 6: "✅ OAuth2 Authentication"), while client/src/app/[site]/search-console/README.md lines 22-27 list OAuth2 as "🔄 In Progress / TODO".
Please ensure both documentation files are synchronized. Based on the code changes in this PR (OAuth services, endpoints, and UI flows), the OAuth2 implementation appears complete, so the client README should be updated to match this implementation status document.
🧰 Tools
🪛 LanguageTool
[style] ~8-~8: ‘with success’ might be wordy. Consider a shorter alternative.
Context: ...nd Integration**: OAuth2 flow in the UI with success/error pages ✅ Service Layer: Search...
(EN_WORDINESS_PREMIUM_WITH_SUCCESS)
🪛 markdownlint-cli2 (0.18.1)
41-41: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
| const verifiedSite = sitesResponse.data.siteEntry?.find( | ||
| (site: any) => { | ||
| const siteUrl = site.siteUrl; | ||
| // Check for exact matches and sc-domain: prefixed matches | ||
| return siteUrl === `https://${domain}` || | ||
| siteUrl === `http://${domain}` || | ||
| siteUrl === domain || | ||
| siteUrl === `sc-domain:${domain}` || | ||
| siteUrl === `sc-prefix:${domain}`; | ||
| } | ||
| ); | ||
|
|
||
| if (!verifiedSite) { | ||
| console.log(`Site ${domain} not found in Search Console. Available sites:`, sitesResponse.data.siteEntry); | ||
| throw new Error(`Site ${domain} is not verified in Google Search Console. Please add and verify your site first.`); | ||
| } | ||
|
|
||
| console.log(`Found verified site: ${verifiedSite.siteUrl}`); | ||
| } catch (sitesError) { | ||
| console.error('Error checking Search Console sites:', sitesError); | ||
| throw new Error('Unable to verify site in Google Search Console'); | ||
| } | ||
|
|
||
| // Try different site URL formats - prioritize sc-domain: format since that's what Search Console uses | ||
| const siteUrlFormats = [ | ||
| `sc-domain:${domain}`, | ||
| `sc-prefix:${domain}`, | ||
| `https://${domain}`, | ||
| `http://${domain}`, | ||
| domain | ||
| ]; | ||
|
|
||
| let successfulFormat: string | undefined; | ||
|
|
||
| // Find a working site URL format | ||
| for (const siteUrl of siteUrlFormats) { | ||
| try { | ||
| console.log(`Testing site URL format: ${siteUrl}`); | ||
| await searchconsole.searchanalytics.query({ | ||
| siteUrl, | ||
| requestBody: { | ||
| startDate, | ||
| endDate, | ||
| dimensions: ['query'], | ||
| rowLimit: 1, | ||
| }, | ||
| }); | ||
| successfulFormat = siteUrl; | ||
| console.log(`Success with format: ${siteUrl}`); | ||
| break; | ||
| } catch (formatError: any) { | ||
| console.log(`Failed with format ${siteUrl}:`, formatError.message); | ||
| if (formatError.code === 403) { | ||
| throw new Error(`Access denied for ${siteUrl}. Make sure your site is verified in Google Search Console.`); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if (!successfulFormat) { | ||
| throw new Error(`Unable to fetch data for domain ${domain}. Please verify the site is added to Google Search Console.`); | ||
| } | ||
|
|
There was a problem hiding this comment.
Ensure siteUrl matches Search Console format
Search Console expects URL-prefix properties to include the trailing / (and there is no sc-prefix: property type). Comparing against https://${domain} and querying with the same string will always fail the verification/API call, so a fully verified site throws “not verified”/403 errors. Google’s docs require https://example.com/ or sc-domain:example.com depending on the property type.(developers.google.com) Please update the equality checks and the candidate list to include the slash and drop the unsupported sc-prefix: entry.
- return siteUrl === `https://${domain}` ||
- siteUrl === `http://${domain}` ||
- siteUrl === domain ||
- siteUrl === `sc-domain:${domain}` ||
- siteUrl === `sc-prefix:${domain}`;
+ return siteUrl === `https://${domain}/` ||
+ siteUrl === `http://${domain}/` ||
+ siteUrl === `sc-domain:${domain}`;
...
- const siteUrlFormats = [
- `sc-domain:${domain}`,
- `sc-prefix:${domain}`,
- `https://${domain}`,
- `http://${domain}`,
- domain
- ];
+ const siteUrlFormats = [
+ `sc-domain:${domain}`,
+ `https://${domain}/`,
+ `http://${domain}/`
+ ];🤖 Prompt for AI Agents
In server/src/services/searchConsoleService.ts around lines 463-524, the code
compares and queries Search Console using incorrect siteUrl formats (missing
trailing '/' for URL-prefix and using a non-existent 'sc-prefix:' type), causing
verification and 403 errors; update the equality checks to match Search Console
formats by removing any 'sc-prefix:' comparisons and by comparing against
`sc-domain:${domain}` and the URL-prefix forms `https://${domain}/` and
`http://${domain}/` (include the trailing slash), and change the siteUrlFormats
array to try, in order, `sc-domain:${domain}`, `https://${domain}/`,
`http://${domain}/`, and optionally `https://${domain}`/`http://${domain}` only
if you intentionally want fallback, ensuring all query calls use the corrected
strings so verification and query requests succeed.
Summary by CodeRabbit