-
Notifications
You must be signed in to change notification settings - Fork 2k
Hosting Dashboard: Add site logs initial dataviews with data #105118
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
coder-karen
merged 16 commits into
trunk
from
add/hosting-dashboard-logs-initial-dataviews
Aug 12, 2025
Merged
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
e81fcfd
Initial site logs dataviews import with data, for hosting dashboard
coder-karen df820d2
Fixing up type declaration and usage
coder-karen f0d878e
Simplify and inline use-data logic
coder-karen c61e39e
Remove siteId check from queryOptions enabled params
coder-karen ebf0ae1
Move toFilterParams to new file within sites/logs/dataviews
coder-karen a283414
Move keepPreviousData placeholderData addition to main component for …
coder-karen 6713651
Use useSuspenseQuery
coder-karen f26bf5d
Fixup translation functions
coder-karen b996f87
Adding one more missed type import - SiteLogsParams
coder-karen 7f1325a
Spread params within fetchSiteLogs instead of adding each individually
coder-karen f5eb778
Simplify data returned from query, add functionality to data layer in…
coder-karen ac3c10b
Remove notice about limited functionality
coder-karen 0129270
Moving types, consta dna components -- fixing ids as needed in the pr…
coder-karen a514a05
Remove siteId as a field of SiteLogsParams, and prevent type exports
coder-karen 7e143ca
Pass placeholderData through as an option - see PR review comment htt…
coder-karen 6232dc0
Merge remote-tracking branch 'origin/trunk' into add/hosting-dashboar…
coder-karen File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import { queryOptions, keepPreviousData } from '@tanstack/react-query'; | ||
| import { fetchSiteLogs, SiteLogsParams, SiteLogsQueryOptions } from '../../data/site-logs'; | ||
|
|
||
| export const siteLogsQuery = ( | ||
| siteId: number, | ||
| params: SiteLogsParams, | ||
| options?: SiteLogsQueryOptions | ||
| ) => | ||
| queryOptions( { | ||
| queryKey: [ 'site', siteId, 'logs', params ], | ||
| queryFn: () => fetchSiteLogs( siteId, params ), | ||
| placeholderData: options?.keepPreviousData ? keepPreviousData : undefined, | ||
| enabled: params.start <= params.end, | ||
| staleTime: Infinity, // The logs within a specified time range never change. | ||
| } ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,137 @@ | ||
| import wpcom from 'calypso/lib/wp'; | ||
|
|
||
| interface SiteLogsAPIResponse { | ||
| message: string; | ||
| data: { | ||
| total_results: number | { value: number; relation: string }; | ||
| logs: ( PHPLogFromEndpoint | ServerLogFromEndpoint )[]; | ||
| }; | ||
| } | ||
|
|
||
| interface PHPLogFromEndpoint { | ||
| timestamp: string; | ||
| severity: 'User' | 'Warning' | 'Deprecated' | 'Fatal error'; | ||
| message: string; | ||
| kind: string; | ||
| name: string; | ||
| file: string; | ||
| line: number; | ||
| atomic_site_id: number; | ||
| } | ||
|
|
||
| export interface PHPLog extends Omit< PHPLogFromEndpoint, 'atomic_site_id' > { | ||
| id: string; | ||
| } | ||
|
|
||
| export type SiteLogsQueryOptions = { keepPreviousData?: boolean }; | ||
|
|
||
| export const LogType = { | ||
| PHP: 'php', | ||
| SERVER: 'server', | ||
| } as const; | ||
|
|
||
| export type LogType = ( typeof LogType )[ keyof typeof LogType ]; | ||
|
|
||
| export interface FilterType { | ||
| [ key: string ]: Array< string >; | ||
| } | ||
|
|
||
| export interface ServerLogFromEndpoint { | ||
| date: string; | ||
| request_type: 'GET' | 'HEAD' | 'POST' | 'PUT' | 'DELETE'; | ||
| status: '200' | '301' | '302' | '400' | '401' | '403' | '404' | '429' | '500'; | ||
| request_url: string; | ||
| body_bytes_sent: number; | ||
| cached: string; | ||
| http_host: string; | ||
| http_referer: string; | ||
| http2: string; | ||
| http_user_agent: string; | ||
| http_version: string; | ||
| http_x_forwarded_for: string; | ||
| renderer: string; | ||
| request_completion: string; | ||
| request_time: string; | ||
| scheme: string; | ||
| timestamp: number; | ||
| type: string; | ||
| user_ip: string; | ||
| } | ||
|
|
||
| export interface ServerLog extends ServerLogFromEndpoint { | ||
| id: string; | ||
| } | ||
|
|
||
| export interface SiteLogsParams { | ||
| logType: LogType; | ||
| start: number; | ||
| end: number; | ||
| filter: FilterType; | ||
| sortOrder?: 'asc' | 'desc'; | ||
| pageSize?: number; | ||
| pageIndex?: number; | ||
| } | ||
|
|
||
| export interface SiteLogsData { | ||
| total_results: number; | ||
| logs: ( PHPLog | ServerLog )[]; | ||
| } | ||
|
|
||
| export async function fetchSiteLogs( | ||
| siteId: number, | ||
| { logType, start, end, filter, sortOrder, pageSize, pageIndex }: SiteLogsParams | ||
| ): Promise< SiteLogsData > { | ||
| const logTypeFragment = logType === LogType.PHP ? 'error-logs' : 'logs'; | ||
| const path = `/sites/${ siteId }/hosting/${ logTypeFragment }`; | ||
|
|
||
| const queryParams = { | ||
| start, | ||
| end, | ||
| filter, | ||
| sort_order: sortOrder, | ||
| page_size: pageSize, | ||
| page_index: pageIndex, | ||
| }; | ||
|
|
||
| // Remove undefined values from queryParams | ||
| Object.keys( queryParams ).forEach( | ||
| ( key ) => | ||
| ( queryParams as Record< string, unknown > )[ key ] === undefined && | ||
| delete ( queryParams as Record< string, unknown > )[ key ] | ||
| ); | ||
|
|
||
| const response = await wpcom.req.get( | ||
| { | ||
| path, | ||
| apiNamespace: 'wpcom/v2', | ||
| }, | ||
| { ...queryParams } | ||
| ); | ||
|
|
||
| const { data } = response as SiteLogsAPIResponse; | ||
|
|
||
| const totalResults = | ||
| typeof data.total_results === 'number' ? data.total_results : data.total_results?.value ?? 0; | ||
|
|
||
| const logs = Array.isArray( data.logs ) ? data.logs : []; | ||
|
|
||
| if ( logType === LogType.PHP ) { | ||
| const normalized = ( logs as PHPLogFromEndpoint[] ).map( | ||
| ( { atomic_site_id, ...logWithoutAtomicId }, index ) => ( { | ||
| ...logWithoutAtomicId, | ||
| id: `${ logWithoutAtomicId.timestamp }|${ logWithoutAtomicId.file }|${ String( | ||
| logWithoutAtomicId.line | ||
| ) }|${ String( pageIndex ?? 0 ) }|${ String( index ) }`, | ||
| } ) | ||
| ); | ||
| return { total_results: totalResults, logs: normalized }; | ||
| } | ||
|
|
||
| const normalized = ( logs as ServerLogFromEndpoint[] ).map( ( log, index ) => ( { | ||
| ...log, | ||
| id: `${ String( log.timestamp ) }|${ log.request_type }|${ log.status }|${ log.request_url }|${ | ||
| log.user_ip | ||
| }|${ String( pageIndex ?? 0 ) }|${ String( index ) }`, | ||
| } ) ); | ||
| return { total_results: totalResults, logs: normalized }; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| import { LogType, FilterType } from '../../../data/site-logs'; | ||
| import type { View } from '@wordpress/dataviews'; | ||
|
|
||
| const getFilterParamsFromView = ( view: View, fieldNames: string[] ): FilterType => { | ||
| return ( view.filters || [] ) | ||
| .filter( ( filter ) => fieldNames.includes( filter.field ) ) | ||
| .reduce( ( acc: FilterType, filter ) => { | ||
| if ( filter.value ) { | ||
| acc[ filter.field ] = filter.value; | ||
| } | ||
| return acc; | ||
| }, {} as FilterType ); | ||
| }; | ||
|
|
||
| export function toFilterParams( { view, logType }: { view: View; logType: LogType } ): FilterType { | ||
| if ( logType === LogType.PHP ) { | ||
| return getFilterParamsFromView( view, [ 'severity' ] ); | ||
| } | ||
|
|
||
| return getFilterParamsFromView( view, [ 'cached', 'request_type', 'status', 'renderer' ] ); | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I wonder why this
idis for... can we just utilize thetimestampfield for that?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When mapping in
siteLogsQuerythe returned data now matches theServerLog/PHPLogtypes due to the added id (same as what is was before).It seems like the timestamp itself wouldn't necessarily be unique 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've had to update this now anyway with further changes -- I started by using the timestamp and another identifier and it still wasn't unique enough (though likely also thanks to the current testing environment where my dummy field data has a fixed timestamp), so there are several identifiers which help ensure the data has a unique id.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see... we ideally should add this in the backend side, but we can do it later. 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point I added a task for it at DOTCOM-14208