Skip to content

Initial front-end metrics implementation - #51171

Merged
sanchitmalhotra126 merged 3 commits into
stagingfrom
sanchit/metrics/initial-front-end
Apr 14, 2023
Merged

Initial front-end metrics implementation#51171
sanchitmalhotra126 merged 3 commits into
stagingfrom
sanchit/metrics/initial-front-end

Conversation

@sanchitmalhotra126

@sanchitmalhotra126 sanchitmalhotra126 commented Apr 6, 2023

Copy link
Copy Markdown
Contributor

Initial implementation for a generic front-end metrics reporter. For now, this isn't used anywhere and isn't hooked up to any backend service, but I wanted to get some initial feedback before moving forward. Quick walkthrough of the new classes:

  • MetricsApi.ts

    • Interface describing an API for communicating with a backend metrics service. This interface allows us to have an abstraction layer between the actual reporter, and the backend service its communicating with. In practice, we'll create an implementation of this interface that communicates with dashboard, once the backend piece is set up.
  • MetricsReporter.ts

    • Metrics reporting class that can be used anywhere within front-end code. For now, it isn't exported anywhere, but I'm thinking this will be best used a singleton, and exported from this file with whatever implementation of the API we are currently using (e.g. export default MetricsReporter(new SomethingThatImplementsMetricsApi()))
    • Currently this only has functions for logging. Next step will be add functions for reporting events and counters.
    • Includes a mechanism for turning off reporting for some period of time (currently set to 30 min) if we get back a 401 unauthorized response from the server. The reason for this is, as we initially implement this as an experiment, we may turn off reporting via DCDO or Gatekeeper, and so this prevents us from spamming the backend if turned off. For an additional gatekeeping, we could also use a front-end DCDO flag, but I wasn't sure if that was necessary. The last check time (in epoch ms) is stored in local storage.

example structure of logs appearing in Cloudwatch (currently reporting to a test log group).

Screenshot 2023-04-06 at 1 47 57 PM

Background/Context

Some context on why this change is happening. We currently have various reporting solutions on the back-end and the front-end, but none currently meet our needs for robust developer-facing debugging. A quick summary of the services we use on the front-end specifically:

  • New Relic (logToCloud.js) - for overall site monitoring and error tracking. Currently not very usable due to data ingest issues
  • AWS Firehose (firehose.js) - used for a mix developer logging and product facing analytics. Cumbersome to query and visualize in Redshift; and product facing analytics reporting is expected to be replaced by Amplitude/GA
  • Google Analytics - used for some product-facing analytics
  • Amplitude (AnalyticsReporter.js) - used for some product-facing analytics

Given the issues with New Relic and Firehose, we have been exploring alternatives. The work started by this PR will add a new reporting path that reports client metrics to AWS Cloudwatch, where logs and metrics can be more easily viewed, retained, and visualized. As of the writing of this PR, we are implementing this metrics path as a time-boxed experiment to gather data on its usefulness. As such, we are including mechanisms such as the ability to turn off reporting via experiment flags.

Testing story

Tested locally. Will add unit tests before shipping or as a follow up, once I get some initial feedback.

@sanchitmalhotra126
sanchitmalhotra126 requested a review from a team April 6, 2023 17:50
@breville

breville commented Apr 6, 2023

Copy link
Copy Markdown
Member

Site note, but one thing I'd love to be captured in an early PR for this work (mainly for future readers who don't have access to internal documents/discussions), is the strategic impetus for this new system; how it relates to our other metrics systems such as Google Analytics, Amplitude, Firehose; how we migrate to this new system; and what backends it might cover.

@sanchitmalhotra126

Copy link
Copy Markdown
Contributor Author

Site note, but one thing I'd love to be captured in an early PR for this work (mainly for future readers who don't have access to internal documents/discussions), is the strategic impetus for this new system; how it relates to our other metrics systems such as Google Analytics, Amplitude, Firehose; how we migrate to this new system; and what backends it might cover.

Good call - I'll add more context in the description.

@molly-moen molly-moen left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all seems reasonable to me, just a couple comments!

Comment thread apps/src/lib/metrics/MetricsReporter.ts Outdated
* Interface for interacting with a metrics service API.
*/
export default interface MetricsApi {
sendLogs: (logs: object[]) => Promise<Response>;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we define the log type more than just an object?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking this could just be a generic JSON object payload; but I guess if we know about some required parameters that the metrics reporter adds (such as device info), this could be defined in a little more detail?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted to just keep this as an object for now for flexibility and to match the backend method signature (which just takes a list of JSON objects to log).

Comment thread apps/src/lib/metrics/MetricsReporter.ts Outdated
}

private fallbackLog(payload: object) {
console.log(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have an idea of how we would handle the case of needing a fallback long term, or is this just for the testing phase? Hopefully we would have alerting if whatever service we were using is down, and we could send logs as a best effort.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this would just be for the experiment/testing phase. This should also only happen if we get an explicit 401 unauthorized from the server. I think long term, yeah we'd just try to send logs as a best effort.

@sanchitmalhotra126
sanchitmalhotra126 merged commit 8aadd9e into staging Apr 14, 2023
@sanchitmalhotra126
sanchitmalhotra126 deleted the sanchit/metrics/initial-front-end branch April 14, 2023 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants