-
-
Notifications
You must be signed in to change notification settings - Fork 831
Event Platform Client for analytics powered by Modern Event Platform #3603
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
tonisevener
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice work! I'm about halfway through but at a good stopping point today to give some feedback. Can I also request that we use camel case for all of this instead of snake case? Also no all caps please 🙏 😁 Thanks so much!
WMF Framework/Event Platform Client/EventPlatformClientProtocols.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
| /** | ||
| * For persistent storage | ||
| */ | ||
| private var storage_manager: StorageManager? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think if EPC does little without a storage_manager or network_manager, and set_storage_manager and set_network_manager will only be ever called once in EPC's lifecycle, maybe you should initialize them within the init, so you can guarantee they are always there and not optional (thus removing all the trailing ! anytime you reference one). I realize this complicates the shared singleton setup, but you can comment that out for now until we implement what network_manager and storage_manager look like.
public init(network_manager: NetworkManager, storage_manager: StorageManager) {
self.network_manager = network_manager
self.storage_manager = storage_manager
EVENTGATE_URI = "https://intake-analytics.wikimedia.org/v1/events"
CONFIG_URI = "https://meta.wikimedia.org/w/api.php?action=streamconfigs&format=json"
ISO8601_FORMATTER = ISO8601DateFormatter()
ENABLED = true
recall_buffer()
configure()
}
then these properties can be non-optional constants:
private let network_manager: NetworkManager
private let storage_manager: StorageManager
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
tonisevener
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Finished 2nd half! I'm looking forward to getting this in - thanks for the work you've put in already. Will NetworkManager and StorageManager implementations be done in a followup PR? We can take a look at connecting that too if you'd rather since it'll be tinkering with the existing codebase. FYI a lot of my suggestions this round already have camel case & some slight renames.
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
+ Renamed everything to camelCase per convention + Thread-safe input buffering and stream determination caching + Changed events from tuples to structs + Cleaner code thanks to Toni + Fewer theoretical crashes and uncaught exceptions - Disabled shared singleton aspect for now, until StorageManager and NetworkManager are implemented
+ Renamed everything to camelCase per convention + Thread-safe input buffering and stream determination caching + Changed events from tuples to structs + Cleaner code thanks to Toni + Fewer theoretical crashes and uncaught exceptions + Added debug logging (for easier development and QA) - Disabled shared singleton aspect for now, until StorageManager and NetworkManager are implemented
|
TIL that PRs on GitHub aren't keen on |
tonisevener
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking great! Just a couple of things. Also FYI we re-named our master branch to main, so you'll need to update it with that for it to be on the latest.
WMF Framework/Event Platform Client/EventPlatformClientLibrary.swift
Outdated
Show resolved
Hide resolved
| DDLogDebug("[EPC] Invalid state, must have streamConfigurations to check for inSample") | ||
| return false | ||
| } | ||
| if let cachedValue = samplingCache[stream] { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually this piece will need some thread-safety too. For this simple serial queue approach you can try queue.sync, that pauses the current thread execution while queue goes off and pulls the value and returns it back to the original thread that needs it. A pattern I've noticed in this codebase is to create a bunch of helper methods for getting and setting to these data structures. That buries all the queue.async and queue.sync calls and cleans up the code. It was a bit long to give an example here so I tried it in this commit.
Also as I'm looking at this I think we might need thread safety when accessing or setting the streamConfigurations and streamCopy properties as well 😕 so I'll leave that for you. The reason being they are referenced in log(...), which I envision will get called from multiple threads as we pepper them throughout our codebase, so we want to make sure these properties stay intact.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Hopefully I did it right in the upcoming commit
Merging Toni's suggestions, to include them in PR #3603
|
@tonisevener Would it be okay if I closed this PR and submitted a new one? |
tonisevener
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Just a few little things, you're welcome to open up a new PR if you need to.
| // Make them available to any newly logged events before flushing buffer | ||
| queue.sync { | ||
| self.streamConfigurations = config | ||
| self.streamCopy = copies |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You'll be losing any previous values that were in streamCopy with this vs. how the old code worked. Seems like that's what you're wanting anyways, but just a heads up in case you expect setStreamConfig to be called more than once.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a good point! As it is we're good since the stream configs are downloaded & processed only once, but once we figure out how to approach multiple stream configs (if at all) in the BUOD <> Apps sync we'll know if this needs to be modified to do, like, a merge operation.
| copies.appendIfNew(key: String(s[0]), value: stream) | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You probably want let jsonString = try copies.toPrettyPrintJSONString() on line 348 below now.
| } | ||
| } | ||
|
|
||
| func getStreamConfig() -> [String: [String: Any]]? { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you also reference this method instead of using streamConfigurations directly in the configure() method and inSample(stream:) methods?
|
Thanks for the green light! Going to re-submit as a fresh PR to deal with branch naming and merge conflicts. |
This patch adds the iOS version of the Event Platform Client (EPC-iOS) for integration with the modern Event Platform. EPC is an effort to unify algorithms, identifiers, and behaviors across MediaWiki, iOS, and Android.
This initial release supports:
For additional background information on instrumentation within the Modern Event Platform, please refer to this page.
Phabricator: T228180