Skip to content

Conversation

@bearloga
Copy link
Member

This patch adds the iOS version of the Event Platform Client (EPC-iOS) for integration with the modern Event Platform. EPC is an effort to unify algorithms, identifiers, and behaviors across MediaWiki, iOS, and Android.

This initial release supports:

  • remote sampling configurations, which puts sampling rates into the centralized stream configuration
  • stream cc-ing, which automatically logs events to multiple streams without requiring multiple calls on the instrumentation side

For additional background information on instrumentation within the Modern Event Platform, please refer to this page.

Phabricator: T228180

@bearloga bearloga requested a review from joewalsh June 10, 2020 20:52
Copy link
Collaborator

@tonisevener tonisevener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! I'm about halfway through but at a good stopping point today to give some feedback. Can I also request that we use camel case for all of this instead of snake case? Also no all caps please 🙏 😁 Thanks so much!

/**
* For persistent storage
*/
private var storage_manager: StorageManager?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if EPC does little without a storage_manager or network_manager, and set_storage_manager and set_network_manager will only be ever called once in EPC's lifecycle, maybe you should initialize them within the init, so you can guarantee they are always there and not optional (thus removing all the trailing ! anytime you reference one). I realize this complicates the shared singleton setup, but you can comment that out for now until we implement what network_manager and storage_manager look like.

public init(network_manager: NetworkManager, storage_manager: StorageManager) {
    self.network_manager = network_manager
    self.storage_manager = storage_manager

    EVENTGATE_URI = "https://intake-analytics.wikimedia.org/v1/events"
    CONFIG_URI = "https://meta.wikimedia.org/w/api.php?action=streamconfigs&format=json"
    ISO8601_FORMATTER = ISO8601DateFormatter()
    ENABLED = true
        
    recall_buffer()
    configure()
}

then these properties can be non-optional constants:

private let network_manager: NetworkManager
private let storage_manager: StorageManager

Copy link
Collaborator

@tonisevener tonisevener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finished 2nd half! I'm looking forward to getting this in - thanks for the work you've put in already. Will NetworkManager and StorageManager implementations be done in a followup PR? We can take a look at connecting that too if you'd rather since it'll be tinkering with the existing codebase. FYI a lot of my suggestions this round already have camel case & some slight renames.

@joewalsh joewalsh changed the base branch from master to main July 13, 2020 17:12
+ Renamed everything to camelCase per convention
+ Thread-safe input buffering and stream determination caching
+ Changed events from tuples to structs
+ Cleaner code thanks to Toni
+ Fewer theoretical crashes and uncaught exceptions
- Disabled shared singleton aspect for now, until StorageManager and NetworkManager are implemented
@bearloga bearloga requested a review from tonisevener July 21, 2020 18:33
bearloga added 2 commits July 21, 2020 16:48
+ Renamed everything to camelCase per convention
+ Thread-safe input buffering and stream determination caching
+ Changed events from tuples to structs
+ Cleaner code thanks to Toni
+ Fewer theoretical crashes and uncaught exceptions
+ Added debug logging (for easier development and QA)
- Disabled shared singleton aspect for now, until StorageManager and NetworkManager are implemented
@bearloga
Copy link
Member Author

TIL that PRs on GitHub aren't keen on git commit --amend

Copy link
Collaborator

@tonisevener tonisevener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great! Just a couple of things. Also FYI we re-named our master branch to main, so you'll need to update it with that for it to be on the latest.

DDLogDebug("[EPC] Invalid state, must have streamConfigurations to check for inSample")
return false
}
if let cachedValue = samplingCache[stream] {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually this piece will need some thread-safety too. For this simple serial queue approach you can try queue.sync, that pauses the current thread execution while queue goes off and pulls the value and returns it back to the original thread that needs it. A pattern I've noticed in this codebase is to create a bunch of helper methods for getting and setting to these data structures. That buries all the queue.async and queue.sync calls and cleans up the code. It was a bit long to give an example here so I tried it in this commit.

Also as I'm looking at this I think we might need thread safety when accessing or setting the streamConfigurations and streamCopy properties as well 😕 so I'll leave that for you. The reason being they are referenced in log(...), which I envision will get called from multiple threads as we pepper them throughout our codebase, so we want to make sure these properties stay intact.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Hopefully I did it right in the upcoming commit

bearloga added a commit that referenced this pull request Jul 23, 2020
Merging Toni's suggestions, to include them in PR #3603
@bearloga
Copy link
Member Author

@tonisevener Would it be okay if I closed this PR and submitted a new one?

Copy link
Collaborator

@tonisevener tonisevener left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Just a few little things, you're welcome to open up a new PR if you need to.

// Make them available to any newly logged events before flushing buffer
queue.sync {
self.streamConfigurations = config
self.streamCopy = copies
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll be losing any previous values that were in streamCopy with this vs. how the old code worked. Seems like that's what you're wanting anyways, but just a heads up in case you expect setStreamConfig to be called more than once.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point! As it is we're good since the stream configs are downloaded & processed only once, but once we figure out how to approach multiple stream configs (if at all) in the BUOD <> Apps sync we'll know if this needs to be modified to do, like, a merge operation.

copies.appendIfNew(key: String(s[0]), value: stream)
}
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably want let jsonString = try copies.toPrettyPrintJSONString() on line 348 below now.

}
}

func getStreamConfig() -> [String: [String: Any]]? {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also reference this method instead of using streamConfigurations directly in the configure() method and inSample(stream:) methods?

@bearloga
Copy link
Member Author

Thanks for the green light! Going to re-submit as a fresh PR to deal with branch naming and merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants