2

I have a use case where I am receiving events from the client and some of these events are logically related (belong to a single session), and there is a definite ordering among events based on timestamp.

Now, I want to build a solution that would should continue aggregating events until the last one of a particular group is received in increasing order of timestamp. So, if an event not in order is received, it should be held on until the events before them, are received. And then, go ahead with storing that in a data store such as HBase based on the key of that particular group.

The problem is the number of such incomplete groups at a time can be in the millions, and would need something that can support fast appending to the incomplete group and holding the events received unordered. How should I go about this?

Improve this question
1
  • Any little insight would be appreciable. Thanks! Commented Apr 3, 2015 at 12:20

1 Answer 1

Reset to default
1

you could use event stream processing or complex event processing frameworks http://en.wikipedia.org/wiki/Complex_event_processing. you write a query/pattern, load it in the engine, feed engine with events and the query is being continually updated or fires when the time comes.

I like Codehaus Esper, it's open source, but restricted in unpaid version, I had to implement storing of events and reloading them on application start for example. another option is Drools Fusion, but I found it buggy when I tried it out some 5 years ago. I was looking into WSO2 CEP recently but this thing is seriously underdocumented.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.