- create mediawiki/services/eventstreams repo
- create mediawiki/services/eventstreams/deploy repo with scap config and node_modules (figure out how services builds with librdkafka .deb)
- puppetize eventstreams
- test in deployment-prep
- figure out where this will be deployed (in REST API hierarchy? with varnish? restbase?)
Description
Description
Details
Details
Related Changes in Gerrit:
Customize query in gerrit
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Declined | Xqt | T125197 Give hint to the current socketIO_client in ImportError of rcstream.py | |||
| Declined | None | T91393 RCStream is not accessible from python client due to using socket-io 1.0 while only socket-io 0.9 is offered | |||
| Declined | None | T68232 Upgrade RCStream backend to use socket.io 1.0 protocol | |||
| Resolved | Ottomata | T130651 EventStreams | |||
| Resolved | Ottomata | T148470 Kafka SSE Prototype | |||
| Resolved | Ottomata | T148779 Prepare eventstreams (with KafkaSSE) for deployment | |||
| Resolved | Ottomata | T143925 Productionize and deploy Public EventStreams | |||
| Declined | Ottomata | T145805 Support node cluster sticky-session in service-runner | |||
| Declined | Ottomata | T148043 Fix consumer.disconnect() node-rdkafka bug |
Event Timeline
Comment Actions
- create mediawiki/services/eventstreams/deploy repo with scap config and node_modules (figure out how services builds with librdkafka .deb)
We use service-runner to build the deploy repo, and pull in librdkafka from WMF's APT repo by [specifying it in package.json](https://github.com/wikimedia/change-propagation/blob/1ff4e2ae9cb0be331b2f8b2adc83eb58e3c5d0e4/package.json#L70-L83).
- puppetize eventstreams
That should be as easy as defining the correct service::node resource. We can help with that.
- figure out where this will be deployed (in REST API hierarchy? with varnish? restbase?)
Definitely in the REST hierarchy. Given its streaming architecture, we need to weigh carefully where exactly to put it, but my first instinct would be to bypass Varnish altogether if possible not to clog connections.
Comment Actions
Change 317874 had a related patch set uploaded (by Ottomata):
Eventstreams with KafkaSSE initial commit on service-template
Comment Actions
Change 317981 had a related patch set uploaded (by Ottomata):
[WIP] evenstreams puppetization
Comment Actions
Change 317874 merged by Ottomata:
Eventstreams with KafkaSSE initial commit on service-template
Comment Actions
Change 319129 had a related patch set uploaded (by Ottomata):
Use Ruby yaml lib to render node::service deployment vars
Comment Actions
Change 319129 had a related patch set uploaded (by Ottomata):
Use ordered_yaml function to render node::service deployment vars
Comment Actions
Change 319129 merged by Ottomata:
Use ordered_yaml function to render node::service deployment vars
Comment Actions
Yes! Deployed to beta via puppet and scap.
This works!
http://codepen.io/ottomata/pen/VKNyEw?editors=0011
Now waiting on node-rdkafka to make a release.