Downloaded 33 times
Uploaded byJose Galarza
Communication in Python and the C10k problem
The document discusses various communication methods in Python for handling the c10k problem, which involves managing over 10,000 simultaneous connections. It compares polling, long polling, HTTP streaming, Server-Sent Events (SSE), and WebSockets regarding efficiency and implementation ease. Additionally, it explores concurrency options in Python using libraries such as asyncio to efficiently handle multiple connections.
More Related Content
Similar to Communication in Python and the C10k problem
Communication in Python and the C10k problem
- 1. Communication in Pythonand the C10k problem Jose Ignacio Galarza @igalarzab
- 2.
- 4. select the best method to communicate
- 5. select the best method to scale
- 7. Index ✤ Frompolling… to pushing ✤ Concurrency ✤ C10k ✤ Asynchronous I/O
- 8. PYTHON 3 PYTHON3 EVERYWHERE
- 9. From polling topushing
- 10.
- 11. polling ➡ Nothingnew to implement ➡ Good support in all platforms ➡ Proxy friendly ➡ Uni-directional ➡ Very low efficiency ➡ No cross-domain
- 12. @app.route("/messages") def messages(): message = check_new_message() return message while True: polling r = requests.get('http://api.yo.com/messages') messages.extend(r.json()) time.sleep(5)
- 13. long polling ClientServer
- 14. long polling ➡Good support in all platforms ➡ “Proxy friendly” ➡ Uni-directional ➡ Low efficiency (better than polling) ➡ No cross-domain
- 15. long polling @app.route("/messages") def messages(): message = wait_until_new_message() return message while True: try: r = requests.get('http://yo.com/messages', timeout=60) except TimeOut: continue messages.extend(r.json())
- 16. HTTP Streaming ClientServer
- 17. HTTP Streaming ➡Better efficiency than (long) polling ➡ “Proxy friendly” ➡ Uni-directional ➡ You need to parse the data manually
- 18. HTTP Streaming ➡Type 1: Connection Close HTTP/1.1 200 OK Content-Type: text/plain Connection: close Hello world This a connection close response
- 19. HTTP Streaming ➡Type 2: Chunked response HTTP/1.1 200 OK Content-Type: text/plain Transfer-Encoding: chunked E Hello World! 19 I am a chunked response 0
- 20. HTTP Streaming @app.route("/messages") def messages(): def content(): for sentence in wait_until_next_sentence(): yield sentence return Response(content()) r = requests.get('http://yo.com/messages', stream=True) for message in r.iter_lines(): messages.extend(message.decode(‘utf8’))
- 21.
- 22. SSE ➡ Wellknown protocol (HTTP) ➡ Good efficiency ➡ JS API ➡ Few client implementations ➡ Uni-directional
- 23.
- 24. WebSockets ➡ Bidirectional ➡ Great efficiency ➡ A lot of implementations (JS API) ➡ Cross-domain ➡ Handshake to update from HTTP ➡ Complete different protocol
- 25. WS Handshake GET/chat HTTP/1.1 Host: server.example.com Upgrade: websocket Connection: Upgrade Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw== Sec-WebSocket-Protocol: chat, superchat Sec-WebSocket-Version: 13 Origin: http://example.com HTTP/1.1 101 Switching Protocols Upgrade: websocket Connection: Upgrade Sec-WebSocket-Accept: HSmrc0sMlYUkAGmm5OPpG2HaGWk= Sec-WebSocket-Protocol: chat Wikipedia
- 26. WebSockets @sockets.route('/messages') defmessages(ws): while True: message = wait_until_new_message() ws.send(message) def msg(ws, msg): messages.extend(msg) ws = websocket.WebSocketApp(‘ws://api.yo.com/', on_message=msg) ws.on_open = on_open ws.run_forever()
- 28. Bytes IN BytesOut Total Bytes Time (seconds) Polling 14640 13564 28204 159 Polling 2 7503 7636 15139 200 Long Polling 7503 7636 15139 120 Streaming 183 1549 1732 120 WebSockets 209 1605 1814 120
- 29. 30000 27000 24000 21000 18000 15000 12000 9000 6000 3000 0 Polling Polling 2 Long Polling Streaming WebSockets Total Bytes Bytes Out Bytes IN
- 30. Concurrency and the C10k
- 31. C1 0k? howto handle more than 10k connections simultaneously?
- 32. concurrency is… theability of running in overlapping time periods, not necessarily at the same time
- 33.
- 34. processes ➡ Preemptivescheduling by the OS ➡ Separate memory space ➡ No GIL related issues ➡ How to communicate them? ➡ They are really heavy
- 35. threads ➡ Preemptivescheduling by the OS ➡ Same memory space ➡ Faster and lighter than processes ➡ You may suffer the GIL ➡ What about synchronisation? ➡ Race conditions
- 36. user threads ➡Same space address ➡ Lightest alternative ➡ No race-conditions ➡ You may suffer the GIL ➡ Very bad with CPU bound tasks
- 37.
- 38.
- 39. again… how tohandle more than 10k connections simultaneously?
- 40. option 1 use10k machines, one per client, VIP service
- 41. option 2 useblocking and synchronous calls, one per client
- 42. option 3 non-blockingcalls to start I/O and then readiness notifications to know when the socket is ready
- 43. option 4 asynchronouscalls to start I/O and completion notifications to know when they’ve finished
- 44.
- 45.
- 46.
- 47. asyncio ➡ Single-threaded ➡ Async I/O ➡ Coroutines ➡ Multiplexes I/O events ➡ …
- 48.
- 49.
- 50.
- 55. X Game OfLife
- 56.
- 58.
- 59.