Repository navigation
KernelClient: factor out generic async client from Qt client - #7196
Conversation
|
Could you point to a code example that used the old behavior? |
|
Sure, here is a (work in progress) Neovim frontend ( even more WIP ipy3-dev port here ). It is based on/similar to ivanov/vim-ipython , expect that communication between Neovim and the kernel is asynchronous. The code is a bit messy, but here is the part connecting to the client: |
There was a problem hiding this comment.
This means that _check_kernel_info_reply() is now getting called in the ioloop thread, whereas previously it was called in the main application thread. I think this is safe, but at a minimum, we should put comments in _check_kernel_info_reply and _handle_kernel_info_reply mentioning this fact, so we don't do something later that assumes they're in the main thread.
There was a problem hiding this comment.
Oh, I didn't realize when i did this, than Qt signals sent in a helper thread are handled in the main thread. I will look a bit more into the client code and try see if this is thread-safe.
There was a problem hiding this comment.
It should be fine - it's handing over the already-parsed message to be handled. If anything in either handler were to modify the dict in-place, it would not be threadsafe, but message dicts should not be modified, only read from, so we should be okay.
There was a problem hiding this comment.
And even then, it's only possible to have an issue when there are two listeners on the same message - one on the QtSignal and one on the inner loop, one of which is modifying the message content, which it shouldn't do.
There was a problem hiding this comment.
I was thinking about the thread safety of the changes it makes to session to set up the protocol adaptation. But on thinking some more, I guess that session is mostly/only used from the ioloop thread once it's been set up, so it should be OK.
There was a problem hiding this comment.
Yes session.adapt_version is both set and used in the ioloop thread, so this should be safe.
|
Thanks, I like this approach - it reintroduces the important bit of flexibility from the old design, but with very minimal extra complexity added to the current state. I do want to think a bit more about the name. There's more than one way to build an async kernel client, and this is rather the quick and dirty way. The more elegant approach would be to integrate the ZMQ sockets into the application's own event loop. Maybe (*this makes our ioloop thread completely redundant: ZMQ is doing the same thing internally, reading from sockets in a thread and making callbacks when a message is ready.) |
The special handling of kernel_info_reply is so that the client can work with kernels that don't speak the newest version of the protocol. It makes a kernel info request when the kernel starts, and the reply tells it what version of the protocol the kernel is using. The client can then translate between the protocol the kernel uses and the protocol the client expects. |
|
Thank you for your comments. I would say |
|
Updated: renamed to Threaded* and added comments about thread-safety of kernel-info reply. |
There was a problem hiding this comment.
Rather than making this a new package with only one code file, let's just have a threaded.py file under kernel. We can transparently turn it into a package later if we need more code in there.
|
Ok, done |
|
Thanks. Once Travis has checked it again, I'll hit the merge button. |
|
There seems to be a name conflict on 2.7 about |
|
Good catch, and I completely agree with your fix. |
|
Thanks, now it passes. |
|
Great, thank-you! |
KernelClient: factor out generic async client from Qt client
before #7108 it was simple for a third-party front-end to recieve message notifications asynchronously from (non-blocking) KernelClient by overriding, or even monkeypatching the
call_handlersmethod on the channels like thisThis is not as simple anymore; now
qt.clientimplements the requied logic (a single ioloop thread) but it is tied to the Qt signal/slot mechanism. This PR splits out a generic async kernel client from QtKernelClient, that essentially preserves the behaviour of the oldKernelClient(execpt that is uses less threads, thanks to #7108).The only (intended) "semantical" change is that
_check_kernel_info_clientis moved outsidecall_handlers(so that subclasses don't need to care about this), but I don't know the exact purpose this have, so I can't be entirely sure this is handled correctly.I'm not sure
AsyncKernelClientis the best name tho, perhapsThreadKernelClientis more explicit?