Rebased on top of #1431.

@ianthomas23 FYI this seems to have caused lockups in nbclassic. See #1471 .

@jph00 It is possible that it is not needed, IDK.

@jph00 It is possible that it is not needed, IDK.

Yeah I've been testing without it and it's working great. To get unstuck I've released a version which removes the lock:
https://github.com/AnswerDotAI/ipyk-unlock

Do you mean it is working great just for your particular use case, or have you tried it with other reasonable use cases of ipykernel?

Do you mean it is working great just for your particular use case, or have you tried it with other reasonable use cases of ipykernel?

I've tested it with Jupyter Lab, nbclassic, and solveit (our proprietary app), using a number of notebooks which previously ipykernel 7 had struggled with.

Unfortunately it would also need to be tested in JupyterLab CI, i.e. an environment in which tens of messages are sent in quick succession for no sensible reason but that it what it chooses to do.

Yeah understood, I'm not suggesting you make changes - it's just FYI.

BTW I also created a test which sends lots of messages across lots of subshells and it fails with both the lock or no lock. I spent a few weeks trying to fix it but it seems very hard indeed to make subshells work reliably. For some reason in solveit and nbclassic they seem to exercise ipykernel in a way that is more susceptible to causing lockups. Anyhoo I'm happy to have something working now and to have a solution for other nbclassic users! :D