If you can start an X server you might have some luck using Xpra. Xpra is a special window manager that uses the Composite extension to get hold of each window; it then wraps the window's contents in an efficient encoding to be viewed by Xpra clients.
Xpra itself is written in Python with a little C glue code and can be installed also in a user's home directory. You can specify the X server and the commands line to use with the --xvfb option on the xpra server side. Although it's named xvfb, any X server will do.
Take notice that with the current OpenGL and graphics driver models on Linux only one X server can hold the GPU at a time. So you can't start multiple Xpra instances on the GPU on the same time. Also if there was already another user of the GPU you can't use it as well. Hopefully this limitation goes away soon with offscreen hardware accelerated EGL support.
In the meantime using Xpra is my personal preferred choice for remote high performance OpenGL rendering.