Skip to content

Refactor WebAgg so it can communicate over another web server - #2420

Merged
mdboom merged 3 commits into
matplotlib:masterfrom
mdboom:webagg-factor
Sep 30, 2013
Merged

mdboom merged 3 commits into
matplotlib:masterfrom
mdboom:webagg-factor

Conversation

@mdboom

@mdboom mdboom commented Sep 13, 2013

Copy link
Copy Markdown
Member

This is to address an immediate need we have at STScI, but probably needs some explanation.

We want to embed interactive WebAgg plots inside of a larger web application, but we don't necessarily want to serve the matplotlib websocket or http requests using tornado. We'd like to, instead, allow people to "bring their own webserver framework" to the party and have matplotlib communicate over that. (@stsci-sienkiew). The requirement for "websockets" specifically is still there.

This refactor should also make it easier to move to IPython's future widget communication pipeline down the road (@ellisonbg).

The easiest way to see how this works is to look at the embedding_webagg.py example, which embeds a plot in an example tornado application. I'm using tornado here for the example, even though that's what the "built-in" matplotlib server uses because it's really the only viable and mature Python tool for websockets I could find. It still makes for a useful example for how to embed in a larger Tornado application which might be doing many other things. I plan to write proper "how to" documentation for this, but thought I should hammer down the details first.

As part of this, the Javascript API has all been moved under a single "mpl" namespace, so that it should be easier to import it alongside other Javascript libraries without fear of name clashes. If anyone is currently using the Javascript API to matplotlib to insert plots on their webpages this will break that. As this stuff is all still new and marked "experimental", I'd rather just break this now to get that right rather than carry around the baggage of not namespacing things correctly to begin with.

Cc: @pelson, @andreabedini (as people who have helped with and been interested in WebAgg).

@pelson: You may also enjoy that, ignoring the example, this is a "net deletion" pull request.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a certain je ne sais quoi missing here, namely the backend_webagg_core code 😊

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doh! Commit has been amended.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excuse my web-programming ignorance, but does this mean that it will be
possible to use WebAgg in a flask application served up via mod_wsgi in
apache? Asking for a friend...

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If such a configuration supports websockets, yes, at least that's the idea. Due to my own web-programming ignorance, I won't guarantee it, but it would be great to have your friend's assistance working out any kinks or shortcomings to this approach.

@pelson

pelson commented Sep 13, 2013

Copy link
Copy Markdown
Member

You may also enjoy that, ignoring the example, this is a "net deletion" pull request.

I'm always a fan of those, though I fear that conclusion is a result of backend_webagg_core.py not having been added 😄.

In principle I'm happy about this refactor, looks like a sane way of going about making the webagg solution better and more re-usable.

@pelson

pelson commented Sep 13, 2013

Copy link
Copy Markdown
Member

Just tried out the example - the image is a little small in Firefox 17.6 (the status update is too big) and it seems much more laggy on localhost than it used to?
Also, I think the example should probably print something to the standard out to tell you to visit the necessary URL.

@mdboom

mdboom commented Sep 13, 2013

Copy link
Copy Markdown
Member Author

@pelson: I've addressed your easy comments.

As for the lag -- the timing is handled differently here than in the built-in server. I was trying to eliminate the dependence on having a scheduler in the server. Rather than scheduling redraws on idle, it just throttles the framerate. I tweaked it until it felt approximately the same for me, but I suspect it's a little to tied to how things work for me. I might experiment with trying to the scheduling on the Javascript side, which would also eliminate needing any special features in the server.

@mdboom

mdboom commented Sep 13, 2013

Copy link
Copy Markdown
Member Author

Consider this experimental -- I have attached another approach that's completely timerless. It has the browser control when new frames should be fetched. It seems to perform much better locally. I haven't been able to tunnel into the office to simulate running over a slow internet link, but obviously we should ensure this works there, too.

@ellisonbg

Copy link
Copy Markdown

Thanks for the ping. @minrk has opened a PR that starts to implement IPython's new message spec for widgets. It is too early to start integration, but we would love to get your eyes on this soon. Overall the messaging is very similar to WebSockets - basically micro comm channels over one WebSocket.

In terms of the requirements for this approach as @mdboom mentions anyone implementing a compliant server would need to have WebSocket support. This means you need to use an asynch framework such as tornado, twisted, gevent, etc. Flask will definitely not work.

@WeatherGod

Copy link
Copy Markdown
Member

Ah, thanks for the clarification, Brian. Like I said, I am -- err, I mean
-- my friend is very inexperienced with web development and has just been
blindly working with very simple examples. He certainly would have never
noticed such an important detail for quite a while. Any suggestions for a
web framework that could be used through Apache (since my friend's
workplace would likely never approve any other web server)?

@ellisonbg

Copy link
Copy Markdown

Apache is the worst possible choice at this point. Apache does not know
how to proxy websockets as far as I know. From this post:

http://serverfault.com/questions/290121/configuring-apache2-to-proxy-websocket

is looks like there are some work arounds, but I would probably not advise
going in that direction. The servers that have good WebSocket proxy support
are nginx and HAProxy. node-http-proxy also has nice support for
WebSockets. But these will still require using something like tornado,
gevent or twisted for the Python side of things. Not good news for your
friend...

On Fri, Sep 13, 2013 at 10:54 AM, Benjamin Root notifications@github.comwrote:

Ah, thanks for the clarification, Brian. Like I said, I am -- err, I mean
-- my friend is very inexperienced with web development and has just been
blindly working with very simple examples. He certainly would have never
noticed such an important detail for quite a while. Any suggestions for a
web framework that could be used through Apache (since my friend's
workplace would likely never approve any other web server)?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/2420#issuecomment-24412522
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

@minrk

minrk commented Sep 13, 2013

Copy link
Copy Markdown
Contributor

I haven't tested it, but I think apache can proxy websockets now.

sends a "draw" message to the browser.  If the browser is not already
waiting on another frame, it sends a "draw" message back to the server
which then returns an actual frame.  The nice thing about this approach
is that it's completely timerless.
mdboom added a commit that referenced this pull request Sep 30, 2013
Refactor WebAgg so it can communicate over another web server
@mdboom
mdboom merged commit 9804c5d into matplotlib:master Sep 30, 2013
@mdboom
mdboom deleted the webagg-factor branch August 7, 2014 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants