Changes to Cascade, and a cautionary tale about defrecord

Since I've been talking more about Clojure lately, I've spent a little more time working on Cascade. I've been stripping out a lot of functionality, so that Cascade will no work with Ring and Compojure, rather than being in competition. Its Clojure, after all, ... there's less of a reason to build a full framework since its so easy to simply assemble your functionality from proper libraries.

It's also been a chance to update some of my code with more modern constructs. For example, the earlier version of Cascade used a (defstruct) for the DOM nodes; the new code uses (defrecord).

Along the way I discovered something interesting about defrecord. Consider this code:

Technically, this is just an optimized way to define a Clojure Map. If I have an instance, I can (:text node) to get the text out of the map.

However, (defrecord) does one other thing that is barely mentioned in the documentation (and not referenced, that I can tell, in Joy of Clojure). Notice the implementation of the stream function (part of the NodeStreaming protocol). It just says text; not (:text node). Inside a protocol method, the fields of the record are bound to local variables, making them easy to use ... another benefit.

I actually found this the hard way, when writing a more complicated example, for the Element DOM node:

Notice the use of clojure.core/name to convert a keyword to a string; originally this was (name (:name node)) and returned nil. This confused me quite a bit!

What ended up happening was that name was bound to the keyword from the record's name field. However, Clojure keywords can be used as functions,and was incidentally passed itself, which is to say (for an Element node representing a <p> element): (name (:name node)) --> (:p :p) --> nil.

So, (defrecord) giveth, but it also taketh away, at least, the first time. In other words, watch out for name collisions between the names of the record's fields, and the names of functions you want to reference from your protocol method implementations.

Back to Cascade; I don't have any metrics available about performance changes with the new code (using records and protocols), but I suspect its faster and more efficient.

A lot of the features that were in Cascade are gone and will come back soon. Ultimately, I'll have Cascade flavors of context and classpath assets from Tapestry, as well as mechanisms similar to Tapestry for adding JavaScript libraries and CSS stylesheets, along with a mechanism similar to Tapestry for organizing them into stacks.

Looking further forward, adding support for Enlive, both reading parsed XML templates in as DOM structure and allowing Enlive transformations onto the DOM structure, seems like a good direction.

When will all this happen? I'm not certain, but I hope that Cascade will become a "must-have" layer on top of Compojure, adding some of the industrial strength concepts from Tapestry into the fast-and-loose world of Clojure web applications.

Progress on Cascade

Meanwhile, in spare minutes (and during sessions at ApacheCon), I've been continuing to work on Cascade. It's been a great learning exercise for me, pushing my understanding of both Clojure and functional programming in general ... and especially, some pretty advanced meta-programming with macros.

I'm also using Cascade as a kind of test bed for ideas that will eventually appear in Tapestry.

Not everything turns out exactly as I've planned. For example, I've been very excited about invariants, portions of the rendered DOM that could be cached from one request to another, to speed up the rendering. Like Tapestry, Cascade views render a DOM structure which can be manipulated (in an intermediate stage) before being streamed as text. This is a useful and powerful concept in a number of ways.

My thinking has been that a typical view will contain sections of the template that are invariant: unchanging, and that there would be a benefit to building that sub-section of the DOM once and reusing it efficiently in later renderings of the view.

Clojure template forms are processed by macros to become normal Clojure code. Thus something like (template :p [ "Hello" ]) will be transformed into code, approximately (element-node :p nil (combine (text-node "Hello"))). My approach was to tag the new Clojure code forms (the list consisting of element-node, :p, etc.) with meta data to identify it as invariant. Eventually this would propagate up to a higher level and code to manage a cache would be wrapped around it: (or (read-view-cache 97) (update-view-cache 97 (element-node :p ...

Fun stuff ... until I put it into practice (after a fair amount of debugging) and discovered that in the views I've created so far (for testing purposes), the number of nodes that can be cached is low; any use of a symbol or a function call mixed into the template "taints" it as variant. I wasn't set up to do performance measurements, but my gut feeling is that the overhead of managing the caches would overshadow the savings from the small islands of reused DOM nodes.

Back to Cascade as a learning experience: just because this didn't work out doesn't mean I didn't learn a lot from heading down that direction, and certainly the amount of code it took was quite small. I have it stored in a branch in case I ever want to give it another shot.

I will have all the basic features of Cascade implemented pretty soon; I'm looking forward to seeing what the larger Clojure community makes of it. In the meantime, it has served as a great way for me to dig deep into Clojure ... I'll be putting together more sessions for NoFluffJustStuff and more articles for the companion magazine based on all this.

Cascade Exception Reporting

I've been taking a little time from my billable projects to continue working on Cascade. One feature that's very important to me is to have great exception reporting, akin to Tapestry's. Here's a current snapshot of where I am:

This is very Tapestry-like (I've even borrowed the CSS styles). You can even see the start of the Request object's properties being displayed.

Something to notice here: Clojure stack frames are in Clojure syntax. To appreciate this, see what you get when you click the "Display hidden detail" button:

The exception report view is omitting a lot of clojure.lang internals, and it is working backwards from the mangled Java class name to the Clojure namespace and function name. This, plus only displaying the stack trace for the root exception, makes it much more reasonable to figure out where problems are actually occurring.

I expect to expand this further, adding a pop-up or hover window to display Clojure source associated with the stack frame.