Tapestry Quicky: ConditionalComment

Here's a quicky I just put together for a client to generate IE conditional comments. This isn't a feature supported by Tapestry's JavaScriptSupport (for libraries; it is support for CSS files).

Fortunately, this is something that comes together in Tapestry in almost no code:

This is what it looks like in a template:

And here's the markup it produces:

And that's the whole point ... the URLs for the referenced JavaScript libraries are now Tapestry assets, and will have access to Tapestry's asset pipeline as unique resources, including GZip compression and minimization.

It is always important to me that Tapestry not get in the way: frameworks that lock you down and prevent you from accomplishing your goals should not be used. Tapestry has a long history, and there are certainly many areas that could have been implemented differently, or simply not implemented at all, but it's nice that most edge cases, like this one, have a simple solution.

CoffeeScript Cautions

Over the last couple of months, I've coded nearly all my client-side code in CoffeeScript. I prefer CoffeeScript to JavaScript ... but there are a few things to watch out for. I thought I'd share some practical experience, rather than the more typical hype for CoffeeScript or the reactionary vibe against it.

My biggest problems with CoffeeScript has been with indentation, which is no surprise, given the basic value proposition of the language. There are places where I've accidentally deleted a single space and completely changed the meaning of my code.

Here's an adapted example. We'll start with the correct CoffeeScript source code, and JavaScript translation:

Now, notice what happens when we delete a single space.

The change in indentation confused the CoffeeScript compiler; it ended the call to View.extend() and created a new expression for an object literal that is created and thrown away.

And remember, this isn't hypothetical; I really did this, and started seeing very odd errors: undefined properties and events not getting triggered. It took me a bit of time, with the debugger, and adding console logging, and eventually some viewing of the generated JavaScript, to realize what had happened.

Part of my solution to this class of issue, in Emacs, is to automatically increase the font size when editing CoffeeScript files. I may have to consider a switch from spaces to tabs as well, something that I've never considered in the past. I've also noticed that some IDEs, including IntelliJ, draw a vertical line to connect code at the same indentation layer. I don't have enough Emacs hacking experience to accomplish that, but I hope someone does.

Another problem I've hit is with function invocation; CoffeeScript style is to omit the parenthesis following the function name, and just immediately start listing function parameters:

element.setAttribute "width", "100%"

However, if your API allows chaining, you may be tempted to:

element.setAttribute "width", "100%".setAttribute "color", "blue"

Alas, the .setAttribute binds to the immediate value, the string:

element.setAttribute("width", "100%".setAttribute("color", "blue"));

One option is to enclose the first call with parenthesis:

(element.setAttribute "width", "100%").setAttribute "color", "blue"

I want to like that option, as it stays consistent with the normal format for function invocation; it just captures the result for chaining. However, each additional setAttribute() call will require an additional nesting layer of parenthesis. This leaves the following as the solution:

element.setAttribute("width", "100%").setAttribute("color", "blue")

And now we are stuck having to parse two different syntaxes for function invocation, depending on some very specific context.

Another issue is that, once you start using the standard function invocation style, which doesn't use parenthesis, you can easily get tripped up: Here the is null bound tightly to the "div" string literal; the result (true or false) was passed to the find() method, rather than the expected "div".

Because of these subtleties, I seem to find myself using the live CoffeeScript to JavaScript preview to check that my guess at how the JavaScript will be generated is correct. Preview is available on the CoffeeScript home page, and also as the CoffeeConsole plugin to Chrome.

What's interesting is that the above concerns aside, most of the time my first pass at the CoffeeScript does produce exactly what I'd expect, making me feel silly to waste the time to check it! Other times, I find that my first pass is actually too verbose, and can be simplified. Here is an example from some code I've written for Tapestry, a simple DSL for constructing DOM elements on the fly: This is invoking the builder() function, passing a string to define the element type and CSS class, then an object that adds attributes to the element, and lastly some additional values that define the body of the element: an array to define a nested element, and a string that will become literal text. This is converted to JavaScript:

Turns out, we can get rid of the curly braces on the objects without changing the output JavaScript; CoffeeScript is able to figure it all out by context:

Even getting rid of the commas is allowed. The commas are equivalent to the line break and indentation that's already present:

And that's where I start really liking CoffeeScript, because quite often, the code you write looks very close to a bespoke DSL.

Then there's the debugging issue; CoffeeScript scores poorly here. First of all, the code has been translated to JavaScript; line numbers at execution time will no longer line up with source CoffeeScript at all, so the first and most useful bit of information in a stack trace is no longer useful. It's very easy to glance back and forth to match the generated JavaScript back to source CoffeeScript, but it still takes some effort.

Of course, in a production application, you'll likely have minimized your JavaScript, which mangles your JavaScript and your line numbers far more than the CoffeeScript compilation does!

The other weakness is that all functions in CoffeeScript are anonymous; this takes another useful bit of information out of stack traces: the local function name. This leaves the only way to map from compiled JavaScript back to source CoffeeScript as involving a lot of processing between your ears, and that's not exactly ideal, given everything else you have to mentally juggle.

Lastly, I've heard of people who think CoffeeScript is only for programming using objects and classes. I find this quite odd; it's nice to have some optimized syntax for classes in CoffeeScript even if it comes at some price (the generated JavaScript can become quite verbose adding runtime support for the syntax that allows super class methods and constructors to be invoked); however, far beyond 95% of the code I write is purely functional, with what state that's present captured in closures of local variables, they way it should be.

I like Scott Davis' summary: CoffeeScript will make you a better JavaScript programmer. It is, largely, JavaScript with better syntax, plus a number of new features. You should never think of CoffeeScript as an alternative to understanding the occasionally ugly nuances of JavaScript; instead I think of it as a "have your cake and eat it too" situation. I certainly don't consider it noobscript, the way many in the Node.js community seem to.

At the end of the day, the translation from CoffeeScript to JavaScript is usually quite predictable, and turns into the JavaScript you probably should write, but often would not: for instance, JavaScript that always evaluates inside a hygienic function, or where every variable is defined at the top of its containing block.

My final take is that you are trading the very rough edges present in the JavaScript language for some sensible tradeoffs, and the occasional hidden "gotcha". I think that's a completely reasonable trade, and will continue to write as much as I can, client-side and server-side, in CoffeeScript.

A little Gotcha with asynch and Streams

I stumbled across a little gotcha using async with Node.js Streams: you can easily corrupt your output if you are not careful.

Node.js Streams are an abstraction of Unix pipes; they let you push or pull data a little bit at a time, never keeping more in memory than its needed. async is a library used to organize all the asynchronous callbacks used in node applications without getting the kind of "Christmas Tree" deep nesting of callbacks that can occur too easily.

I'm working on a little bit of code to pull an image file, stored in MongoDB GridFS, scale the image using ImageMagick, then stream the result down to the browser.

My first pass at this didn't use ImageMagick or streams, and worked perfectly ... but as soon as I added in the use of async (even before adding in ImageMagick), I started getting broken images in the browser, meaning that my streams were getting corrupted.

Before adding async, my code was reasonable:

However, I knew I was going to add a few new steps here to pipe the file content through ImageMagick; that's when I decided to check out the async module.

The logic for handling this request is a waterfall; each step kicks off some work, then passes data to the next step via an asynchronous callback. The async library calls the steps "tasks"; you pass an array of these tasks to async.waterfall(), along with the end-of-waterfall callback. This special callback may be passed an error provided by any task, or the final result from the final task.

With waterfall(), each task is passed a special callback function. If the callback function is passed a non-null error as the first parameter, then remaining tasks are skipped, and the final result handler is invoked immediately, to handle the error.

Otherwise, you pass null as the first parameter, plus any additional result values. The next task is passed the result values, plus the next callback. It's all very clever.

My first pass was to duplicate the behavior of my original code, but to do so under the async model. That means lots of smaller functions; I also introduced an extra step between getting the opened file and streaming its contents to the browser. The extra step is intended for later, where ImageMagick will get threaded in.

The code, despite the extra step, was quite readable:

My style is to create local variables with each function; so openFile kicks off the process; once the file has been retrieved from MongoDB, the readFileContents task will be invoked ... unless there's an error, in which case errorCallback gets invoked immediately.

Inside readFileContents we convert the file to a stream with file.stream(true) (the true means to automatically close the stream once all of the file contents have been read from GridFS).

streamToClient comes next, it takes that stream and pipes it down to the browser via the res (response) object.

So, although its now broken up into more small functions, the logic is the same, as expressed on the very last line: open the file, read its contents as a stream, stream the data down to the client.

However, when I started testing this before moving on to add the image scaling step, it no longer worked. The image data was corrupted. I did quite a bit of thrashing: adding log messages, looking at library source, guessing, and experimenting (and I did pine for a real debugger!).

Eventually, I realized it came down to this bit of code from the async module:

The code on line 7 is the callback function passed to each task; notice that once it decides what to do, on line 21 it defers the execution until the "next tick".

The root of the problem was simply that the "next tick" was a little too late. By the time the next tick came along, and streamToClient got invoked, the first chunk of data had already been read from MongoDB ... but since the call to pipe() had not executed yet, it was simply discarded. The end result was that the stream to the client was missing a chunk at the beginning, or even entirely empty.

The solution was to break things up a bit differently, so that the call to file.stream() happens inside the same task as the call to stream.pipe().

So that's our Leaky Abstraction for today; what looked like an immediate callback was deferred just enough to change the overall behavior. And that, in Node, anything that can be deferred, will be deferred, since that makes the overall application that much zippier.

Tapestry 5.4: Focus on JavaScript

Tapestry 5.3.1 is out in the wild ... and if Tapestry is to stay relevant, Tapestry 5.4 is going to need to be something quite (r)evolutionary.

There was some confusion on the Tapestry developer mailing list in advance of this blog post; I'd alluded that it was coming, and some objected to such pronouncements coming out fully formed, without discussion. In reality, this is just a distillation of ideas, a starting point, and not a complete, finalized solution. If it's more detailed than some discussions of Tapestry's evolution in the past, that just means that the mailing list discussion and eventual implementation will be that much better informed.

In posts and other conversations, I've alluded to my vision for Tapestry 5.4. As always, the point of Tapestry is to allow developers to code less, deliver more, and that has been the focus of Tapestry on the server side: everything drives that point: terseness of code and templates, live class reloading, and excellent feedback are critical factors there. Much of what went into Tapestry 5.3 strengthened those points ... enhancements to Tapestry's meta-programming capabilities, improvements to the IoC container, and reducing Tapestry's memory footprint in a number of ways. I have one client reporting a 30% reduction in memory utilization, and another reporting a 30 - 40% improvement in execution speed.

Interestingly, I think that for Tapestry to truly stay relevant, it needs to shift much, much, more of the emphasis to the client side. For some time, Tapestry has been walking a fine line with regards to the critical question of where does the application execute? Pre-Ajax, that was an easy question: the application runs on the server, with at most minor JavaScript tricks and validations on the client. As the use of Ajax has matured, and customer expectations for application behavior in the browser have expanded, it is no longer acceptable to say that Tapestry is page based, with limited Ajax enhancements. Increasingly, application flow and business logic need to execute in the browser, and the server-side's role is to orchestrate and facilitate the client-side application, as well as to act as a source and sink of data ultimately stored in a database.

As Tapestry's server-side has matured, the client side has not kept sufficient pace. Tapestry does include some excellent features, such as how it allows the server-side to drive client-side JavaScript in a modular and efficient way. However, that is increasingly insufficient ... and the tension caused by give-and-take between client-side and server-side logic has grown with each release.

Nowhere is this more evident than in how Tapestry addresses HTML forms. This has always been a tricky issue in Tapestry, because the dynamic rendering that can occur needs to be matched by dynamic form submission processing. In Tapestry, the approach is to serialize into the form instructions that will be used when the form is submitted (see the store() method of the FormSupport API). These instructions are used during the processing of the form submission request to re-configure the necessary components, and direct them to read their query parameters, perform validations, and push updated values back into server-side objects properties. If you've ever wondered what the t:formdata hidden input field inside every Tapestry forms is about ... well, now you know: it's a serialized stream of Java objects, GZipped and MIME encoded.

However, relative to many other things in Tapestry, this is a bit clumsy and limited. You start to notice this when you see the tepid response to questions on the mailing list such as "how to do cross-field validation?" Doing more complicated things, such as highly dynamic form layouts, or forms with even marginal relationships between fields, can be problematic (though still generally possible) ... but it requires a bit too much internal knowledge of Tapestry, and the in-browser results feel a bit kludgy, a bit clumsy. Tapestry starts to feel like it is getting in the way, and that's never acceptible.

Simply put, Tapestry's abstractions on forms and fields is both leaky and insufficient. Tapestry is trying to do too much, and simply can't keep up with modern, reasonable demands in terms of responsiveness and useability inside the client. We've become used to pages rebuilding and reformatting themselves even while we're typing. For Tapestry to understand how to process the form submission, it needs a model of what the form looks like on the client-side, and it simply doesn't have it. There isn't an effective way to do so without significantly restricting what is possible on the client side, or requiring much more data to be passed in requests, or stored server-side in the session.

The primary issue here is that overall form submission cycle, especially combined with Tapestry's need to serialize commands into the form (as the hidden t:formdata field). Once you add Ajax to this mix, where new fields and rules are created dynamically (on the server side) and installed into the client-side DOM ... well, it gets harder and harder to manage. Add in a few more complications (such as a mix of transient and persistent Hibernate entities, or dynamic creation of sub-entities and relationships) into a form, it can be a brain burner getting Tapestry to do the right thing when the form is submitted: you need to understand exactly how Tapestry processes that t:formdata information, and how to add your own callbacks into the callback stream to accomplish just exactly the right thing at just exactly the right time. Again, this is not the Tapestry way, where things are expected to just work.

Further, there is some doubt about even the desirability of the overall model. In many cases, it makes sense to batch together a series of changes to individual properties ... but in many more, it is just as desirable for individual changes to filter back to the server (and the database) as the user navigates. Form-submit-and-re-render is a green screen style of user interaction. Direct interaction is the expectation now, and that's something Tapestry should embrace.

What's the solution, then? Well, it's still very much a moving target. The goal is to make creating client-side JavaScript libraries easier, to make it easier to integrate with libraries such as jQuery (and its vast library of extensions), make things simpler and more efficient on the client side, and not sacrifice the features that make Tapestry fun and productive in the first place.

Overall Vision

The overall vision breaks down into a number of steps:

• Reduce or remove outside dependencies
• Modularize JavaScript
• Change page initializations to use modules
• Embrace client-side controller logic

Of course, all of these steps depend on the others, so there isn't a good order to discuss them.

Reducing and removing outside dependencies

Tapestry's client-side strength has always been lots of "out of the box" functionality: client-side validation, Zones and other Ajax-oriented behaviors, and a well-integrated system for performing page-level initializations.

However, this strength is also a weakness, since that out of the box behavior is too tightly tied to the Prototype and Scriptaculous libraries ... reasonable choices in 2006, but out-of-step with the industry today. Not just in terms of the momentum behind jQuery, but also in terms of very different approaches, such as Sencha/ExtJS and others.

It was a conscious decision in 2006 to not attempt to create an abstraction layer before I understood all the abstractions. I've had the intermediate time to embrace those abstractions. Now the big problem is momentum and backwards compatibility.

Be removing unnecessary behaviors, such as animations, we can reduce Tapestry's client-side needs. Tapestry needs to be able to attach event handlers to elements. It needs to be able to easily locate elements via unique ids, or via CSS selectors. It needs to be able to run Ajax requests and handle the responses, including dynamic updates to elements.

All of these things are reasonable to abstract, and by making it even easier to execute JavaScript as part of a page render or page update (something already present in Tapestry 5.3), currently built-in features (such as animations) can be delegated to the application, which is likely a better choice in any case.

Modularizing JavaScript

Tapestry has always been careful about avoiding client-side namespace polution. Through release 5.2, most of Tapestry's JavaScript was encapulated in the Tapestry object. In Tapestry 5.3, a second object, T5 was introduced with the intention that it gradually replace the original Tapestry object (but this post represents a change in direction).

However, that's not enough. Too often, users have created in-line JavaScript, or JavaScript libraries that defined "bare" variables and functions (that are ultimately added to the browser's window object). This causes problems, including collisions between components (that provide competing definitions of objects and functions), or behavior that varies depending on whether the JavaScript was added to the page as part of a full-page render, or via an Ajax partial page render.

The right approach is to encourage and embrace some form of JavaScript module architecture, where there are no explicit global variables or functions, and that all JavaScript is evaluated inside a function, allowing for private variables and functions.

Currently, I'm thinking in terms of RequireJS as the way to organize the JavaScript. Tapestry would faciliate organizing its own code into modules, as well as application-specific (or even page-specific) JavaScript modules. This would mean that de-referencing the T5 object would no longer occur (outside of some kind of temporary compatibility mode).

For example, clicking a button inside some container element might, under 5.3, publish an event using Tapestry's client-side publish/subscribe system. In the following example, the click events bubble up from the buttons (with the button CSS class name) to a container element, and are then published under the topic name button-clicked.

Consider this an abbreviated example, as it doesn't explain where the element variable is defined or initialized; the important part is the interaction with Tapestry's client-side library: the reference to the T5.pubsub.publish function.

Under 5.4, using the RequireJS require function, this might be coded instead as:

Here, the t5/pubsub module will be loaded by RequireJS and passed as a parameter into the function, which is automatically executed. So, this supports JavaScript modularization, and leverages RequireJS's ability to load modules on-the-fly, as needed.

Notice the difference between the two examples; in the first example, coding as a module was optional (but recommended), since the necessary publish() function was accessible either way. In the 5.4 example, coding using JavaScript modules is virtually required: the anonymous function passed to require() is effectively a module, but its only through the use of require() (or RequireJS's define()) that the publish() function can be accessed.

This is both the carrot and the stick; the carrot is how easy it is to declare dependencies and have them passed in to your function-as-a-module. The stick is that (eventually) the only way to access those dependencies is by providing a module and declaring dependencies.

Change page initializations to use modules

Tapestry has a reasonably sophisticated system for allowing components to describe their JavaScript requirements as they render, in the form of the JavaScriptSupport environmental (an environmental is a kind of per-thread/per-request service object). Methods on JavaScriptSupport allow a component to request that a JavaScript library be imported in the page (though this is most commonly accomplished using the Import annotation), and to request the initialization functions get executed.

Part of Tapestry's Ajax support is that in an Ajax request, the JavaScriptSupport methods can still be invoked, but a completely different implementation is responsible for integrating those requests into the overall reply (which in an Ajax request is a JSON object, rather than a simple stream of HTML).

Here's an example component from the TapX library:

The @Import annotation directs that a stack (a set of related JavaScript libraries, defined elsewhere) be imported into the page; alternately, the component could import any number of specific JavaScript files, located either in the web application context folder, or on the classpath.

Inside the afterRender() method, the code constructs a JSONObject of data needed on the client side to perform the operation. The call to addInitializerCall references a function by name: this function must be added to the T5.Initializers namespace object. Notice the naming: tapxExpando: a prefix to identify the library, and to prevent collisions with any other application or library that also added its own functions to the T5.initializers object.

The JavaScript library includes the function that will be invoked:

Under 5.4, this would largely be the same except:

• There will be a specific Java package for each library (or the application) to store library modules.
• The JavaScriptSupport environmental will have new methods to reference a function, inside a module, to invoke.
• Stacks will consist not just of individual libraries, but also modules, following the naming and packaging convention.

Embrace client-side controller logic

The changes discussed so far only smooth out a few rough edges; they still position Tapestry code, running on the server, as driving the entire show.

As alluded to earlier; for any sophisticated user interface, the challenge is to coordinate the client-side user interface (in terms of form fields, DOM elements, and query parameters) with the server-side components; this is encoded into the hidden t:formdata field. However, it is my opinion that for any dynamic form, Tapestry is or near the end of the road for this approach.

Instead, it's time to embrace client-logic, written in JavaScript, in the browser. Specifically, break away from HTML forms, and embrace a more dynamic structure, one where "submitting" a form always works through an Ajax update ... and what is sent is not a simple set of query parameters and values, but a JSON representation of what was updated, changed, or created.

My specific vision is to integrate Backbone.js (or something quite similar), to move this logic solidly to the client side. This is a fundamental change: one where the client-side is free to change and reconfigure the UI in any way it likes, and is ultimately responsible for packaging up the completed data and sending it to the server.

When you are used to the BeanEditForm component, this might feel like a step backwards, as you end up responsible for writing a bit more code (in JavaScript) to implement the user interface, input validations, and relationships between fields. However, as fun as BeanEditForm is, the declarative approach to validation on the client and the server has proven to be limited and limiting, especially in the face of cross-field relationships. We could attempt to extend the declarative nature, introducing rules or even scripting languages to establish the relationships ... or we could move in a situation that puts the developer back in the driver's seat.

Further, there are some that will be concerned that this is a violation of the DRY pricipal; however I subscribe to different philosophy that client-side and server-side validation are fundamentally different in any case; this is discussed in an excellent blog post by Ian Bickling.

Certainly there will be components and services to assist with this process, in term of extracting data into JSON format, and converting JSON data into a set of updates to the server-side objects. There's also a number of security concerns that necessitate careful validation of what comes up from the client in the Ajax request. Further, there will be new bundled libraries to make it easier to build these dynamic user interfaces.

Conclusion

In this vision of Tapestry's future, the server-side framework starts to shift from the focus of all behavior to the facilitator: it paints the broad stokes on the server, but the key interactions end up working exclusively on the client.

I'm sure this view will be controversial: after all, on the surface, what the community really wants is just "jQuery instead of Prototype". However, all of the factors described in the above sections are, I feel, critical to keeping Tapestry relevant by embracing the client-side in the way that the client-side demands.

I think this change in focus is a big deal; I think it is also necessary for Tapestry to stay relevant in the medium to long term. I've heard from many individual developers (not necessarily Tapestry users) that what they really want is "just jQuery and a restful API"; I think Tapestry can be that restful API, but by leveraging many of Tapestry's other strengths, it can be a lot more. Building something right on the metal feels empowering ... until you hit all the infrastructure that Tapestry provides, including best-of-class exception reporting, on-the-fly JavaScript aggregation and minimization, and (of course) live class reloading during development. java

I'm eager to bring Tapestry to the forfront of web application development ... and to deliver it fast! Monitor the Tapestry developer mailing list to see how this all plays out.

Better Namespacing in JavaScript

In my previous post, I discussed some upcoming changes in Tapestry's client-side JavaScript. Here we're going to dive a little deep on an important part of the overall package: using namespaces to keep client-side JavaScript from conflicting.

I'm not claiming to originate these ideas; they have been in use, in some variations, for several years on pages throughout the web.

Much as with Tapestry's Java code, it is high time that there is a distinction between public JavaScript functions and private, internal functions. I've come to embrace modular JavaScript namespacing.

One of the challenges of JavaScript is namespacing: unless you go to some measures, every var and function you define gets attached to the global window object. This can lead to name collisions ... hilarity ensues.

How do you avoid naming collisions? In Java you use packages ... but JavaScript doesn't have those. Instead, we define JavaScript objects to contain the variables and functions. Here's an example from Tapestry's built-in library:

Tapestry = {

  FORM_VALIDATE_EVENT : "tapestry:formvalidate",

  onDOMLoaded : function(callback) {
    document.observe("dom:loaded", callback);
  },

  ajaxRequest : function(url, options) {
    ...
  }, 

  ...
};

Obviously, just an edited excerpt ... but even here you can see the clumsy prototype for an abstraction layer. The limitation with this technique is two fold:

• Everything is public and visible. There's no private modifier, no way to hide things.
• You can't rely on using this to reference other properties in the same object, at least not inside event handler methods (where this is often the window object, rather than what you'd expect).

These problems can be addressed using a key feature of JavaScript: functions can have embedded variable and functions that are only visible inside that function. We can start to recode Tapestry as follows:

Tapestry = { 
    FORM_VALIDATE_EVENT : "tapestry:formvalidate"
};

function initializeTapestry() {
  var aPrivateVariable = 0;

  function aPrivateFunction() { }

  Tapestry.onDOMLoaded = function(callback) {
      document.observe("dom:loaded", callback);
  };

  Tapestry.ajaxRequest = function(url, options) {
    ...
  };
}

initializeTapestry();

Due to the rules of JavaScript closures, aPrivateVariable and aPrivateFunction() can be referenced from the other functions with no need for the this prefix; they are simply values that are in scope. And they are only in scope to functions defined inside the initializeTapestry() function.

Further, there's no longer the normal wierdness with the this keyword. In this style of coding, this is no longer relevant, or used. Event handling functions have access to variables and other functions via scoping rules, not through the this variable, so it no longer matters that this is often not what you'd expect ... and none of the nonsense about binding this back to the expected object that you see in Prototype and elsewhere. Again, this is a more purely functional style of JavaScript programming.

Often you'll see the function definition and evaluation rolled together:

Tapestry = { 
    FORM_VALIDATE_EVENT : "tapestry:formvalidate"
};

(function() {
  var aPrivateVariable = 0;

  function aPrivateFunction() { }

  Tapestry.onDOMLoaded = function(callback) {
      document.observe("dom:loaded", callback);
  };

  Tapestry.ajaxRequest = function(url, options) {
    ...
  };
})();

That's more succinct, but not necessarily more readable. I've been prototyping a modest improvement in TapX, that will likely be migrated over to Tapestry 5.3.

Tapx = {

  extend : function(destination, source) {
    if (Object.isFunction(source))
      source = source();

    Object.extend(destination, source);
  },
  
  extendInitializer : function(source) {
    this.extend(Tapestry.Initializer, source);
  }
}

This function, Tapx.extend() is used to modify an existing namespace object. It is passed a function that returns an object; the function is invoked and the properties of the returned object are copied onto the destintation namespace object (the implementation of extend() is currently based on utilities from Prototype, but that will change). Very commonly, it is Tapestry.Initializer that needs to be extended, to support initialization for a Tapestry component.

Tapx.extendInitializer(function() {

  function doAnimate(element) {
    ...
  }

  function animateRevealChildren(element) {
    $(element).addClassName("tx-tree-expanded");

    doAnimate(element);
  }

  function animateHideChildren(element) {
    $(element).removeClassName("tx-tree-expanded");

    doAnimate(element);
  }

  function initializer(spec) {
    ...
  }

  return {
    tapxTreeNode : initializer
  };
});

This time, the function defines internal functions doAnimate(), animateRevealChildren(), animateHideChildren() and initializer(). It bundles up initializer() at the end, exposing it to the rest of the world as Tapestry.Initializer.tapxTreeNode.

This is the pattern going forward as Tapestry's tapestry.js library is rewritten ... but the basic technique is applicable to any JavaScript application where lots of seperate JavaScript files need to be combined together.

Rethinking JavaScript in Tapestry 5.3

I've always had a love/hate relationship with JavaScript; some of the earliest motivations for Tapestry was to "encapsulate that ugly JavaScript stuff so I don't have to worry about it again." However, as I've come to appreciate JavaScript, over time, as a powerful functional language, and not as an incompletely implemented object oriented language, my revulsion for the language has disappeared ... even reversed.

Back around 2006, I started adding the client-side JavaScript features to Tapestry 5; this started with client-side form field validation, and grew to include a number of more sophisticated components. The good news is these features and components are fully encapsulated: they can be used freely throughout at Tapestry application without even knowing JavaScript. Tapestry includes the libraries (and related CSS documents) as needed, and encapsulates the necessary initialization JavaScript. The APIs for this were revamped a bit in Tapestry 5.2, but the core concept is unchanged.

The bad news is that the client-side is directly linked to Prototype and Scriptaculous (which are bundled right inside the Tapestry JAR file). These were great choices back in 2006, when jQuery was new and undocumented (or so my quite fallible memory serves). It seemed safe to follow Rails. Now, of course, jQuery rules the world. I've been talking for a couple of years about introducing an abstraction layer to break down the Prototype/Scriptaculous dependency; meanwhile I've recently seen that Rails and Grails are themselves moving to jQuery.

However, that abstraction layer is still important; I have clients that like MooTools; I have clients that are using YUI and ExtJS.

Certainly, it would have been too ambitious to try to start with such an abstraction layer from day 1. At the time, I had no real idea what the relationship between JavaScript on the client, and the application running on the server, would look like. Also, my JavaScript skills in 2006 are a fraction of what they are now. With several years of coding complex JavaScript and Ajax components for Tapestry, for TapX, and for clients, I think I have a much better understanding of what the APIs and abstraction layers should look like.

So suddenly, I have a number of goals:

• Allow Tapestry to work on top any JavaScript framework
• Support Prototype/Scriptaculous and jQuery as substrate frameworks "out of the box"
• Make the built-in Tapestry library first class: documented and release-on-release compatible
• Keep backwards compatibility to Tapestry 5.2

What I'm proposing is a gradual transition, over Tapestry 5.3 and 5.4, where new, documented, stable JavaScript APIs are introduced. and Tapestry and 3rd party libraries can code to the new APIs rather than to Prototype/Scriptaculous. The goal is that, eventually, it will be possible to switch the default substrate from Prototype/Scriptaculous over to jQuery.

Rethinking Tapestry's approach to JavaScript

I've been doing a lot of work for a client using the ExtJS library and that, combined with many other things I've been looking at, has started to shift my attitude to the correct approach to client-side JavaScript. Not a sea change, just a minor adjustment.

Until now, I've had the approach that the page should popup "complete" and that the JavaScript should only manipulate (or add to) the content already present. In most cases, such as adding validations to user input fields, that works fine: you write out the HTML for the field, remember the id for the field, then add JavaScript that adds event handlers to the field, finding it by its id.

However, for more complex cases, such as Tapestry's Palette component, I've been coding like its 2001 for too long.

The Palette component renders out two <select> elements, plus a number of <divs>, a few buttons, and a chunk of JavaScript to connect it all together. This means generating a lot of related ids on the server (to match against the generated HTML markup) and passing those down to the client.

It's effective but it reflects my relative naivete with JavaScript back in 2001. It's now so much easier to create a DOM structure on the client, using any of the major JavaScript libraries. That, in turn, makes it much more reasonable to just write out an empty <div> element with an id and install into that empty space all the markup for the component, generated on the client side. In fact, I envision a version of the Palette for Tapestry 5.2 that starts as a <select> configured for multiple selection and is converted on the fly to the familiar Palette component entirely on the client side ... which means that it could actually operate in a functional, if limited way, even if JavaScript is disabled in the client.

ExtJS includes a number of other ideas that are worth thinking about; I've been very impressed by the concept of a separate controller hierarchy on the client side (ExtJS calls these "components" as differentiated from the DOM nodes they create and manage). That's something that's ad-hoc and inconsistent in Tapestry's client-side library. I think embracing a similar, more structured approach could make it easier for Tapestry to embrace even more dynamic Ajax behavior.

Combining JavaScript Libraries

I've been continuing to do work on speeding up Tapestry performance; I think part of the Tapestry message is not just how easy it is to get started, but how the framework grows with you as your application, and your user base, grows.

One aspect of performance is speeding up server side operations, and Tapestry 5.1 has gone a long way down that path. However, there's only so far that can take you, especially since the lion's share of server-side time is spent waiting for database queries.

Another approach is to minimize both the number of requests and the size of the replies. Tapestry 5.1 will GZIP compress replies for dynamic page content and static asset files. That reduces the size of each reply (when the client supports GZIP compression, which all modern browsers do).

What about reducing the number of requests? As with all web applications, the HTML for the page is just the start: there's also requests for images, stylesheets and JavaScript libraries. Tapestry is already good at giving these resources version-numbered URLs and far future expires headers, so that the client browser can cache them aggressively.

Tapestry 5.1 now adds a feature to combine JavaScript libraries. Whatever JavaScript libraries are requested by components when the page renders are combined into a virtual asset: a URL that represents the combination of the individual JavaScript libraries. This virtual asset has an odd URL (there's some Base64 encoded data as part of the file name), but it also has a far-future expires header and may be served to the client GZIP compressed.

This helps a lot with first-time access to an application; page rendering is interrupted only long enough to download a single JavaScript resource, rather than a series of requests for the individual files. Given that JavaScript must be loaded, parsed and executed serially ... and that rendering of the page is on hold until that occurs, combining JavaScript in this way is a pretty big win, especially given the amount of server-side and client-side caching involved.

The best part is, like many features in Tapestry, there's no configuration to worry about. It just works, it's free, there's no special scripts or tasks to run as part of your build. Better yet, it defaults to off for development mode and defaults on for production mode.

Next release, we may try to minimize the JavaScript as well as compress it; minimizing means stripping out comments and unnecessary whitespace and shortening variable and function names. Again, this would happen on-the-fly.

Comparing Prototype and jQuery

Glenn Vanderburg has written a careful treatise on why he prefers Prototype to jQuery. The thing about Glenn is that he doesn't write (or talk, or think) in terms of "feelings" or "impressions"; his thoughts are clear, and organized, and fact based, and detailed.

I looked at jQuery about two years ago; to me it seemed a little iffy, a little skewed towards web developers doing bespoke web page development, rather than the dynamic generation Tapestry is all about. That impression may or may not be accurate, but Glenn has nailed some significant differences in the design and implementation of the two frameworks, which is why Prototype comes out on top.