mu

A response to Paul English's "How to be a great programmer"

2018-07-22T00:00:00+00:00

There are only two things you need to do to become a great, world-class programmer. First, you must maximize customer impact, and second, you must maximize team output.

– Paul English, How to be a great programmer

Paul English asserts there are merely two ingredients to becoming a “great, world-class programmer”. Unfortunately, the article does not deliver on its title; it is not even about programming. The concerns of the article–namely, “maximizing customer impact” and “maximize team performance”–are architectural concerns.

By “architectural” I am not even speaking about application architecture. No, I mean enterprise architecture, i.e. the discipline that deals with the question “What are the business priorities that drive technical decisions?” This kind of architectural thinking relies on defining one’s priorities, and then imposing a strict total order on them.

We can tease out some of English’s architectural priorities and their order:

you must make smart trade-offs of working quickly vs. working thoroughly. […] If you try to be thorough with all tasks … this will result in your being able to do fewer tasks, of which a certain percent will fail. […] the best engineers try to narrow scope of tasks so they can achieve and test them quickly with customers.

This passage highlights the priorities speed to market, the feedback loop, and thoroughness. That is also how English orders their importance.

The trade-off here is that we should expect to see more failures. The expectation is that several small failures are less harmful than one large failure; and that small failures are more easily mitigated by a rapid feedback loop.

A little later, more priorities are added:

you must always write clean, understandable code.

These terms are not defined, and for many programmers these terms are fraught. What is “clean”? “Understandable” to whom? Making “understandibility” a priority runs the risk of reducing the team’s intelligence to the lowest common denominator.

[Clean, understandable code] gives super powers to other engineers who might have to change your code later.

One could argue that the constraint of “clean, understandable” code is in conflict with the priority of “speed to market”. But in this case, “speed to market” is the primary goal, and “understandability” is meant to serve that goal. English prioritizes “understandability” because that enables rapid refactoring, and that helps optimize speed to market.

The dominance of speed to market as an architectural priority is pervasive in the article:

When we try to accomplish tasks quickly, so that we can test many things with customers, the way to save time is not to write crappy code, but instead, to narrow the initial scope of a task.

Of course, one programmer’s “crappy code” is another’s JavaScript.

The priorities here include teamwork, and understandibility/cleanliness, which I take to be a synonym for “refacorability”. Teamwork is higher priority than refactorability, although both are secondary to speed to market.

This struck me as an odd assertion:

Speed is under-appreciated by most programmers. The best engineers always make sure that their code is fast …”

I have not found this to be true. I have seen developers of all levels fetishize performance, trying to squeeze microseconds out of some function without regard for whether such optimization matters.

As a general rule, I feel like you should only optimize code for performance when

you know the code is correct, and
you can prove that the optimization is worth the cost.

Writing performant code is not free, it involves trading off with other priorities. Focusing on performance is almost always in conflict with “refactorability”. For English, in this case as well, performance is necessary for a rapid feedback loop; and the feedback loop helps optimize speed to market:

The best engineers always make sure that their code is fast, resulting in a customer experience of ask-question-get-answer with as little delays as possible between those two things.

The overall ordered list of priorities I gather from this article is:

Speed to market
Feedback loop
Performance
Teamwork
Understandibility/Cleanliness/Refactorability

...

Thoroughness

I have no argument with these being Paul English’s priorities. He certainly knows what works for his business. I do have two basic objections to the article:

I object to the clickbait-and-switch title about becoming a great programmer when that is really not what the article is about at all. Now, one could argue English is really saying that a programmer can increase their value to their employer by being acutely aware of the architectural constraints of the business and using those to guide technical decisions. I would not argue with that at all, that is practically a truism. But that is not what the article says. I would counter that the thesis of the article is explicitly about "greatness", not "value"; and its examples of "greatness" are all bound to particular architectural concerns, especially speed to market, as shown above.
I object to the assertion that "adhering to my priorities" is a recipe for "greatness" in general.

It is not hard to imagine business scenarios where speed to market is not the primary concern, and in fact, could be positively harmful. For example, consider developing a life-or-death medical device. Speed to market with a rapid feedback loop is not a good fit for such a product: “Well, 60% of the patients died with v1, but we got version 2 out in two weeks and cut the mortality rate to 48%!” In such a case, thoroughness, accuracy, correctness would (I hope) be higher priorities than speed to market.

The tacit assumption in English’s definition is that “greatness” in programming may only occur in a business context. A more apt title for the article might be “How to be a great employee for Paul English”. It is also not hard to imagine a notion of “great programmer” outside of a business context. But that would be a whole other article.

No, Promise is not a monad

2018-04-10T00:00:00+00:00

If you want to see how sloppy your thinking is, try writing. If you want to see how sloppy your writing is, try writing math.
– Someone said this

“Monad” is a well-defined mathematical object that must satisfy axioms known as the monad laws: Left Identity, Right Identity, and Associativity. One way to implement a monad is to define the functions return and bind such that they obey the monad laws. (And once you have that, you can derive other functions, as well.)

return :: a -> m a is the “unit” operation for the monad m. Essentially, return takes any value and puts it into the monadic context. The candidate for an analogous function on the Promise API is resolve. However, because of the OO nature of JavaScript we can’t use Promise.resolve as a pure function. I cannot simply write .then(Promise.resolve) – that gives me this delightful error: “Receiver of Promise.resolve call is not a non-null object”. So to be fair, we can use the lambda x => Promise.resolve(x) as our return.

bind :: m a -> (a -> m b) -> m b enables you to compose, flattening as you go. The analogous Promise function for bind is then.

Note that in the examples below, the ≡ symbol in an expression like a ≡ b indicates that you can confidently replace one side of ≡ with the other side, and your program will behave the same way.

The question is: Does Promise satisfy the monad laws?

Left Identity

Promise.resolve(a).then(f) ≡ f(a)

Left Identity appears to hold as long as the function f that takes an a and returns a Promise (i.e. a -> Promise b). But not quite. What if the type a is itself a Promise? Suppose f is

const f = p => p.then(n => n * 2)

In this example f is a function from Promise Number -> Promise Number. Let’s plug in a Promise Number for a to match up with the expected input for f. Then we can evaluate the expression:

const a = Promise.resolve(1)
const output = Promise.resolve(a).then(f)    
// output :: RejectedPromise TypeError: p.then is not a function

The value that gets passed to f is implicitly unwrapped. It is a TypeError to pass a function to then that takes a Promise for an argument. Note that we can evaluate the right-hand side of the equivalence to get a different result:

const output = f(a)  
// output :: ResolvedPromise 2

Therefore, Left Identity does not hold for every function of the form a -> Promise b: It only holds if a is not itself a Promise.

Right Identity

p.then(x => Promise.resolve(x)) ≡ p

This holds (where p is some Promise).

Associativity

p.then(f).then(g) ≡ p.then(x => f(x).then(g))

As with Left Identity above, Associativity is only satisifed if f and g play nice. If f or g take a Promise argument, then, for the same reason as shown above, this law will not hold.

Conclusion

Failing to satisfy one law would have been sufficient to disqualify Promise from being a monad. The examples above show that Promise satisfies only one out of three: Right Identity.

Is this a problem?

Laws matter. They can give you confidence about the properties of the code you write, and how you can rewrite it. They permit you to port concepts across languages – to the extent the language can support them. On the other hand, if you are using JavaScript, then lawfulness may not be a priority.

Chain, chain, chain

2016-12-29T00:00:00+00:00

Consider this problem, from a question on stack overflow. Given a list x::xs, how can one write a function f that will output x::xs::x? In other words, write a function that takes the head of the list and appends it to the end of the list?

The description above almost writes the function for us. Here are the component parts:

Take the head of the list: We have a function for that: head. Note the type of head:

head :: [x] -> x

That signature simply says that when you give head a list, it will return the first element of that list.
Append it to the end of the list: We have a function for that as well: append. The type of append is:

append :: x -> [x] -> [x]

When you give append an element and a list, it gives you back a new list with the element at the end.

What will our function f look like?

It will be of the type [x] -> [x]
It will be composed of the parts above, viz. head :: [x] -> x and append :: x -> [x] -> [x].
Somehow we have to get the output of head fed into append to get us a new function [x] -> [x].

Before I pull a rabbit out of this hat, a few words about chain:

chain :: Chain m => (a -> m b) -> m a -> m b

Let’s break this signature down into bite-sized pieces and digest them one-by-one.

Chain m: There is some thing m that is a Chain.
=>: This tells us that whenever we see an m in the rest of the singature, we know it is a Chain thing.
(a -> m b): The first argument to chain is a function from some type a to a Chain that “contains” a value of type b.
m a: The second argument to chain is a Chain wrapping a value of type a.
m b: This is our return type. chain will return a Chain wrapping a value b.

Here is an example of chain. It is often called flatMap:

    const dup = x => [x, x];
    chain(dup, [1, 2, 3]); //=> [1, 1, 2, 2, 3, 3]

In this example,

(a -> m b) is the function dup, a is Number, and the Chain m b is an Array of Numbers.
The Chain m a is an Array of Numbers.
The returned value m b is the same type as m a: A Chain of an Array of Numbers.
Note that in this example, both type variables a and b refer to the same type: an Array of Numbers. Likewise, m a and m b both refer to a Chain of an Array of Numbers. This need not always be the case, but it is also not a problem when it is the case.

It should be pretty clear how chain works in the context of Array of Something. In this case, though, we want to output a Function f. Is Function a Chain?

Recall the signature:

    chain :: Chain m => (a -> m b) -> m a -> m b

Replace Chain m => m with Function x:

    chain :: (a -> Function x b) -> Function x a -> Function x b

You can look at a function from x to a as a wrapper around its return value a. It is analogous to an Array wrapping its elements.

Now use x -> y as sugar for Function x y:

    chain :: (a -> x -> b) -> (x -> a) -> (x -> b)

Adding an extra pair parenthesis to make this clearer:

    chain ::      (a -> (x -> b)) -> (x -> a) -> (x -> b)
                         ^^^^^^       ^^^^^^      ^^^^^^
    // analogous to:       m b          m a         m b
    // analogous to:       [b]          [a]         [b]

That looks like it should work! Let’s compare this chain signature with the components we are going to use to build f:

f :: [x] -> [x]
head :: [x] -> x
append :: x -> [x] -> [x]

append is of type a -> m b; head is of type m a. Recall that what we are trying to produce is a function f of type m b. Now our types line up perfectly:

    const f = chain(append, head); //=> f :: [x] -> [x]`
    f([1, 2, 3]); //=> [1, 2, 3, 1]

But wait – there’s more! Does this mean that you can do this with any function that satisfies those signatures? That’s exactly what it means. For example, you could chain toLower and concat over a String:

    chain(concat, toLower)("ABCD"); //=> "abcdABCD"

… or chain assoc(keyname) and keys over an Object:

    chain(R.assoc('keylist'), keys)({a: 1, b: 2, c: 3}); //=> {a: 1, b: 2, c: 3, keylist: ['a', 'b', 'c']}

… and so on.

You may be surprised to learn that was a demonstration of how Function is a Monad. Proving that – by defining join and return for Function and showing that it satisfies associativity, left identity, and right identity is left as an exercise for the reader.

Exceptional composition

2016-02-05T00:00:00+00:00

Lately I’ve been studying OCaml. It’s a very nice language. In grad school I worked with Standard ML a bit, but never got a good feel for it. OCaml feels very comfortable, although I am still learning to write idiomatically.

OCaml will let you be as pure as you need to be, but will also let you wallow in the gutter of mutable variables if you need to. It is less judgmental than some other languages.

One of these compromises is in how OCaml uses exceptions. That got me thinking about handling exceptions in ramda. For example, suppose you have a list of a billion integers, and you need to get a product over the whole list:

    (* product : int list -> int *)
    let rec product = function
      | []    -> 1
      | x::xs -> x * (product xs)

This is a bit naive. The biggest problem is what if I encounter a zero? At that point I know that the whole product will be zero. I can stop multiplying. Adding an if check doesn’t quite solve the problem:

      | x::xs -> if x = 0 then 0 else x * (product xs)

What if zero is the last integer in the list? I have built up nearly a billion recursive calls in my stack that I still have to unwind. This is the kind of grimy, real-world consideration that some functional programmers like to sweep under the rug.

OCaml lets you solve this very nicely using exceptions. In the example above I could define an exception Zero and raise it if I encounter a zero in my list:

    exception Zero

    (* product : int list -> int *)
    let rec product = function
      | []    -> 1
      | x::xs -> if x = 0 then raise Zero else x * (product xs)

Now we have to handle the exception:

    try product billion_ints with Zero -> 0

How nice is that? The exception will short-circuit the evaluation, throw away any stack I have built up, and return me an answer.

Unfortunately, there is nothing in the signature to indicate that this function might raise an exception:

    val product : int list -> int = <fun>

I can get a “safe” function by wrapping up product inside a function that will trap the raised exception:

    let safe_product ints = try product ints with Zero -> 0

I thought this was really nice syntax, and got me thinking that we could use a function in ramda that would wrap exceptions and let the composition keep rolling along. One approach to doing that is to use a data type like Maybe or Either to wrap up the value. That is fine for some cases, but it means you will be map-ping the rest of the way down your composition. That may be overdoing it sometimes.

So I added a simple function tryCatch:

    tryCatch : (a -> b) -> ((e, a) -> b) -> a -> b

tryCatch evaluates the first function (i.e. a -> b). If it does not throw, it returns b. If it does throw, then the second function catches the exception. It is evaluated with the thrown Error and the original arguments, and must return something of type b. Et voila! You can compose with exceptions.

The usual JavaScript caveats apply; The second function must return the same type as the first function. It is up to the user to enforce this, since JavaScript will offer no help in this regard. And, of course, the syntax is nowhere near as nice as OCaml’s. And it’s unlikely there is any performance benefit to this approach in JavaScript, but there definitely is an advantage in OCaml.

On the other hand, exceptions are a fact of life in JavaScript. So it feels right that ramda should provide some way to deal with them.

tryCatch will be introduced in ramda 0.20.

Is JavaScript getting worse?

2015-01-08T00:00:00+00:00

Any headline which ends in a question mark can be answered by the word “no.”
– Betteridge’s law of headlines

There is a lot of anticipation about the next release of JavaScript, ES6. There are certainly going to be some very nice features:

But JavaScript is not content merely to improve. It has to maintain – and even increase – its lead in WTF’s per minute.

Classes. Whoop-dee-freakin’-doo. Very hard to get excited about quadrupling down on a bad bet.

Default parameters. By itself, this is another “so what” feature. But default parameters have an obnoxious side effect: They are not reflected in the function’s length property. This makes currying tricky indeed. With Ramda you could hack around this infirmity by specifying arity when currying with curryN, but geez. C’mon, man.

let considered harmful The problem with let is that it is not “hoisted” to the top of its block, as var is with its containing function. So if you:

      (function() {
         "use strict";
         console.log(x);
         let x = 1;
      }())

you will get “ReferenceError: can’t access lexical declaration ‘x’ before initialization.” This means that typeof can now throw. It’s like the let keyword has inadvertently introduced a new kind of nil value to a language that already has at least one too many.

So is Javascript getting worse? Well … no. But that doesn’t mean it’s only getting better.

Functors and Applicatives

2014-10-26T00:00:00+00:00

I recently gave a talk at Hartford JS on Functors and Applicatives in JavaScript. This subject assumes familiarity with currying and function composition. You can also look over the original slide deck.

Q: When is a function not a function?

A: When it is a method.

We’re talking here about functions and not methods. For the purposes of this discussion, a function:

Maps some input a to some output b;
For any input a, there is only one output b (although any b may be mapped to by more than one a);
Does not mutate any state, encapsulated or otherwise, nor does it produce side effects.

Contrast this with a method:

May return something, or not;
May return a different b for an input a;
May mutate state somewhere. May produce side effects.

Let’s look at a simple example function. You want to take your cat to the vet to get a shot for it.

A function from Cat → Cat

Here we have a simple function from Cat to Cat. We input a cat a on the left, and we get out an immunized cat b on the right. It’s important to note that we should not mutate the input cat. What should happen instead is we input cat a on the left, and get out a brand new cat that is immunized. Then cat a can be made available for garbage collection.

Cat ready for garbage collection

Of course, there’s no guarantee that people coming into the vet’s office have a cat–or any pet at all, for that matter. If someone brings a null where we are expecting a Cat, then our nice function from Cat to Cat doesn’t work so well:

A function from ~~Cat → Cat~~ null → Shit

We could have the vet ask everything that comes by if it is null or not. But that is not his job, really. His job is giving shots to pets. I don’t want all of my functions littered with null checks. Surely, there must be some way to keep my functions clean and simple, yet make them null-safe as well?

The answer is “Maybe”–more specifically, the “Maybe Functor”.

Note that when you are composing functions, all the functions in the composition have to fulfill their contracts. If somebody returns a null in there, the whole daisy chain is borked. Consider:

    // capitalize :: String -> String
    var capitalize = R.compose(
        R.toUpperCase, // String -> String
        R.prop('textContent'), // Object -> String
        R.bind(document.getElementById, document) // String -> DOMElement
    );

What happens if the call to getElementById doesn’t find anything? You get a null back. Then that null gets fed to R.prop, and R.prop will fail.

To get around this problem, we can use a Functor.

The same function Cat → Cat, applied by mapping

A Functor is an object that can be mapped over. By “map” I mean that we can use the same function from Cat to Cat even though the Cat is now inside a container. Note that in the example above, The cat is in the kitty carrier, but we we are still using the same immunization function.

This is a big deal, because it allows us keep the null-checking separate from the internal logic of the function. Now if we get an empty container for input, then there is no reason to even try to apply the function to its content. We simply get an empty container back.

The same function Cat → Cat, now null safe!

This example is essentially the “Maybe” functor. “Maybe” let’s you wrap up a function from a to b so that you can apply it to the contents of the Maybe. Take a look at how we have implemented Maybe in Ramda Fantasy. You can see that Maybe is a sum type; it is a Just of some value, or it is Nothing. You can use the direct your composition into the proper channel with something like:

    function fromNullable(x) {
        return x == null ? Maybe.Nothing : Maybe.Just(x);
    }

Now let’s go back to the capitalize example. We can rewrite that to make use of the Maybe functor:

    // capitalize :: String -> Maybe String
    var capitalize = R.compose(
        R.map(R.toUpperCase), // Maybe String -> Maybe String
        R.map(R.prop('textContent')), // Maybe Object -> Maybe String
        fromNullable,  // Object -> Maybe Object
        R.bind(document.getElementById, document) // String -> DOMElement
    );

Now this function is guaranteed to output something–Maybe. If the call to getElementById fails, we’ll wind up with a Maybe null at the end; otherwise we’ll have a Maybe String. We didn’t have to guard inside our functions, so that is nice. Instead, we have wrapped up the value we are composing over inside this container. So we can only access the value by mapping over the container.

(Well, actually, since this is JavaScript, and value is sitting there as a public property on Maybe, you could just query it. But then you are right back where you started, needing to test for null.)

The Maybe functor only gives us one bit of information: Did we hit a null somewhere along the way or not? There are other functors that can wrap up more info. The Either functor is either a Left or a Right; In the Left you can put an error message (or a function, or whatever). If it is a Right, then everything ran fine.

Promises can also be looked at as functors. Consider:

    R.map(function(a) { return a + 1; }, Promise(2)); //=> Promise(3)

There are many, many more ways to skin this cat: Validation, IO, Reader, Writer, State, Future … and of course Array.

    R.map(function(a) { return a + 1; }, [10, 20 ,30]); //=> [11, 21, 31]

Arrays are Functors. After all, it’s a container you can map over. Arrays are a bit trickier, though, since they are intertwined with the notion of iteration.

No reason to stop here, though. We can wrap up the object we’re mapping over in a Functor. We can also wrap up the function we want to apply inside a container:

Applying a wrapped function to a wrapped value

Once again, it’s the same function from Cat to Cat, but now both function and data are wrapped in containers. The good folks from Fantasy Land call an object that can handle this situation an “Apply”, and the function you use to apply a wrapped function to a wrapped value is ap. For Maybe, that could look like this:

    Maybe.prototype.ap = function(vs) {
      return (typeof this.value !== 'function') ? new Maybe(null) : vs.map(this.value);
    };

The implementation of ap for Array is a bit more complicated, of course, because of iteration:

    R.ap = curry(function(fns, vs) {
      return foldl(function(acc, fn) {
        return concat(acc, map(fn, vs));
      }, [], fns);
    });

Maybe you are asking yourself, “How do you get into a situation where you have a function inside a container?” I covered such a situation recently. That article also covers the ability to create an Applicative, by wrapping up an arbitrary object inside a container via of:

`of` wraps up something in a container

Here’s how we implemented of for Maybe:

    Maybe.of = function(x) {
      return new Maybe.Just(x);
    };

And the implementation for Array is trivial:

    R.of = function(x) { return [x]; };

Please read my article on Applicatives for a real-world example of these concepts in action. Gleb Bahmutov has also written a good article on using Applicative Functors with Promises that goes more in depth in the code than this article did. And I also should mention the excellent illustrated article Functors, Applicatives, And Monads In Pictures, by Aditya Y. Bhargava.

Thanks to J. C. Phillipps for the illustrations. No animals were harmed in the writing of this blog post.

It turns out that JavaScript is not Haskell

2014-10-02T00:00:00+00:00

We were pretty excited about one feature of Ramda v0.5.0. I even wrote a blog post about it.

The problem we thought we were solving was: How do you curry infix operators in prefix form, when the operation is not commutative? Haskell solves this problem very elegantly. For example, exponentiation. Haskell uses the caret symbol (^) for this operation. You can use it infix as you’d expect:

    λ 2^10
        1024
        :: Num a => a

Converting to prefix and currying is beatiful. If you want a function to calculate e.g. 2^n:

    λ (2^) 
        :: (Integral b, Num a) => b -> a
    λ (2^) 10
        1024
        :: Num a => a

And if you want a function for n^2:

    λ (^2) 
        :: Num a => a -> a
    λ (^2) 10
        100
        :: Num a => a

That’s what we wanted; that’s what we made ourselves believe we were getting. But it turns out–SPOILER ALERT–Javascript is not Haskell. What we got instead was confusing and inconsistent. David Chambers, a regular, and very valuable Ramda contributor, observed:

This is very surprising:

R.divide(10, 2) //=> 5

R.divide(10)(2) //=> 0.2

Uh-oh. Then J. A. Forbes pointed out:

Why is it that flipping twice removes the inconsistency?

R.divide = R.compose(R.flip,R.flip)(R.divide)

R.divide(10,2) //=> 5

R.divide(10)(2) //=> 5

This cat sums up the process pretty well. Started with so much optimism …

Clearly, our shiny, new implementation of op–a way to curry operators in a left-section/right-section fashion–was not only confusing, but inconsistent. It was holed below the waterline and sinking fast.

So, my apologies. I really thought we had something cool there. But it wound up being a dud.

We have learned from it, however. In the master branch of Ramda now, I have re-implemented op with consistent behavior and semantics, essentially using a “placeholder” approach, e.g:

    R.divide = R.op(function(a, b) { return a / b; });
    
    var half = R.divide(__, 2); // `__` here is some undefined value
    half(100) //=> 50
    
    var reciprocal = divide(1);
    reciprocal(4) //=> 0.25

But we may not stop there. This has us thinking we can have a generalized placeholder approach to currying. Early experiments are promising. I’ll try to keep you posted.

The (pre)fix is in(fix)

2014-09-14T00:00:00+00:00

***UPDATE***

Turns out this didn't work well! It was both confusing to users and inconsistent in its behavior. Read the next post "It turns out that JavaScript is not Haskell to see what we're doing about it.

For Ramda we’ve wrapped several JavaScript infix operators as functions, i.e. in prefix form. You gotta do that if you want to curry and compose with those operations. Some of the time, that works out fine. add, for example, converts to prefix form with no confusion:

    R.add(10, 3) // 10 + 3
    var add10 = R.add(10);
    add10(3); // 13, duh

add converts to infix easily because it is commutative: a + b == b + a. Infix operators that are not commutative convert awkwardly. Consider divide:

    R.divide(10, 2) // 10 / 2
    var divide10 = R.divide(10); // uh ... wait a second ...
    divide10(2); // 5 ... something smells funny ...

Here the order of arguments matters, and the semantics start to wobble. add(10) clearly means “If you give me a number, I will add 10 to it.” divide(10) on the other hand, seems to be saying “If you give me a number, I will divide it by 10.” But that is not what it means. What divide(10) means is “if you give me a number, I will divide that number into 10.”

The partially applied version of divide is also probably not very useful. How often do you store a number so you can divide it by a bunch of other numbers? I’d guess not very frequently. It’s typically more useful to curry in the divisor of this operation, not the dividend.

We noticied this peculiar property of the non-commutative operators converted to prefix functions pretty early on. It always rankled us, but we didn’t have a good solution for it. To get around it, we defined functions like divideBy. divideBy was a flipped version of divide; so when you partially applied divideBy it would read more like what the behavior is:

    R.divideBy(2, 10) // 10 / 2
    var divideBy2 = R.divideBy(2);
    divideBy2(10); // 5, duh

… but that always felt like a hack. Why should this one operation take two functions to express?

This behavior was especially obnoxious with curried relational operators, viz. <, <=, >, >=. (Equality operators are commutative, so those were cool.)

For example, consider < (less than) converted to the prefix function lt:

    R.lt(2, 10) // 2 < 10
    var lt2 = R.lt(2); // oh no
    lt2(10); // true. wtf?

We couldn’t hack around these by flipping them and giving them separate names, because the flipped versions have the opposite meaning, e.g. R.flip(R.lt) == R.gte. The cure is worse than the disease.

Scott posted this problem as a desparate plea on Stack Exchange, and got a spectacular answer and solution. Not surprisingly, the insight comes from Haskell. I recommend you read the response by Aadit M. Shah, it’s well worth a read. Go ahead, I’ll wait.

The essence of the solution is to treat infix operators differently. Instead of currying them from left to right, they are curried in the following manner:

When given 1 argument, that argument is applied right. So lt(10) returns a function like x -> x < 10
When given 2 arguments, but the second one is undefined, then it is applied left, i.e lt(10, undefined) :: x -> 10 < x.
When given 2 arguments and both are defined, evaluate the function and return the result.

This approach threads the needle of converting from infix to prefix while maintaining (improving) readability. An added bonus: we can get rid of all the flipped/renamed functions, so the API footprint is a little smaller. Win-win-win.

This distinction in the way we are treating infix operators is coming up in ramda v0.5.0. We will also be exposing an op function on ramda, so you can make your infix-style functions behave this way, should you so desire.

The Tao of λ: Applicatives, Ramda style

2014-08-12T00:00:00+00:00

I’ve been working on making Ramda integrate with Fantasy-land-style objects. Specifically, I added the functions ap and of to Ramda. These functions satisfy the Apply and Applicative specifications, respectively.

of is so trivial it appears almost useless. It simply wraps its argument in an array:

    R.of = function(x) { return [x]; };

(The actual implementation of of in Ramda checks if you have also passed in a Fantasy-land-style object with an of method, and dispatches to that method–but that is beside the point for this discussion.)

ap on the other hand, seems so obscure that at first glance, it’s difficult even to see how you would use it. In Ramda, ap takes a list of functions (fns), and a list of arguments (args) to apply the list of functions to. It returns an array of results from applying each function to each argument. The output of ap is like a cross-product of the fns by the args, with a length of fns.length * args.length:

    R.ap([R.multiply(2), R.add(3)], [1,2,3]); // => [2, 4, 6, 4, 5, 6]
    R.ap([R.multiply(2), R.add(3), R.subtractN(2)], [1,2,3]); 
        // => [2, 4, 6, 4, 5, 6, -1, 0, 1]

I added these functions to Ramda on faith; that is, even though I did not clearly see real-world use cases for them, I trusted that these functions are useful. After all, ap and of can both be found in Haskell (of is called pure in Haskell). And those Haskell folks are pretty sharp.

Then Thomas Deutsch left a comment on Scott’s blog post Favoring Curry. He was working on a function hasAllTags that takes a list of letters and returns true if the passed in list contains all of the letters in an array inside the function. In his posted example of hasAllTags, the internal list is ['a', 'd']:

    hasAllTags(['a', 'e', 'd']); // => true
    hasAllTags(['a', 'e']); // => false

Mr. Deutsch’s question: “How is it possible to construct this function only by composition or currying?”

Good question! Let’s take apart what this function is supposed to do:

Take a list of letters
For each letter in the internal list (['a', 'd']), see if the passed in list contains that letter
If all the internal letters are found, return true, otherwise false

This is very close to contains. The difference is that we need to apply contains repeatedly, with different arguments, viz.:

    contains('a', list);
    contains('d', list);

And somehow we need to and all of those results together to see if the whole expression is true. So we want our composition to:

Take a list of letters
pass it to contains once for each internal letter
evaluate all the results of all the calls to contains
output true or false based on #3

Here we have a real use case for ap: I can take my input list and apply a list of functions to it! In this case, I am going to apply a list of contains partially applied to each letter in the internal list. I generate that list of functions by mapping over the internal letters and partially applying each one into contains. Ramda’s auto-curried functions makes this easy:

    R.map(R.contains, ['a', 'd'])
        //=> returns a list of partially-applied `contains` functions.

Then I take the list of functions from that mapping, and pass them to ap. Of course, ap is also curried. I am giving just the list of functions to it:

    R.ap(R.map(R.contains, ['a', 'd'])) //=> Function ::[Args] -> [Bools]

This is the guts of the composition. Given just the list of functions, ap returns a function that is waiting for a list of arguments to apply its list of functions to. And then ap will return an array of results of those applications. In this case, it will be an array of booleans.

The final step in the composition is to reduce that output array of booleans from ap to a single boolean. Simplest thing to do is to pass the output from ap to all, partially applied to the Identity function:

    R.compose(R.all(R.I), R.ap(R.map(R.contains, ['a', 'd'])))

All that remains to do is to prepare the original list to be input to this composition. We can’t just pass a list of letters in. Recall that ap will apply its array of functions to each one of its list of arguments. If we passed in the array ['a', 'b', 'c', 'd'], ap would call each function four times! Worse than that, we have partially applied contains inside ap, and those functions are expecting an array, not a string.

In other words, we want to pass the input array inside the arguments list to ap. The solution is to wrap the input in an array. That is exactly what of does. So the complete composition of the hasAllTags function is:

    hasAllTags = R.compose(R.all(R.I), R.ap(R.map(R.contains, ['a', 'd'])), R.of);

    hasAllTags(['a', 'e', 'd']) // => true
    hasAllTags(['a', 'e'])      // => false

Hey, those Haskell folks are pretty sharp!

You can find ap and of and many more fun functions in Ramda 0.3.0. Please take it for a spin.

Haiku Review: Learn You a Haskell For Great Good

2014-08-10T00:00:00+00:00

Pure and elegant
See the beautiful patterns…
Monads? WTF?

Constraint notation revisited

2014-07-18T00:00:00+00:00

A little while ago I wrote a post about a where function I was adding to Ramda. The function takes two objects, a spec and a test object, and returns true if the test object satisfies the spec. Since where is curried, it is great for generating predicates; and in context, e.g. with a find or filter, it produces code that reads well. Here’s a toy example to illustrate what I mean:

    function isDivBy3(x) { return x % 3 === 0; }
    R.find(R.where({x: isDivBy3}), listOfObjects);

The Ramda where function was inspired by Lodash’s object constraint notation (e.g., _.find({x: 10})), but the two functions are not the same. Lodash’s does some things Ramda’s doesn’t, and Ramda’s does some things Lodash’s doesn’t.

I used where in my examples when I introduced Ramda, and Scott also used where in his examples in Why Ramda? Then we got a bit of a shock. John-David Dalton (of Lodash fame) pointed out that Ramda where was dog slow (Caution: The graphs at this link are fuuuuuuugly).

The good news is that we were able, after much struggle, to achieve parity with Lodash’s performance. This warrants a major hat-tip to Dalton for the heads up, and again to Dalton and Mathias Bynens for Benchmark.js and jsperf.com which made these improvements possible.

Now for the bad news. We had to turn the implementation of where from this artful, compact code (if I do say so myself):

    where = curry(function(spec, test) {
        return all(function(key) {
           var val = spec[key];
           return (typeof val === 'function') ? 
               val(test[key], test) : 
               (test[key] === val);
           }, keys(spec));
    });

… to this horrific imperative monstrosity:

    function satisfiesSpec(spec, parsedSpec, testObj) {
        if (spec === testObj) { return true; }
        if (testObj == null) { return false; }
        parsedSpec.fn = parsedSpec.fn || [];
        parsedSpec.obj = parsedSpec.obj || [];
        var key, val, i = -1, fnLen = parsedSpec.fn.length, 
            j = -1, objLen = parsedSpec.obj.length;
        while (++i < fnLen) {
            key = parsedSpec.fn[i];
            val = spec[key];
            if (!(key in testObj)) {
                return false;
            }
            if (!val(testObj[key], testObj)) {
                return false;
            }
        }
        while (++j < objLen) {
            key = parsedSpec.obj[j];
            if (spec[key] !== testObj[key]) {
                return false;
            }
        }
        return true;
    }

    where = function where(spec, testObj) {
        var parsedSpec = R.partition(function(key) {
            return typeof spec[key] === "function" ? "fn" : "obj";
        }, keys(spec));
        switch (arguments.length) {
            case 0: throw NO_ARGS_EXCEPTION;
            case 1:
                return function(testObj) {
                    return satisfiesSpec(spec, parsedSpec, testObj);
                };
        }
        return satisfiesSpec(spec, parsedSpec, testObj);
    };

Two while loops? Ugh. The primary problem with the original where’s performance was that it was parsing the spec on every call. Parsing that object is expensive. The revised, performant version does the parsing once.

And now, the irony: I started writing Ramda with Scott because I wanted to write Javascript in the elegant, terse functional style. Now it turns out that to do that I have to write some really inelegant, prolix code.

Introducing Ramda

2014-05-16T00:00:00+00:00

For the past year plus, my colleague Scott Sauyet and I have been working in our free time on Ramda, “a practical functional library for Javascript programmers.” When we signed up for Frontend Masters “Hardcore Functional Programming with Javascript” workshop, we were surprised to learn that they had selected Ramda to illustrate their examples. With that vote of confidence, we figured it is time to announce the arrival of Ramda.

There are already some excellent libraries with a functional flavor, such as Underscore and Lodash. Ramda includes all of the favorite list-manipulation functions you expect, e.g. map, filter, reduce, find, etc. But Ramda is significantly different from libraries like Underscore and Lodash. The primary distinguishing features of Ramda are:

Ramda takes the function first, and the data last. Brian Lonsdorf explains why this parameter ordering is a big deal. In a nutshell, the combination of currying and function-first enables the developer to compose functions with very little code (often in a “point-free” fashion), before finally passing in the data. So instead of this:

        // Underscore/Lodash style:
        var validUsersNamedBuzz = function(users) {
          return _.filter(users, function(user) { 
            return user.name === 'Buzz' && _.isEmpty(user.errors); 
          });
        };

… you can do this:

        // Ramda style:
        var validUsersNamedBuzz = R.filter(R.where({name: 'Buzz', errors: R.isEmpty}));

Ramda methods are automatically curried. While you can curry (or partially apply) functions in Underscore and Lodash, Ramda does it for you. Virtually all methods of 2 or more arguments are curried by default in Ramda. For example:

        // `prop` takes two arguments. If I just give it one, I get a function back
        var moo = R.prop('moo');
        // when I call that function with one argument, I get the result.
        var value = moo({moo: 'cow'}); // => 'cow'

This auto-currying makes it easy to compose functions to create new functions. Because the API is function-first, data-last, you can continue composing and composing until you build up the function you need before dropping in the data. (Hugh Jackson published an excellent article describing the advantages of this style.)

    // take an object with an `amount` property
    // add one to it
    // find its remainder when divided by 7
    var amtAdd1Mod7 = R.compose(R.moduloBy(7), R.add(1), R.prop('amount'));

    // we can use that as is:
    amtAdd1Mod7({amount: 17}); // => 4
    amtAdd1Mod7({amount: 987}); // => 1
    amtAdd1Mod7({amount: 68}); // => 6
    // etc. 
    
    // But we can also use our composed function on a list of objects, e.g. to `map`:
    var amountObjects = [
      {amount: 903}, {amount: 2875654}, {amount: 6}
    ]
    R.map(amtAdd1Mod7, amountObjects); // => [1, 6, 0]

    // of course, `map` is also curried, so you can generate a new function 
    // using `amtAdd1Mod7` that will wait for a list of "amountObjects" to 
    // get passed in:
    var amountsToValue = map(amtAdd1Mod7);
    amountsToValue(amountObjects); // => [1, 6, 0]

Ramda is available on npm, so please take it for a spin. And please let us know what you think, and how the library can be improved.

Make Your AngularJS Directive Wait for the DOM

2014-03-20T00:00:00+00:00

Sometimes you want your AngularJS directive to execute after the affected element has been added to the DOM and had its styles computed. This can often be solved by putting your DOM-specific logic inside a $timeout and executing it on the next tick. But the $timeout trick does not guarantee that the element will be ready to manipulate.

I was working with a directive that needed to access the element’s computed width property, using the $window.getComputedStyle method. If the directive fires before the styles are computed, it simply returns “auto” for the width value.

I needed to get some numeric value back (e.g. “421px”); “auto” is not a useful response:

    .directive('example', ['$window', function($window) {
      return {
        restrict: 'A',
        link: function(scope, element, attrs){
          var width = parseInt($window.getComputedStyle(element[0]).width, 10); 
          // Uh-oh, the element is not on the DOM yet. `getComputedStyle` 
          // returns "auto", so `parseInt` evaluates to NaN, and my 
          // calculations are borked
        }
      };
    }]);

In this scenario, wrapping this up in a $timeout was insufficient to solve the problem. getComputedStyle was still returning “auto”, even when I tried passing fudge-factor milliseconds to $timeout. Up to 200ms was still not guaranteed to return the pixel value I needed.

The solution was to use the magic of recursion:

  
        // ...
        link: function(scope, element, attrs){

          function doDomStuff() {
            $timeout(function() {
              var width = parseInt($window.getComputedStyle(element[0]).width, 10); 
             
              if (width) {
                // if width is computed, the element is ready to be manipulated
                // so manipulate the element here, e.g.:
                element.css({ backgroundPositionX: width/2 + "px" });
                // ... and more directive logic ...
              } else {
                // otherwise, the element is not ready, so wait a bit and try again:
                 doDomStuff();
              }
            }, 100);
          }

          doDomStuff();
        }
        // ...

That’s not bad, but we can do better:

Backstop the recursion with a limited number of tries.
Make the timeout milliseconds and number of tries configurable.

Taking those two requirements into account, the directive’s link function winds up looking something like this:

  
        // ...
        link: function(scope, element, attrs){

          function doDomStuff(tries) {
            // a sanity check, just in case we reuse this function as a handler, 
            // e.g. for `orientationchange`
            if (isNaN(+tries)) {
              tries = attrs.maxTries || 10;
            }

            if (tries > 0) {
              $timeout(function() {
                var width = parseInt($window.getComputedStyle(element[0]).width, 10); 
                if (width) {
                  // if width is computed, the element is ready to be manipulated
                  // so manipulate the element here, e.g.:
                  element.css({ backgroundPositionX: width/2 + "px" });
                  // ...
                } else {
                  // otherwise, the element is not ready, so decrement the tries, 
                  // wait a bit, and try again:
                  doDomStuff(tries - 1);
                }
              }, attrs.msDelay || 100);
            } else {
              // if we got here, we've exhausted our tries, so we probably
              // want to log or warn or throw here.
            }
          }

          doDomStuff(attrs.maxTries);
          
          // maybe you need to do this too:
          $window.addEventListener('orientationchange', doDomStuff);
          // ... etc.
        }
        // ...

Et voilá—this directive will now delay running until the element’s computed styles are available, or die trying.

IndexedDB

2014-03-13T00:00:00+00:00

I’ve been given an opportunity to do a little freelance writing for ProgrammableWeb.com. My first article is online now. It’s an overview of the API of the new HTML5 IndexedDB spec.

Read my IndexedDB article at ProgrammableWeb.

AngularJS $http.post unexpected promises

2014-03-07T00:00:00+00:00

Me, today: “Why am I getting two different objects back from an $http.post service call?”

This is a tale of unexpected promises. There were two functions calling two methods on the backend service using an $http.post call. The client-side handlers were written by two different developers. Because the handlers should have been working with the same kind of object, I saw an opportunity for some code reuse. But somehow, the handlers were getting different objects.

Both backend service methods return the same kind of object, something like this:

    // Handler A gets passed this as its first argument:
    {
      status: {
        // ...
      }
      payload: {
        // ...
      }
    };

Handler A received an object just like the one described above. But Handler B got the expected object wrapped inside another object, like this:

    // Handler B gets passed this as its first argument:
    {
      data: { // This is the object I expected
        status: {
          // ...
        },
        payload: {
          // ...
        }
      },
      headers: {
        // ...
      },
      status: 200,
      config: {
        // ...
      }
    }

Both handlers were chained to $http.post calls. Why were they getting different objects?

It turns out that $http.post can return promises with different interfaces, depending on how you attach your handlers to the promise.

The $http.post method returns a Q-style promise. (It’s instructive to look at the Angular q.js source code.) That promise only has the properties then, catch, and finally. Angular’s $http service decorates that promise with two additional properties: success and error. These two extra properties unpack the full response object into its components, and take a function like:

    // note the mapping from params to `then` object properties
    $http.post(url).success(function(data, status, headers, config) { 
      /* etc. */ 
    });

Handler A was relying on these extra success and error methods, and got the expected object in the first parameter.

Handler B attached a then to the $http() call, and got the full $http response object instead. The expected object comes attached to the response object’s data property. Worse, the promise returned from then on $http does not have success and error properties. So anyone trying to add their own success or error handler to the chain will discover they don’t work.

I rewrote Handler B to use success instead of then et voila: Handler B started getting the expected object, I could reuse handler code, and everything was copasetic.

Constraint notation in Javascript

2014-02-14T00:00:00+00:00

Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

– Donald Knuth

One of the things I really like about working with Lo-Dash is its object constraint notation. For example, suppose I want to find an item in a list where x === 10. Lo-Dash lets me express that like this:

    var xIs10 = _.find(list, {x: 10});

I find this syntax easy to write, and at least as important, easy to read. The intent is perfectly clear.

Naturally, I wanted to incorporate this elegant syntax into Ramda. But this would not be an easy fit. Ramda is significantly different from Lo-Dash. Since Ramda emphasizes function composition, the biggest obstacle to integrating the object constraint syntax is that Ramda methods always take the function first and the list last.

I pitched the idea to my co-author, Scott. We didn’t want to rewrite Ramda to type-check the arguments. So we found solution that preserves the composability of Ramda functions, while gaining the readable, expressive object syntax. And then we took it a little bit further….

Here is what the API looks like:

    var xIs10 = find(where({x: 10}), list);

where takes a spec object and a test object and returns true if the test satisfies the spec (false otherwise, of course). Since Ramda automatically curries its methods, when called with one argument (the spec) as in the example above, where returns a function that takes a test object. This makes it a good helper for generating predicates for methods like find and filter.

This would be a fine place to stop, and we could be satisfied with a nice object-to-predicate method. But this where can only express equality relations. If we want to match objects where x > 10, or more complex queries, such as x > 2 AND y > (x * 2) AND y % 2 === 0, then we would have to write a function.

So we opted to take the object constraint notation one step further: If a property on the spec object maps to a function, we use that function as the constraint on that property. Here’s how the examples above would look:

    // example 1: find objects where x > 10
    var xGt10 = filter(where({x: function(x) { return x > 10; }}), list);

    // example 2: find objects where x > 2 && y > x*2  && y is even
    var xGt2etc = find(where({
      x: function(x, obj) { return x > 10; },
      y: function(y, obj) { return y > obj.x * 2 && y % 2 === 0; }
    }), list);

This is not as immediately readable as the straight equality object syntax. But I like specifying the constraints property-by-property and having the ability to express more complex relations than just equality. And we still have the object notation for plain equality relations.

If you’re curious, here is how where is implemented in Ramda:

    where = curry(function(spec, test) {
      return all(function(key) {
        var val = spec[key];
        return (typeof val === 'function') ? 
          val(test[key], test) : 
          (test[key] === spec[key]);
      }, keys(spec));
    });

KingHunt

2013-11-28T00:00:00+00:00

Uneasy lies the head that wears a crown.

– Shakespeare, Henry IV Part II

I wanted to write an app for FirefoxOS.

It would help me sharpen my mobile development skills;
I could learn more about AngularJS;
I own a Firefox phone, and wanted to learn its API.

I decided to write a chess puzzle app. The parts were all out there already. Chess.js enforces legal moves, and detects check, checkmate, and draw. ChessboardJS provides the UI: the pieces, the board, the drag-n-drop. I found a large collection of puzzles at Yet Another Chess Problem Database. I contacted Dmitri Turevski at the site and confirmed that I could use the problems in my app, and he very graciously agreed. Dmitri also mentioned I should include the problem’s author with the problem, and I followed that advice.

So with all the pieces lined up, the problem was mostly a matter of wiring everything together. Here is what the app does:

Loads a default “book” of about 200 problems.
You move the pieces for both sides looking for a solution to the problem in the stipulated number of moves.
When you are satisfied that you have solved the problem, then you can mark the problem solved. The app stores the problems you have solved so that you may filter them out when browsing the list, or moving from one problem to the next.
You can undo moves, or reload the original position. You can also navigate to the next or previous problem.
The app does not require network access, it works entirely offline.

What the app doesn’t do (yet):

There is no AI, so you gotta find the best moves for both sides. I think this is reasonable for this type of problem which is training you to explore lines from both sides point of view. On the other hand, I may add an AI mode to try and find the defender’s optimal line in future.
You can’t load other books of problems. I really want to add this feature. The book is just a JSON file, so it should be straightforward to integrate that. But I haven’t had time to look into it yet.
I would also like the ability to use more problem types than just “mate in N” problems, e.g., general tactical studies, endgames, etc.

Despite the “work in progress” state that the app is in, I submitted it to the Firefox Marketplace. I found out a few days ago that my application KingHunt was accepted. Although it says it’s just for FirefoxOS, it should work in any modern browser. But I have only tested it in FirefoxOS.

You can check out the source code at the KingHunt GitHub repository.

AngularJS, RequireJS, AlmondJS, and Google Maps

2013-11-15T00:00:00+00:00

I had to solve the problem of integrating AngularJS and RequireJS with Google Maps. This is not usually a huge challenge, there is even an async module for RequireJS, and an Angular-UI map plugin to make integration pretty seamless. The wrinkle that made this integration trickier is AlmondJS—and Almond doesn’t support asynchronous module loading.

When you include the Google Maps script, you can specify a callback in the querystring that will fire when the Google Maps API is loaded. You could then bootstrap your Angular app in that callback, e.g.:

    <script>
    function mapReady() {
      angular.bootstrap(document.getElementById("gmap"), ['app.ui-map']);
    }
    </script>

    <!-- funky formatting to fit the whole url on the screen -->
    <script 
    src="https://maps.googleapis.com/maps/api/js?v=3.exp&sensor=false&callback=mapReady">
    </script>

This will work, but it will block your app until you hear back from Google Maps. I didn’t want to do that; there was only one spot where a user might use the map. Therefore, I wanted to lazily load it.

When you are configuring the $routeProvider, you can specify an object for the route parameter’s resolve property. Each property on the resolve object is a function that can return a promise. The controller will not get instantiated until the promise is resolved. And when the promise is resolved, the value returned from the promise resolution gets passed into the controller as a parameter.

So the plan of action was to resolve a promise when Google Maps was ready.

    // ...
    $routeProvider.when({
      path: '/path/to/where/ever',
      controller: 'SomeCtrl',
      route: {
        resolve: {
          gmap: function() {
            var dfd = $q.defer();
            // load google maps
            // fire callback
            // resolve promise
            return dfd.promise;
          }
        }
      }
    });
    //...

Here’s more-or-less how I implemented the gmap function.

    gmap: function() {
      var dfd = $q.defer();
      var doc = $window.document;
      var scriptId = 'gmapScript';
      var scriptTag = doc.getElementById(scriptId);

      if (scriptTag) {
        // if `scriptTag` exists, then gmaps must be available, so...
        dfd.resolve(true);
        return true;
      }

      // load google maps
      scriptTag = doc.createElement('script');
      scriptTag.id = scriptId;
      scriptTag.setAttribute('src', 
        'https://maps.googleapis.com/maps/api/js?v=3.exp&sensor=false&callback=mapReady');
      doc.head.appendChild(scriptTag);

      // fire callback, specified in the callback attribute of the script tag's src 
      // attribute.
      // The callback has to be in global (window) scope, so that Google Maps can find
      // it. I used a closure to store a reference to the deferred object we will 
      // resolve when the callback is fired.
      $window.mapReady = (function(dfd) {
        return function() {
          // resolve the promise:
          dfd.resolve(true);
          // cleanup the global space now that we are done:
          delete $window.mapReady;
        };
      }(dfd));
      
      return dfd.promise;
    }

With this approach, we don’t risk blocking the entire app; just this one route might block. Of course, this example just shows how to handle the “happy path”; it doesn’t handle if the Maps API fails to load for some reason. I also wound up extending it to return the current positions coordinates, instead of just true when the maps API is ready. That is left as an exercise for the reader …

The Tao of λ: Undoing with higher-order functions and closures

2013-10-09T00:00:00+00:00

Sometimes you need to temporarily change the value of an object property. Maybe you need to stub something for a test; or maybe you need to revert to the prior value for that property. You can do this imperatively, of course, and that is fine. But I want a more generic solution.

I want a function that I pass an object, the property to override, and the new value for the property, and have it return me an undo function to restore the object’s prior state:

    var undo = revalue(obj, prop, newValue);
    // do whatever
    undo();
    // hey, my original obj is back!

Here’s a first try that stores the original value in the closure of the returned undo function:

    var revalue = function(obj, prop, newValue) {
        var undo, orig = obj[prop];
        obj[prop] = newValue;
        undo = function() {
            obj[prop] = orig;
            return orig;
        };
        return undo;
    };

That pretty much does the trick. Then a colleague asked: “If the value is a function, why not add a restore property to it?” OK, it’s a bit redundant, since we are returning the undo function anyways, but WTH.

    var revalue = function(obj, prop, newValue) {
        var undo, orig = obj[prop], isFn = typeof newValue === "function";
        obj[prop] = isFn ? 
            function() { return newValue.apply(obj, arguments); } : 
            newValue;
        undo = function() {
            obj[prop] = orig;
            return orig;
        };
        if (isFn) {
            obj[prop].restore = undo;
        }
        return undo;
    };

Note that we have to wrap the newValue function inside a lambda:

    obj[prop] = isFn ? function() { return newValue.apply(obj, arguments); } : newValue;

If we don’t wrap newValue this way, then the restore property will be attached to newValue and it will leak out of the revalue function. We don’t want that. It’s safe to attach the restore property to the lambda:

    if (isFn) {
        obj[prop].restore = undo;
    }

And since anything worth doing is worth overdoing, the final version extends undo to take a callback function. That callback can be passed in as an optional fourth argument to revalue, or as an optional argument to the undo function itself. It’s up to the user to make the callback return something useful:

    var revalue = function(obj, prop, newValue, undoCallback) {
        var undo, orig = obj[prop], isFn = typeof newValue === "function";

        obj[prop] = isFn ? function() { return newValue.apply(obj, arguments); } : newValue;
        undo = function(cb) {
            obj[prop] = orig;
            cb = isFunction(cb) ? cb :
                    (isFunction(undoCallback) ? undoCallback : function() { return orig; });
            return cb(obj, prop, orig, newValue);
        };
        if (isFn) {
            obj[prop].restore = undo;
        }
        return undo;
    };

The Tao of λ: Deduping

2013-09-27T00:00:00+00:00

My Y Combinator article received a lot of attention, thanks to DailyJS and JavaScript Weekly. While the Y Combinator is a fascinating thing in itself, it is not exactly practical. In fact, it’s almost entirely academic (not that there’s anything wrong with that).

Discussions of functional programming are prone to wander into etherealness. It is tempting to keep abstracting and abstracting until the world of real problems is left way down below. But this need not be the case. Functional programming is a powerful and practical tool for solving real-world problems. And just the right amount of abstraction can be very productive.

Here is an example of a common real-world problem: Deduping an array of objects. Let’s start with an array of simple objects:

    var xs = [
      {x: {y: 1}}, 
      {x: {y: 2}}, 
      {x: {y: 3}}, 
      {x: {y: 4}}, 
      {x: {y: 1}}, 
      {x: {y: 4}}, 
      {x: {y: 5}}, 
      {x: {y: 6}}
    ];

The output we want is:

    [
      {x: {y: 1}}, 
      {x: {y: 2}}, 
      {x: {y: 3}}, 
      {x: {y: 4}}, 
      {x: {y: 5}}, 
      {x: {y: 6}}
    ]

For this example, we don’t care about the order of the output, if it is sorted, that’s just gravy.

I saw code to dedupe an array like this in production:

    function removeDuplicates(xs) {
      var i, ilen, prop, xObj = {};
      for (i = 0, ilen = xs.length; i < ilen; i++) {
        xObj[xs[i].x.y] = xs[i];
      }
      xs = []; // Yikes!
      for (prop in xObj) {
        if (xObj.hasOwnProperty(prop)) {
          xs.push(xObj[prop]);
        }
      }
      return xs;
    }

It works, but there’s a lot to dislike about removeDuplicates. For example, removeDuplicates mutates the input array. But the primary problem, for this discussion, is that it requires knowledge of the internals of the passed-in array. The key line is:

    xObj[xs[i].x.y] = xs[i];

Since duplicate values of the x.y property will overwrite any prior value on xObj, this is where the definition of object equality is (even though it may not look like it). Objects are considered duplicates if they have the same y property. We can rewrite this to make a generic deduping function that takes an equality predicate and a list of objects. And this generic version will not mutate the passed-in object either.

Here is the proposed API:

    function dedupe(equalityPredicate, list) { 
      // dedupe logic here...
      return dedupedList;
    }

Aside: I deliberately made the predicate the first argument. I am assuming that I can curry this function. I may want to create a specific xs deduper to reuse from my dedupe function. Or I may want to define different deduping functions and store them for future use.

Here’s the equality predicate for our xs object:

    function equalXs(objA, objB) {
      return objA.x.y === objB.x.y;
    }

And here’s an example of how to call dedupe:

    var newXs = dedupe(equalXs, xs);

All that remains is to implement dedupe. The simplest approach is to compare the head of the list to each element in another list (the “accumulator”) of objects we’ve already tested for uniqueness, using the equality predicate. (Therefore, the first element always goes into the accumulator.) Then recursively work our way through the list testing the object against the accumulated list. If we don’t find the object in the accumulator, then add it to the accumulator. As usual, I will use functions from Ramda to build this.

Let’s take the spec above apart and look at its components. First, we need a way to iterate over the accumulator and test it for equality with the current object from xs. The some function takes a function and a list returns true if any element in the list satisfies the function:

    some(function(accElem) {            
        return predicate(accElem, current); // the equality predicate we will pass in.
      },                                    // `some` will be true when the `current` 
                                            //  element is in the accumulator.
      acc);                                 // `acc` is the accumulator.

And we need to iterate over the passed-in array (xs) and use the some test above to determine which objects to accumulate. This sounds like a job for foldl.

    foldl(function(acc, curr) {
        // if the element is in the accumulator, return the accumulator;
        // if not, then add the element to the accumulator and return it.
        return (some(function(accElem) { return predicate(accElem, curr); }, acc)) ? 
            acc : 
            append(curr, acc);
      }, 
      [],        // the accumulator
      list);     // `xs` or whatever

Put it all together, and we have our generic dedupe function:

    function dedupe(predicate, list) {
      return foldl(function(acc, curr) {
        return (some(function(accElem) { return predicate(accElem, curr); }, acc)) ? 
            acc : 
            append(curr, acc);
        }, [], list);
    }

There you have it. dedupe does not require any knowledge of the array that gets passed to it. The dedupe function is generic; it can be used to dedupe any array of objects that the user can define an equality predicate for. It is entirely up to the caller to define what equality is for his objects with the equality predicate.

This design makes dedupe robust. Imagine that somehow the spec of xs changes tomorrow, and now its objects have a z property, e.g. {x: {y: 1, z: 100}}. In that case, removeDuplicates would require some major rewriting. On the other hand, dedupe would not change at all. All we have to do to accommodate the z property is change the equality predicate that we pass to dedupe like so:

    function equalXs(objA, objB) {
      return objA.x.y === objB.x.y && objA.x.z === objB.x.z;
    }

I hope this example demonstrates that functional programming isn’t all ivory tower stuff. In fact, it can be used to improve real code.

Update

Scott Sauyet has provided tests for the code above; also see the tests’ source code. Scott also took it another step by currying the some function and producing even leaner code.

Haiku Review: Learn You Some Erlang For Great Good

2013-09-26T00:00:00+00:00

Covering Erlang
Completely as a snowstorm
Takes time to shovel

AngularJS and FirefoxOS

2013-09-15T00:00:00+00:00

I have been happily developing an app for FirefoxOS. I got it into a state of almost working-ish, so on Sunday, I decided it was time to move from testing in the browser to installing it as a “packaged” app; either in the simulator or on the actual phone.

I read the docs on MDN about how to build the app, and how to install it. I got it installed in the simulator and on the phone. I could load the app, and visit its “About” page, but when I tried to load up the “Board” page, it bombed, with the error:

The address wan’t understood

Firefox doesn’t know how to open this address, because the protocol (unsafe) isn’t associated with any program.

WTF? This had never occurred when testing in the browser. After searching around for a while, I posted in the mozilla.dev.b2g group, and after a few hours, I got a solution. The problem is that AngularJS sanitizes URLs that it generates dynamically. Angular has a whitelist of protocols that it supports (viz. http, https, ftp, mailto). Any URL with a protocol not in that list gets prefixed with the protocol “unsafe:”. FirefoxOS packaged apps use the pseudoprotocol “app:”. So if you want to use a URL in your app that relies on AngularJS’s data binding in a FirefoxOS packaged app, you must add the “app” protocol to the whitelist.

Fortunately, this is simple to do. When you configure your Angular app, update the regex it uses for its whitelist:

    var app = angular.module( 'myApp', [] ).config(['$compileProvider', function($compileProvider) {
        $compileProvider.urlSanitizationWhitelist(/^\s*(https?|ftp|mailto|app):/);
      }
    ]);

Thanks to Julien Wajsberg for pointing me to the solution, and to this post on stackoverflow for the details.

Javascript Y Combinator

2013-09-05T00:00:00+00:00

At the end of The Little Schemer, the authors lead you step-by-step through the process of deriving the Y Combinator. They do this repeatedly abstracting a length function–and then magically, the Y Combinator appears. It is a pretty neat trick, and certainly mind-bending on your first read through the book.

Since Javascript has first-class functions, we can derive the Y Combinator in Javascript as well. I will take a different approach than The Little Schemer. This approach owes a lot to a couple of blog posts I read on the subject¹.

Since this is a post about recursive functions, then we can use that old chesnut, factorial. Here is a possible implementation of factorial in Javascript:

    function basicFactorial(n) {
      return n === 0 ? 1 : n * basicFactorial(n-1);
    }

Let’s start by making a non-recursive basicFactorial, which we’ll call nonRecursive:

    function nonRecursive(f) {
      return function(n) {
        return n === 0 ? 1 : n * f(n-1);
      }
    }

All we’ve done here is replace the recursive call of basicFactorial. Instead, we pass in a function that will get called. We can pass any function that returns something that supports the * operator:

    nonRecursive(function(x) { return 0; })(100); // => 0
    nonRecursive(function(x) { return 1; })(100); // => 100
    nonRecursive(function(x) { return 10; })(100); // => 1000
    // ... etc.

But it starts to get a little interesting when we pass basicFactorial in there. Then we get back … basicFactorial!

    nonRecursive(basicFactorial)(4) === basicFactorial(4); 
    nonRecursive(basicFactorial)(10) === basicFactorial(10); 
    nonRecursive(basicFactorial)(17) === basicFactorial(17);
    // ... etc.

In other words, basicFactorial is a fixed point of the function nonRecursive.

This is pointless in this case, since we have defined basicFactorial already. But suppose we had not defined basicFactorial. Wouldn’t it be nice if there was a function that we could pass nonRecursive to that would return the fixed point of it, i.e. the factorial function?

That is what the Y Combinator does. Pass nonRecursive to Y, and out comes the factorial function:

    Y(nonRecursive)(100); // 9.33262154439441e+157

Note that:

    Y(nonRecursive)(100) === nonRecursive(basicFactorial)(100);

Or in other words:

    Y(nonRecursive)(100) === nonRecursive(Y(nonRecursive))(100);

So if we have Y, we do not need to define basicFactorial at all, we let Y derive it from the non-recursive function nonRecursive. Now let’s look at it from the other direction, and build up to Y. Here again, is the functional nonRecursive that we want to calculate the fixed point of:

    function nonRecursive(f) {
      return function(n) {
        return n === 0 ? 1 : n * f(n-1);
      }
    }

As noted above, pass basicFactorial in, and nonRecursive returns basicFactorial. Notice that we have pretty much defined factorial in the body of nonRecursive: return n === 0 ? 1 : n * f(n-1);–why not use that? So here’s our next try: Apply nonRecursive to itself. This requires a small change to the body of nonRecursive, to self-apply the passed-in function to get the body out and apply it to the inner argument.

    function nonRecursive(f) {
      return function(n) {
        return n === 0 ? 1 : n * f(f)(n-1); 
      };
    }
    nonRecursive(nonRecursive)(5); // => 120

Now we want to isolate the fixed point function. Let’s wrap that in a function g:

    function nonRecursive(f) {
      return function(x) {
        var g = function(q) {
          return function(n) {
            return n === 0 ? 1 : n * q(n-1);
          };
        };
        return g(f(f))(x);
      };
    }

Since inner function g does not depend on anything in closure, we can pull it out:

    function g(f) {
      return function(n) {
        return n === 0 ? 1 : n * f(n-1);
      };
    }

The pulled-out function may look familiar–it’s nonRecursive again. Here’s what’s left over after g (a.k.a. nonRecursive) is pulled out; let’s call it almostY:

    function almostY(f) {
      return function(x) {
        return g(f(f))(x);
      };
    }
    almostY(almostY)(5); // => 120

We’ve pulled g out of almostY, but almostY still depends on g. The final step is to wrap almostY in a function that takes the functional g as an argument. Then almostY will have no dependencies.

So, let’s wrap it in a function that takes our non-recursive factorial functional and returns the fixed point of it. And since this is the last step, let’s call that function Y:

    function Y(f) {
      var p = function(h) {
        return function(x) {
          return f(h(h))(x);
        };
      };
      return p(p);
    }
 
    Y(g)(6); // => 720

Holy crap! It works! But it’s not just for factorial–Y will provide a fixed point for any unary function, e.g.

    function nonRecursiveFibonacci(f) {
      return function(n) {
        return n < 2 ? n : f(n-1) + f(n-2); 
      };
    }
    Y(nonRecursiveFibonacci)(10); // => 55

As presented, this version of Y can only handle unary functions, and it will blow up the stack for relatively low values of n. It is straightforward to extend Y to handle functions of any arity, and to memoize it.

Notes

I found these articles helpful: Fixed-point combinators in JavaScript: Memoizing recursive functions and Deriving the Y combinator.

“I don’t care” parameters

2013-08-29T00:00:00+00:00

I’ve been studying Erlang lately. It’s a lot of fun, and the language has some great features. One aspect of Erlang that appeals to me is pattern matching. You implement a function with different behaviors based on the arguments. For example, the factorial function in Erlang can be defined as:

    factorial(1) -> 1;
    factorial(N) ->
       N * factorial(N-1).

This makes for tight and elegant code; you don’t need to check your arguments and have branching logic. Of course in this case, I would probably add a constraint to the second one, i.e. factorial(N) when N > 1 -> ..., since if N < 1 then we’ll have a problem. Constraints are another great, expressive feature of the language.

Erlang takes pattern matching an extra step, borrowing from its ancestor Prolog: It uses the underscore character _ (or any parameter name that starts with _) as a throw-away parameter. So you can pattern match on a function with a signature like:

    some_func(_, X, _) when X =:= 0 ->
        X + 1;
    some_func(_, _, Y) when Y =:= 0 ->
        Y + 2;
    some_func(_, _, _) ->
        ok.

Again, I find this clear and elegant.

At work, I’m on a very large project that uses some jQuery, probably more jQuery than most of us would like. It seems like at least once a day, I get burned by writing code like this:

    $.each([10,20,30,40], function(item) {
      console.log(item);
    });

See the bug? What do you think this will log to console? The answer is [0,1,2,3]. Oops. The bug is that jQuery’s each function, for some God-forsaken reason, takes the index of the collection as its first argument, and the item as its second argument.

So I’ve borrowed the “I don’t care” parameter idea from Erlang, and now I write jQuery.each like so:

    $.each([10,20,30,40], function(_, item) {
      console.log(item); // 10, 20, 30, 40
    });

I like the look of this–the underscore character suggests “I could be any value; I don’t care” to me. I prefer it to function(index, item) where index never gets used. It seems like index should get used–it has a name!

There are some downsides to this approach:

If you lint your code (and you should), JSLint will complain about the underscore character, although you can turn that warning off.
Using the underscore for multiple “I don’t care” parameters will also fail lint, e.g. function f(_, _, x). You can turn that off too, but probably shouldn’t.
If you are using a library such as Underscore or Lodash, the underscore parameter will mask that variable inside the function in question.
Some dev in the future may work on the code and decide he wants to use the “I don’t care” argument, but not change the parameter name. Well, you’ll be long gone by then, right?

So, caveat emptor.

The Firefox has landed

2013-08-25T00:00:00+00:00

The delivery range was listed as “August 21 - August 21.” It was August 24 when my new FirefoxOS phone arrived. The phone is inexpensive, and it is underpowered compared to high-end iPhones and Android devices. But it has a major advantage over both iPhones and Android phones: It’s not made by Apple or Google.

I am a phone Luddite. This is my first smart phone. I find Apple’s treatment of users and developers repellant, and I don’t want to carry an ad-lead-generator for Google around in my pocket. The alternative was nothing, since obviously a Windows phone is completely out of the question. I suppose there’s Blackberry, too. Didn’t care.

These phone-resisting tendencies were reinforced by Stallman’s comments on cell phones:

I don’t have a cell phone. I won’t carry a cell phone. It’s Stalin’s dream. Cell phones are tools of Big Brother. I’m not going to carry a tracking device that records where I go all the time, and I’m not going to carry a surveillance device that can be turned on to eavesdrop.

The quote is from Network World, March 16, 2011. At the time, Stallman was mocked as a crank (e.g., “Richard Stallman is kind of a lunatic. He’s been tinfoil hat for many years now,” from the same article). Recent revelations about the surveillance state have proven him right–it’s not paranoia when it turns out that you have understated the problem.

In any case, the Firefox phone has the advantage of being free software, as far as I know. That was enough for me. Firefox has been my browser of choice at home for years, even through the lean years; even when Chrome was pushing the boundaries of browser performance and capability. As Google slides more and more into the antithesis of its famous slogan, Mozilla is once again in the position of being the defender of open standards and user freedom on the Web. That’s worth supporting, even if it means an underpowered phone.

As Linus Torvalds put it:

Technologies that lock things down tend to lose in the end.

Don’t bet against the open Web.

The project project

2013-08-20T00:00:00+00:00

What should the contract of a function called project be? Suppose you have a collection of objects like this:

var table = [
  {a: 1, b: 2, c: 3},
  {a: 10, b: 20, c: 30},
  {a: 12, c: 32},
  {b: 22, c: 42},
  {c: 300}
];

… and we want to fetch the a and b attributes of each object in table. Maybe I want to get back a set like this:

[
  {a:1, b:2}, 
  {a:10, b:20}
]

i.e., if an attribute is undefined on the input object, then the object doesn’t appear in the output. In terms of relational algebra, getting this output takes two operations; one to select the objects that have defined properties, and one project those properties onto the output objects. They are associative, but it’s smart to select first, then project, since you may have fewer objects to iterate over after the selection.

For now, the simple, algebraic approach is what we’re going to take with project in Ramda. In other words, project will simply project; it’s up to the caller to pre-select the input table or filter the output results. But there is more to it than that. The implementation of pick checks for the existence of a name on the input object before copying the value over to the projected object, i.e.:

project(['a', 'b'])(table)

will output:

[
  {a: 1, b:2}, 
  {a:10, b: 20}, 
  {a: 12}, 
  {b: 22}, 
  {}
]

So we’ve got a projected object out for every object in the input table, but if the attribute was undefined in the table, it doesn’t exist in the output. (Incidentally, this is how underscore does it.) Is that what we want? Or does this output make more sense for a simple project:

[
  {a: 1, b:2}, 
  {a:10, b: 20}, 
  {a: 12, b: undefined}, 
  {a: undefined, b: 22}, 
  {a: undefined, b: undefined}
]

This is essentially how a SQL-style project works (substitute null for undefined). This is simple enough to achieve by writing a new version of pick, called say, pickAll, that does not test for properties on the input object:

R.pickAll = curry(function(names, object) {
    var copy = {};
    each(function(name) {
        copy[name] = object[name];
    }, names);
    return copy;
});

Then compose map and pickAll, and you’ve got a project-style function that guarantees that the output objects will have the desired properties, even if they are undefined. There are two benefits of this; first, this is consistent with relational algebra and familiar from SQL; and second, projected objects will report the same result when enumerating their keys. Of course, they will still report the value of that key as undefined, exactly as the original pick would.

Function composition

2013-08-15T00:00:00+00:00

My friend and former (and future?) colleague, Paul Grenier, recently posted on the topic of function composition in Javascript. Paul takes the angle that we can avoid some of the performance problems associated with the functional style (although at the cost of some maintainability–constructed functions are not the most pleasant thing to debug).

We use a similar approach to “memoizing” functions by arity in Ramda that Paul recommends (cf. nAry function). Our implementation of a variadic compose function is straightforward:

var compose = function() { 
  var fns = slice(arguments);
  return function() {
    return foldr(function(fn, args) {return [fn.apply(this, args)];}, slice(arguments), fns)[0];
  };
};

Since we’ve implemented foldr in an imperative fashion (eeeeew), the performance penalty is negligible. And since the intent of compose is typically to generate a function once to use over and over, then compose may not be a great target for performance optimization in the first place.

Ultimately, what I want to do is be able to compose some really useful functions on the fly, with as little code as possible. I’m not there yet. Part of the problem is how to handle variadic functions. (That is a topic for another time: Composing Curried Variadic Functions. That should have a broad audience.)

For example, I’d like to compose map and pick to give me a function that would project attributes over an array of objects. This is analogous to a SELECT x, y, z statement in SQL. (Scott started down this road, but when he took some time off, I took up the torch.) This would be nice:

var project = compose(map, pick);
var selected = project(attrs, objects);

At present, this doesn’t work in Ramda because pick comes out of curry not knowing what its arity is. You can work around that like this, but isn’t as elegant:

var project = compose(map, pick);
var selected = project(attrs)(objects);

Why do something if you can’t do it elegantly? Furthermore, the “function butt” makes some Puritanical developers uncomfortable.

In Ramda we have another issue. Since pick will return an empty object if the passed-in object doesn’t have any of the passed in properties, this simple project will not behave like a SQL-style projection should: It won’t filter objects with undefined properties; it will include as many objects in its output as are given to it in its table parameter.

You can hack around that, of course, by filtering the table on the way into project:

project = curry(function(keys, table) {
  return compose(map, pick)(keys)(filter(function(row) {
    return all(function(key){ return row[key] !== undef; }, keys);
  }, table));
});

… but now this is starting to get ugly. A better approach may be to compose project out of a function that won’t return an empty object.

Well, it’s a work in progress, and there is plenty of room to improve!