Apply tokenizer truncation before post-processor - #307

When we apply the tokenizer it does:

1. Truncate the sequence if configured (accounting for the number of tokens added by 2.)
2. Applies post-processor (in particular templating that adds leading trailing tokens)
3. Applies padding if configured

So for example, lets say we have sequence that tokenizes to [10, 20, 30, 40] and the processor that adds 0 at the beginning and 1 at the end. And let's say we want to enforce length 4. If we tokenize without truncation, it's going to give us [0, 10, 20, 30, 40, 1], and then we manually truncate to [0, 10, 20, 30], which looses the added end token. On the other hand if we configure truncation it's going to compute [10, 20, 30, 40], then first truncate to 4 - 2 addeditional tokens, so [10, 20], and then add the tokens [0, 10, 20, 1].

Now the issue is that Tokenizer.set_truncation copies the underlying tokenizer. It is relatively cheap (bumping some refcounts and copying added_vocabulary, not the full vocabulary). That said ideally we would configure the tokenizer once. So what we can do deprecate options like :length, :truncate_direction, :pad_direction, and instead have a separate function to configure that. @josevalim wdyt?

As you prefer!