kate: intelligent code completion for all languages!

... well, maybe that's a bit of an exaggeration, but it's certainly much more intelligent than before. Look:

Code completion in CSS

... bash

... Lua

... PHP

even Gnuplot!

Note how this one has a different set of possible items for the same query, respecting the context.

Even Mathematica ;) This image shows a problem which still needs to be fixed: in case-insensitive languages, all completion suggestions are lowercased (which is not technically wrong of course, but a  bit ugly). It's easy to fix but simply not done yet.

There's unexpected profit from this in quite some areas, even KDevelop: for example, through this we now get code completion for all keywords in doxygen comments:

Completion for doxygen keywords inside a doxygen comment

Of course, those only appear inside actual doxygen, and not C++. When the cursor is in C++ code, it shows the C++ keywords instead (but they will not be very visible in KDevelop, since they're sorted below KDevelop's suggestions, which are better).

How does this work?

Short answer: magic! Correct answer: it uses the highlighting files. For highlighting, kate has a list of possible keywords for languages listed in highlighting files (/usr/share/apps/katepart/snytax/$language,xml). Those keywords are even context-sensitive: you will notice that e.g. the PHP highlighter does not highlight PHP function names inside comments or strings. So, the highlighting engine needs to know which keywords are valid at which position. Those are precisely the keywords which are suggested in the list.

What now?

Now that we have this feature, I think we can make more out of it in quite some cases. Especially, I want to invite you to have a look at your favourite language, and make sure all keywords / builtin functions / etc. are actually listed. Because of this feature, it might make sense to list keywords for languages where they are not terribly helpful for highlighting; a prominent example would be HTML, where currently the highlighter is totally generic and does not actually look at e.g. the tag names (thus, there's no completion). If you'd fix that by actually listing all valid HTML tag names, you'd (1) get better highlighting, e.g. you can mark undefined elements (think typos) as errors and (2) completion for free with that.

Another thing which can be improved is the context sensitivity. Some languages already do this rather well, but many languages will higlight keywords also in places where it'd be easy to detect that the keyword does not make sense there. That doesn't matter that much for highlighting only, because generally users write code which makes sense, but still -- if you can detect it, both consumers of the highlighting data (the actual highlighting, and the completion engine) gain something from it. So, extra motivation for making things more exact! ;)

I'm sure we can do more cool stuff with this. If you can come up with a good idea -- tell me, I'm happy to talk about it.

KDevelop / Kate sprint in Barcelona in January 2014

Last week I have been in Barcelona at the KDevelop / Kate sprint with all the other nice people working on those projects. As always, it was very cool to meet everyone again and spend a week together improving software. A big thanks to the organizers and sponsors, too!

Since most of the time I work on a fairly encapuslated subsystem (Python support for KDevelop) and only a smaller part of my contributions goes towards the KDevelop / Kate core repositories, I don't  have any world-changing news to announce -- unlike for kate and KDevelop, no Qt5 / KF5 porting is happening yet in Python support. It doesn't make much sense to start porting until all the interfaces have stabilized.

Still, quite some interesting things have piled up in the last few months, and especially during the sprint.
In the following, I want to give a quick summary of those.

Advanced code completion filtering in kate / KDevelop

I have implemented two new ways to filter the code completion popup in kate: filtering the list using an abbreviation, and filtering the list using text not occuring at the word's beginning. This can probably best be demonstrated by lots of pictures:

You can match completion items by their abbreviation. This works for both camel case and underscore notation.

You can also match entries by words they merely contain but do not start with. This matching is only allowed at word borders (capitals or underscores). This feature makes it far easier to find that damn class which has an unexpected name prefix, or the m_foo variable you thought was called foo.

The abbreviation expansion engine also allows you to type parts of the words from the abbreviation, making your search more specific in a convenient way.

This feature is not specific to kdev-python, it works in all kate-based apps. It is available in kate's master branch, and will be available in KDE SC >= 4.12.

If you have more suggestions or cases which are not handled well, I'm happy to discuss this further. Have fun hacking!

P.S. If anyone can come up with an efficient algorithm for doing what is depicted in the last image, I'd be interested. The current one is quite slow for some corner cases.