So, the last two days I've been concentrating on introducing locking into the definition-use chain which I've been writing for KDevelop. I've taken quite a coarse-grained approach, with one lock for each document's chain, and with separate object local locks each time that an object can reference another object that is on a different chain.

It took several hours yesterday to complete the basic scheme where the chains weren't referencing each other. I didn't even realise they weren't referencing each other until later, when I recognised that a speedup commit that Matt had contributed had removed some of my code which made this happen before. In fact, it was fortunate, because it allowed me to develop the code in two steps.

So today I've been trying to make the chains play nice while referencing each other. The going hasn't been easy. First I was confronted with some more unprotected accesses - these aren't too hard to fix, thanks to an extensive assert system that I've put in place to detect when locks aren't being held that should be. Second, there was a deadlock in the code completion code which took a little while to figure out - note to anyone using QReadWriteLock: if you ask for a second read lock while someone else is asking for a write lock, you'll deadlock. Third, I hit a race inside katepart where text edits and smart range updates could occur simultaneously, which was also bad. That wasn't too hard to fix. Now, I'm cleaning up some access of deleted memory with the help of Valgrind.

I'm not sure that this code can even be made to be reliable, given its complexity. Adam Treat pointed me at an article recently, The Problem with Threads, and I'm starting to agree. Having multithreaded definition-use parsing and manipulation would be a good goal, but perhaps not more than being stable. So, I plan to give it some time in svn to see if it can be made reliable enough for kdevelop's purposes.

In other news, my latest hardware purchase arrived today - a Dell 30" flat panel screen. At 2560x1600, when you combine it with another large screen (I have 2x 24" already, at 1920x1200), my graphics card can't handle them in xinerama mode, as the virtual screen size maximum is 4096x4096. So, I've been trying it in separate screen mode, but it's not a particularly pleasant setup - problems including the loss of ability to alt-tab between screens, kicker popping up on the other screen to the one that you're working on (and crashing), and of course not being able to drag apps between screens. Anyway, the 30" screen is excellent quality. I took a photo but it doesn't do it justice at night time...

Firstly, thanks to clee for adding me to the Planet. When I left Trysil I planned to blog more often, and now you too can read about it

So, for the last 2 weeks I've been on annual leave, and have had the fortune to be "on a roll" with my coding. I'm currently concentrating on KDevelop 4. We're privileged to have a new parser framework by Roberto Raggi and Jakob Petsovits. I've been working on extending this good work with a new type system and definition-use chain.

The definition-use chain is a sequence of scopes in a code file, and the associated definitions which occur in those scopes. These are created on the first pass after parsing to an AST (abstract syntax tree), along with the data types. On the second pass, we search for a definition which corresponds to every variable use, and complete the chain. This work is now nearing completion; remaining issues are template support, and better forward declaration and typedef support. I also have to reconsider the use of shared pointers in the type system, because as Roberto pointed out, C++ allows recursive types (which don't work well with shared pointers). Here's a look at some of the highlighting possible with the definition-use chain: (please ignore the bugs

On Thursday, I implemented code completion. This is the first real use of the new model-based code completion that I wrote for Kate. By expressing the possible completions in a model, and properties associated with each (eg. public/protected/private, etc), Kate can then perform some advanced functions such as sorting, filtering, and grouping. I'll post a screenshot when I can undo some recent breakage.

On Friday I implemented a symbol table, which contains a reference to every definition currently in memory (from all files). I should have done this a week ago - it turned out to be quite simple. Now, hopefully the definition searching will scale up to handle the load when we have all includes parsed in.

[Side note: did you know that Print Screen invokes KSnapshot? a neat trick I learned by accident]

Some work has also gone into performance of this new system, which has paid handsome dividends thanks to valgrind/callgrind/KCachegrind. Parsing speed was increased by 80%, thanks to the removal of dependance on QTextStream in the preprocessor, an unused symbol table in the parser, and KUrl comparisons in the definition-use chain (which I was surprised about... I thought it would be much quicker given that they should be implicitly shared). More recently, I've optimised the most heavily used paths in the definition searching algorithms, and was pleasantly surprised at the performance increase when switching a critical foreach loop into a const iterator style. Also, I've discovered QVarLengthArray in optimising our QualifiedIdentifier class; as they are created and destroyed very often, reducing the list overhead was important (note to self: maybe making them implicitly shared would help even more).

So, where to from now? We're reaching a fairly stable api, and one which allows for code refactoring, intelligent navigation, improved automatic code generation (eg. I'm looking forward to "create switch statement"), context-sensitive code completion, integration of documentation, debugger integration, a code structure view, call graph, static code analysis etc. If you're interested in helping, we'd love to hear from you [as manyoso would say, especially if you have QGraphicsView mad skillz].

One of bugzilla features is displaying various headlines, probably in order to cheer up the poor bugreporter or bughunter. KDE bugzilla actually doesn't seem to have this enabled, but I liked this in the old SUSE bugzilla, it had a nice collection of funny quips for the headlines. Those are gone now though after the switch to Novell bugzilla, which seems to prefer various encouraging quotes from famous people (I can't help it but sometimes they kinda remind me of the old communist times with their slogans). However, looking at the complete list I can see that some funny comments are finding their way even here, so all is not lost .

Anyway. Among the various lines I found some that show that even wise and well-known people have their limits. Consider, for example, this one:

If you think C++ is not overly complicated, just what is a protected abstract virtual base pure virtual private destructor, and when was the last time you needed one? -- Tom Cargil, C++ Journal.

The name rings a bell, I think I read a C++ book by this person once (I don't remember if it was a good or bad one though, I've read both kinds). But if he doesn't known, well, good luck I'm here to the rescue. The answers are "some nonsense just made up" (because, if nothing else, private destructor effectively disables inheriting, so it can't be a base class) and therefore the second answer is "never", for everybody. I admit though that the question looked really tricky. Let's try if I can make up something like that as well. How about this one?

If you think plain C is not overly complicated, just what is a static const non-volatile pointer to a register non-const volatile integer, and when was the last time you needed one?

(Just in case you wonder, the answers happen to be remarkably similar. Bonus points for people who can explain why a correct answer may be "&" or "" and "all the time" if it's changed to "register const non-volatile pointer to a static const non-volatile X" depending on the type of X. Extra bonus points if I didn't get it right and you can point it out )

And there's one more interesting quote that shows that a person often doesn't see all the posibilities of their invention:

I invented the term Object-Oriented, and I can tell you I did not have C++ in mind. -- Alan Kay

I don't see why one should feel so depressed about not seeing all the possible uses. It happens to everybody, all the time. I wonder if something like this would cheer this guy up:

I invented the term Programming, and I can tell you I did not have C in mind. -- Charless Babbage

(Although, I suppose, Alan Kay would probably like it more to find out Charles Babbage had no idea Smalltalk would exist one day. My bad.)

I think I should even try to submit these to the Novell bugzilla, I know I can't compare myself with the others, but perhaps they'll make it through. Everybody needs to cheer up sometimes when digging through bugreports.

Anyway. I have one more that probably says it all:

Bugrit. -- Foul Ole Ron

Yesterday I did some more work on KDevelop's c++ parser, specificially the definition-use chain (duchain). This was in response to some bugs that were being revealled by using it as the basis of advanced code highlighting.

I'd been struggling with a problem surrounding contexts where definitions are made (eg. in a funtion's parameter list, and then uses ocurring after that context has been closed (ie., using the open/close brackets as the context); also, in for and if statements). Previously I had been making those contexts span the whole function definition which didn't feel quite right, but allowed the context structure to remain a simple tree design.

Finally I realised that I had already implemented the solution; as I was using a directed acyclic graph (DAG), I could close the contexts as seemed right, then import them as subsequent parent contexts for the function body, etc. Indeed, this is probably what Roberto meant when he suggested I use the DAG design to start with, but it had taken me until now to realise it. I was thinking that he had meant to use the DAG features to incorporate other chains from #include-d files which had already been processed.

So, the duchain work is now more complete and correct. Improving it further now relies on proper importing of definitions from other files, and speeding it up by getting incremental parsing working. I've started on the former by separating the objects required for persisting the AST into a clean encapsulation in the parser, and separating the preprocessing from the parsing in the c++ parser. Next, I am going to make use of Threadweaver's job dependancy feature to order parsing such that when #include statements are encountered, the information from those included files will be available when it comes time to parse the file.

On the performance side, although Roberto's parser is quite fast, there is a noticable delay between changing a document and updating of the highlighting. I suspect that it's not all parsing time, but as there are many passes (preprocess, parse, code model binding, duchain creation (which probably needs to be two passes itself)) it's going to be important that we use incremental parsing (ie only reparsing changed text) as much as possible.

So, there are still many challenges in the way of the complete KDevelop4 experience, but the project continues to gain momentum...

For those of you who might not be aware, KDevelop v4 is starting to take shape, at least behind the scenes. For some time now we have had Roberto's new c++ parser in the code base but not doing very much. Then Matt and Adam worked hard on improving project management support (including native support for CMake) and the background parser / code model, so now we have a functional code model (even if not in the final state).

More recently, Jakob's Java and C# parsers (along the lines of the c++ parser) have been added to the codebase. They are just beginning to get codemodel integration.

While this was happening, I took over Roberto's work on creating definition-use chains. These are trees which represent contexts or scopes in which definitions are made, and then links from each definition to all of its uses. This turned out to be relatively easy once I had spoken in person to Roberto about it at kde four core in Norway (for some reason I didn't really understand what they were about when we talked online).

Yesterday, I managed to apply syntax highlighting to the katepart editor, based off the definition-use chain. This means that kdevelop will have separate highlights for variables defined in different ways, eg. local / function / class member / global, and also unrecognised variable usage. Here's a screenshot:

(yes, there are bugs visible there) This work is still quite raw and several major crashes need fixing, but this work has much potential. I'll save the details for my next blog entry...

So, for me the KDE four core meeting in Trysil has finished today. It was an amazing experience, meeting people for the first time and seeing what we could do when we concentrated (most) all of our efforts on improving KDE4.

I'll blog a bit more about what happened at the meeting later, but now I'm waiting in London's Heathrow airport waiting to get on a longhaul flight for 24 hours to Melbourne via Singapore. I decided to make the most of my time on the way back, hacking as much as I could. So far, I've done some programming on the bus, at Norway's airport (both while having dinner and while waiting for the flight at the terminal), in the plane while we sat on the tarmac for 30 minutes waiting for flight clearance, and again in the air - it's no surprise that my 2 batteries were completely exhausted at the end of that (although amazingly they held up pretty well considering).

I'm working on kdevelop's parser, and more specifically on the definition-use chain logic. Roberto Raggi's code is simply amazing, or perhaps I think so because I don't have a formal IT education. A little background: Roberto has already written a comprehensive and fast preprocessor and parser which delivers a very detailed AST (abstract syntax tree). There is some hacky support for a code model, but it really needs replacing. Thus earlier this year Roberto started on a new type system (framework ready, not much logic) and definition-use parser (some framework)

At Trysil, Roberto visited for a day and we discussed what was needed to be done for kdevelop4, as unfortunately Roberto has very little to no time to help these days. We'd had similar discussions online, but the power of face-to-face meeting is not to be underestimated.

I've managed so far to read and understand Roberto's latest code, and have now completed the definition-use chain parser, and made stubs for the definition-use chain data structures, and a new class which is aimed at texteditor integration (the texteditor integration class I'm particularly proud of, because it was a mess before).

Once this is finished, I will hopefully move on to fleshing out the type system. Then, all that's left to do is to bring it all into the ui, with a model for code completion; and refactoring, highlighting, and navigation support. A long way off yet, but I personally think that this has the potential to make a much stronger case for developers switching to katepart/kdevelop.

Well, my first battery is recharging quickly, so I should get back to the hacking while I still have AC The only other problem is I've been assigned a non-laptop-friendly seat on the plane, so here's hoping that they might take pity on me after the flight closes (but I doubt it...).

Signing off from Heathrow, Hamish.

(This entry is, of course, in relation to logs by ruurd, ruurd and mpyne)

A couple of updates on the code completion changes. In current CVS code completion can:

1. Complete in TRY/CATCH blocks...
2. Complete on vars declared in if/for/while...
3. Lots of other fixes...

And I have this working although the patch is up for review and not in cvs:







There are a few issues, because it relies on richtext and the problem is I have to strip out some chars programmatically which becomes a problem for apps who use katepart and completion entries with legitimate richtext chars...

Just came across this article via msdn, C++: The Most Powerful Language for .NET Framework Programming. Microsoft seems especially pleased with their new version of Managed C++. They've done a complete revamp of their previous .NET support for C++ and submitted it to ECMA and ISO as a standard. This brings C++ inline with C# as far as first class support for the Common Language Runtime. I really like what they've done to improve the C++ experience for .NET, but still think it will be incredibly hard to adapt the Qt/KDE libraries to fit into this framework. Consider this annoyance, the new proposed standard mandates PascalCase be used for the core API. For a technical challenge you'd have to consider how to integrate MOC and the new C++.NET properties. Read on.

Since I am somewhat new to C# and .net I decided to follow up on a
thread I had read on Joel On Software about the new Microsoft Visual
Studio Express product
line which is targeted at enthusiasts and hobbyists. So I went, and
sure enough, Visual C# Express Beta was a free download, and only 50mb.
Note that they require that you give them a valid email address, and
also, from what I've read, the evaulation expires next year.