Monday, 23 March 2009

Breaking Existing Code

We finally decided last week to have a go and try and move one of our applications from Delphi 2006 to Delphi 2009. Most of our other applications had been moved to Delphi 2007, this one, due to unfortunate timing, remained with Delphi 2006. I say try, because we are wary of the changes, and if it’s too much upheaval, we just may decide it’s not worth it.

It’s taken us a while to decide on whether we should make the move or not because this application had been made fully unicode compatible using widestrings and TNT components. We don’t actually need Delphi 2009’s unicode support, although we realise the difference between widestrings and Unicodestrings. The fact we’d done that work actually worked against us in this case.

Now I’ve done many of these upgrades. I’ve gone from Delphi 1 to Delphi 2, 2 to 3 so on and so forth, so I’m in a position to compare the experience. So far it hasn’t been easy. We use quite a few third party components which are no longer actively maintained, so we’re not getting a Delphi 2009 version anytime soon. Luckily, we have the source (actually luck doesn’t come into it, we’d never use a component if we didn’t have source!). Now the convention for creating buffers has always been to use PChar. This obviously doesn’t work anymore, and while our code isn’t too bad in this respect, the third party components we use are riddled with PChar buffers, pointer arithmetic etc etc

Now, I got to thinking. How could Codegear have made my life easier? Well for starters, they shouldn’t have broken my existing code! If PChar was a pointer to a (single byte) character, then it should stay that way! If a string is  an Ansistring, then it should have stayed that way. In fact I should have been able to recompile my code in Delphi 2009, and my application should have worked exactly the way it did in Delphi 2007 (I’d accept it if older code didn’t completely work first time, but Delphi 2007 code should more or less work the same) Unicode support is a great thing, and hats off to Codegear for implementing it in Delphi 2009, but I would have done it differently. I’d have left a string as an Ansistring, and created a new string for unicodestrings, PChar would’t have changed, and TEdit would still be, in essence a TAnsiEdit. They would then have created a parallel VCL, and appended say a W to the names (TEditW) for the unicode equivilents. So anyone who wants a unicode application needs to do a little work. Not too much, but hey, nothing comes for free. Those who don’t want it, and there must be a few out there who don’t really care, get a compiled application which behaves exactly as it did in Delphi 2007. They can upgrade to unicode, if they wish, at their own pace.

If Codegear had really wanted to be helpful, I guess they could have gone the extra mile and added some compiler directives. D2007 compatibility on or off. That way everybody is happy.

Codegear, and Borland before them, keep making the same mistake. They keep assuming that technology alone can create these magic buttons. Press them, and you get this wonderful transformation. That would be nice, but it just doesn’t work that way in the real world. They tried it with Kylix (just compile your existing Win32 code in Kylix, and you have a Linux application) and it didn’t work, they tried it with Delphi for .NET (just compile your existing Win32 code with Delphi 8, and you have a .NET application) and it didn’t work, and they’ve tried it again with Unicode support (just compile your existing non-unicode code with Delphi 2009, and you have full unicode support), and in my opinion it hasn’t worked either. Sure it works with a contrived example, but try it with any decent sized, real world application, and you are in trouble pretty quick.

I’m sure there’s many out there who have compiled their existing Delphi 2007 code in Delphi 2009, and with a little tweaking here and there have been able to get it to work. I just think it should be the developer’s decision when to introduce Unicode support. I guess it still is our decision, but currently ,if we decide that completely changing the way our code behaves is a step too far, our only option is to remain with Delphi 2007. And if that happens, Codegear are one step closer to losing a another customer.


Ritsaert Hornstra said...

This is exactly the staus we have with two major projects here ( each 250k lines+ ) We keep on developig in Delphi 2007 because switching to Delphi 2009 is NOT an option. Too bad for CodeGear this means no new income from new licenses. I once aksed a codegearian about it and the response was: then you should stick to D2007.

Anonymous said...

We are facing about the same problem. We have a very large Delphi 2007 application that compile with 0 warning. We have got more than 34000 warnings exploring the code using Delphi 2009 using Asta ( not yet ported to Delphi 2009, not mentionning tens of errors that we have resolved until now. So far the application (strongly database oriented with tiers servers) does not behave correctly and we will put an end to its developpement. At least it will not go behond Delphi 2007.

Charles said...

I couldn’t have said this better. A few weeks ago a colleague of mine and I sat down with DavidI at the “Delphi Productivity” meeting at the Embarcadero headquarters in San Francisco. During that meeting DavidI seemed genuinely excited about the new Delphi 2009; however I think he is a bit out of touch with real world examples such as yours and mine. The meeting turned into more than a sales pitch than the type of meeting I thought it was going to be which was an opportunity to listen to the needs and concerns of real Delphi programmers.
Our organization has ten products written in Delphi. That’s hundreds of thousands of lines of code that has evolved through the various versions of Delphi starting with version 1. Each version transition has been fairly painless up to this point; however we have been unwilling to transition with every version of Delphi. We went from Delphi 1 to Delphi 5 (skipping 2-4), to Delphi 7 (skipping 6) and then on through the “2000” series of 2005 through 2007. I think we are loyal Delphi users despite the fact that we are so heavily invested in the language. So now comes the decision of whether or not to “upgrade” to Delphi 2009, but is the Unicode compatibility enough to entice us to do so, or just wanting to keep up with the latest version and its bells and whistles? This no doubt will be a daunting task for us (our shop has nine programmers). We don’t really need the Unicode support in our applications, at least at this point, and the fact that our Delphi interfaces with ANSI-COBOL is another factor that will determine if and when we convert our code. Is it doable? Sure. Is it productive? Hardly. Embarcadero should have learned from Borland’s mistakes and taken the exact approach that you described. That would have made our lives much easier and probably even helped them not only retain but gain clients. Great post, thanks! – Charles Gates

Anonymous said...

Similar story here.

95% of the conversion to Delphi 2009 went smoothly, but the last troublesome 5% is proving to take up 95% of the conversion time.

Despite the Borland to CodeGear transition, those are still the same people at the helm with the same "aim foot, shoot" strategy. Sigh.

Moritz Beutel said...

> If Codegear had really wanted to be helpful, I guess they could have gone the extra mile and added some compiler directives. D2007 compatibility on or off. That way everybody is happy.

You don't think this has been considered, do you? Or even intensively discussed but discarded for certain reasons?

I agree that a Unicode switch would ease the upgrading process for many. But for this to work properly, two versions of RTL and VCL are required, one with Unicode support, one without. For designtime support, the IDE would need to have both loaded - how would you handle this?
Also, when Commodore finally arrives, we'll have 64-Bit support, which will of course be optional. Thus, we end up with four different versions of RTL and VCL, all of which need to be explicitly supported by component vendors.

Anonymous said...

The US are only 5% of the world population. That menas that 95% of all Delphi users need Unicode. I adapted my app within 2 or 3 hours. Now I have Unicode support! When I need Ansi then I write AnsiString etc. Where is the problem?

Ritsaert Hornstra said...

After reading the article and the remarks, what most strikes me is this:

Codegear thinks that everybody needs Unicode (which might not be the case)

Codegear, given this view (and perhaps because it's limited by dev hours) implements Unicode in a take it or leave it manner.

Codegear has not reacted in any positive way to it's own customers saying they still need an ANSI version of Delphi. Nor did they backport any relevant code, compiler helpers to validate your code under 2007. They even say to them: do not pay us and stick with 2007.

A lot of developers have problems with D2009, mainly because they have *a lot* of code that assumes quite a bit on the language feautures of 2007 like char=ansichar etc. Some of these projects are NOT ported because it is not economically feasable given the current D2009 state.

It wonders me: who at codegear invented this? a developer who only thinks about the latest and the greatest, a financial executive who thinks we should only support an unicode variant.

What happens if they alienate their customers now with this step. As a customer I start to think: what will be the next show stopper? and what are my alternatives.

As much as I do like Delphi (Our shop has used it since D1) it is still a tool, and when the tool cannot do it's job anymore, it might be time for a new one, how well the old one was in the past.

Dennis S said...

I totally disagree. Sure it is a lot of work porting such apps from nonunicode to unicode (have done this by myself, 300k+ lines), but why should I write new code with special TEditW lines - who needs pure ANSI code today? The problem is that unicode support should already be in D2007 - at a time when many ppl upgraded their delphi (d7+).

Now I never want go back to any other version, looking forward to the 64bit compiler.

Go Codegear. Go.

Anonymous said...

I dont post much on blogs. I just read and quitely think to myself. But I had to post saying I 100% agree. My life has been a nightmare with D2009. My app calls C dlls to integrate with 3rd party api's and uses pchar etc all over the place. Its been a nightmare, to the point of me thinking of just dumping it and going with C#. Im starting to think the only reason people stay with Delphi is just the vcl.

Thanks for the blog, good read.
Kind regards

Robin said...

Great article. I am chewing the carpet with frustration trying to upgrade products to work with D2009. And joy upon joy: I'm migrating from D7 to D2009 so am enjoying the benefits of the new help system and the marvellous new layout at the same time. Not.

Perhaps I'll get them all done just in time for Commodore to come along. :-(

"Go Codegear. Go" That's the gist of what I've been saying as well, but perhaps more floridly.

But in fact, it will probably be me that leaves: I run a small business, already under pressure (I suppose you may have heard that there is some economic turbulence right now?) and I simply cannot afford to keep modifying code which does not benefit my customers.

As Ritsaert says: time for a new toolbox.

Jason Sweby said...

Yes well said, this has been the single reason stopping me from moving to 2009 - many legacy applications with third-party components using PChar. Why should *I* have to rewrite someone else's code to get it to work?

agorbachenko said...

Some people are satisfied with new compiler, some are not satisfied. Such is a life. UNICODE switch cannot be implemented in reasonable time. Is Delphi 2012 more preferable for you? :)

> The US are only 5% of the world population. That menas that 95% of all Delphi users need Unicode.

+1 You are not alone on the earth. There are other people with other requirements. WE NEED UNICODE!

If you don't need Unicode then use Delphi 2007. It is fair.

I have migrated a large project (~1kk lines) to Delphi 2009 within a week. What are you talking about?
When your code is well-done there is no problem to migrate your application within several days.

Ritsaert Hornstra said...

One of the arguments that I do not understand is: we need two (or four for 32/64bit) times the VCL to be supported.
I *realy* do not understand this. It is normal (like in C++) and do-able to have multiple targets and a single source. Yes, it takes a bit more time but you end up with multiple possible targets. If you look at C++ applicatrions you can use a few compiler switches and have the executable to be Unicode, ANSI, 64bit, 32bit etc etc.

WHat I also don't understand is that Codegear does not try to give us the oppertunity NOW to prepare for the coming changes for 64bit. For example, introduce an IntPtr type in D2009/D2007 that has the save size as a pointer or should we use an ifdef on the comiler version to cast a pointer to an Integer or Int64?

agorbachenko said...

> many legacy applications with third-party components using PChar

Workaround for you is...
> When I need Ansi then I write AnsiString etc

compaspascal said...

We're in the process of porting a 500K lines of source code project to Delphi 2009. For now, things look pretty easy, it seems rather easy to convert, and the benefits are very significant.

Anonymous said...

1) You may like it or not, but Unicode is the present and the future. ANSI is not.
2) If you used Delphi 1, you know "string" already changed from TP-like string to AnsiString. For example S[0] no longer returns the string length. And did you complain when Delphi 2 didn't support 16-bit apps? Or Delphi 1 didn't support DOS apps? How much code had to be rewritten?
3) Delphi didn't remove ANSI support. It is just no longer the default. If you replace string/char with AnsiString/AnsiChar it will convert data for you using teh current codepage. As it happened the other way round any time a Windows API gets called.
4) C/C++ can heavily use the preprocessor to handle different targets, anyway without source code different libraries to link to are needed. And each piece of code has to be tested in any supported mode. Like it or not, going to 64 bit would have required to mantain four set of libraries, and I guess CG has not the resources.
5) GIven the resources, better to work hard to improve the IDE and the VCL

Tuyen said...

To the ignorant buffoon who said that 95% of Delphi developers "need Unicode" because only 5% of the world's population is American, please do yourself a favour and never ever open your mouth in public. If you make this much of an ass of yourself on the internet, I can only imagine how idiotic you sound in public.