Re: The Present of D
June 25, 2009
Jarrett can complain like no one else. But he also makes many good points about the state of D. There are tons of inconsistencies and flaws in the language and its toolchain. But that has to be quite obvious. After all, if it wasn’t for these deficiencies, D would certainly be a major player in the programming industry.
It’s true – the future of D is uncertain, but it has always been such. Has it worsened considerably in the recent future? Nope. Has it improved? Yes, but not much.
Some time ago there wasn’t even a good system library for D1. As Jarrett successfully notes, Phobos is underwhelming. I’d go even further by being politically incorrect and saying that it’s a complete mess. My recipe for spaghetti: try drawing an import graph of Phobos. It’s full of cycles and as we know, the DMD frontend hates cycles. They produce some of the most incomprehensible error messages the compiler can spew.
Fortunately, we don’t have to use Phobos anymore with D1. Tango is a much better option, as most of the community have already figured.
Not that long ago, D could only be reasonably used on two platforms: Windows and Linux. Now it can also run on OSX, FreeBSD, Solaris and there are folks that use it on Nintendo DS.
The other areas of D are also improving – QtD is great, Descent never ceases to amaze with its IntelliSense and compile-time code views, and LDC can do awesome code optimization, sometimes beating C++ and usually matching it.
I don’t agree with Jarrett on the matter of Tango – in my opinion, it’s quite stable and not hard to keep up with at all. If you’re using the SVN revisions, you’ll at least have to follow the Trac timeline for it. An RSS feed read daily will suffice just fine. But if you don’t feel like riding the cutting edge, sticking to the larger releases is what most people do anyway. In this case, you’d just have to read the ‘Breaking changes’ section when it’s out. And except the major I/O system overhaul, it never gets any lengthy. Writing a library to support both the cutting edge and the stable version might be quite tricky, but that’s what we have version blocks for.
As for the supposed difficulty configuring it, there are the effortless bundles as well as instructions for manual installation.
So D actually is improving, thus Jarrett is a madman and a drunkard. Well, no, he’s not, at least not always. You need to take his latest post with a grain of salt. The thing about frustration is that it stacks up. And folks are getting more and more frustrated with specific parts of D. As far as community goes, it’s doing fine: several folks working on a native D kernel, the first demoscene production, a few games in the making and many projects that I personally lack comprehension in order to discover fully. But unfortunately, the opinion that is becoming increasingly common is that the makers of D, most notably Walter can’t keep up. The specification is vague on many topics, to say the least. It’s not clear whether we’re really programming in D or perhaps in “DMD.languageof”. This issue is extremely efficient at depressing the makers of alternative compilers, like LDC and Dil. They would all like the corner cases to be cleaned up. Sure, they could submit patches and hopefully Walter would include them. But at one point, you can’t even know whether something is a bug or a feature. How do you fix something that is not well specified, but it’s known that the specification is frozen?
Another part that I think deserves special attention is my personal favorite: OPTLINK. The linker that you will most probably use if you program in D on Windows. That is, unless you’re adventurous and compile with a custom-patched version of GDC on top of MinGW or LDC without exceptions. The thing about OPTLINK is that it’s awesome AND it sucks. I mean, it’s very fast and we all love it for that. But at the same time, anyone doing non-trivial development with D probably knows the dreadful MessageBox Of Death. A recipe for obtaining it: build a medium-sized D project with debug info. You’ll get to know the x86 register values as OPTLINK comes crashing down at your face.
What Walter says: it’s hard for a module to reach the limit of “fixups”, it can be split up.
What happens in the real world: templates. And with them, ClassInfo, TypeInfo, debug info. Sum it all up and it’s not hard for a module to reach the fixup limit (which is an OPTLINK bug, not a problem inherent in OMF – the object file format it uses – DDL can process these ‘too large’ files just fine). It does happen in practice, and when it happens, it’s really bad. It’s grown to the status of voodoo, and in fact magic is used to prevent it from happening as well. “Try putting the modules in a different order when passing them to the compiler” and “Compile everything at once instead of one module at a time” are two such incantations. Sometimes they work – the compiler will put template instances in different modules, depending on which ones are encountered first. But automatic build tools don’t really make this well controllable. And if you’ve tried all the combinations of module order and you’re still getting the error? Remove -g when compiling. No debug info for you. But when that doesn’t suffice? You’re screwed. Like me.
I was working on ‘xpose2′, a meta-data processing and generation lib that allowed me to effortlessly expose D classes to MiniD, serialization and whatever else I wanted. Using xpose2 with MiniD and my game was quite easy and pleasant initially. That is, until OPTLINK issues struck. So I put the modules that used xpose2 into a separate package, compiled them into a lib separately from the rest of the project and built them without -g. I could lose debug info for them with relatively little disadvantage. Fine for a while, I continued binding more classes to MiniD. The linker was more persistent than that – the issues came back… But I don’t give up so easily – I split the bindings into more modules and worked with it for a while. Then it turned out DMD didn’t like me for splitting it up and gave me some strange crashes at my program’s runtime. After a lengthy investigation, it turned out it was a codegen bug related to the supposedly fixed issue of template mixins not being instantiated enough times, so e.g. typeof(this) yielded the incorrect type. After a whole process of giving up on it, doing ‘hg rm minid’, then reverting it, I had to settle on a hack that would mean I would not be screwed that much. I’ve modified xpose2 to rely more on string mixins (which I despise but have to use them in order to do the more complex metaprogramming) and added a custom pragma to DMD which would cause it not to emit symbols or debug info for functions and types used only for CTFE.
If you’d like to support DMD-win for your medium-to-large D project, these are just the issues you might face. QtD, MiniD and my yet unannounced project are just a few of these that do. The #d IRC channel holds more instances of this story.
I’ve known quite a few people using D for about 5 or more years now. They have collectively written probably a few hundred thousand lines of multi-paradigm D code that often pushes the limits of the compilers. I believe they’re an important part of the community, but I might be alone in this belief. You’d think that they have a huge impact on D because after all, Walter says that D is community-driven. I began to suspect that the community is Walter, Andrei and Bartosz. Don’t get me wrong – I wouldn’t dare to diminish their great work and all the efforts and sweat they put into making D a reality. The thing is – they don’t know or use D that much as one would think. They sure are keen on specific parts of the spec or how the compiler *should* behave, but I doubt they know all the traps and workarounds one learns about when doing some real D. Who does? The frustrated guys. They aren’t frustrated in order to upset Walter and co., but because they deeply care about the language into which they put their hopes and spent their time instead of going mainstream. I think they all realize that D has a great potential and allows to program things in ways impossible in most other languages.
Issues with .tupleof? Try doing it in C++, C# or Java. Many bugs that pop up in DMDFE are just that. We’re all grateful to Walter for enabling us to use a language with such an amazing feature set and we hate him at the same time because when forced to use other languages, we feel like slitting our wrists. But after the hundredth time something breaks in the compiler that was reported two years earlier, you begin to question the point of sticking with D. Perhaps the same could be done with an elaborate C preprocessor hack or a snippet of Bash script that will generate code as a pre-build step? We stick to D because we believe that Walter will finally clean it all up. Yet, there seems to be a disjoint vision of D by the people who use it and the people who shape it.
D2 is currently on the horizon, as well as a book about it. If I remember correctly, it’s meant to be out by the end of 2009. The new features it will have could just be groundbreaking. Sharing, proper constness, purity, enhanced meta-programming – they will each attract their share of new programmers. But if these people find that next to these features, there are some long-standing warts in the toolchain, like improper protection for renamed imports or forward reference errors declared inexistent by the specification, they’ll run away screaming… And damn me if I’m wrong, but the OPTLINK issues we’ve had are going to be nothing compared to what will come up when when people start using Phobos 2 with its template-heavy code. D needs a new toolchain and it needs it badly.
Does D have a future? I don’t know. Many people have decided that it doesn’t and left for other (greener?) pastures. The DSource graveyard is just a proof of that. Fortunately, many of the dead projects are in a shape good enough for reuse. Some good code can be still found e.g. in Core32, MinWin, Kong or heck, DSP. It takes a bit of necromancy at times, but overall, working with D can be quite productive. I’ve recently uncovered Enki2, and it’s doing a great job at parser generation for a larger project.
And as long as I can be productive with D, I like to believe that it has a future. Whether that future is Bright, is mostly up to one man to determine.
Filed under: Uncategorized | Comments (1)
