BUILD Breakage

Armed with my excellent reasons for wanting to move my company’s build system to the DDK, I sat down and tried to get one of our static libraries to build. This particular library gets linked with a couple of different binaries that we produce, includes headers from three other static libraries of ours, and yet is small and discrete enough to not be too much of a handfull to build. So, I made a SOURCES file, tweaked the include paths a bit, set up my 3790 DDK, and started whacking at trial builds.

Problem #1: Missing SDK header files. Yes, the DDK includes a (mostly) complete SDK, along with all of the ATL madness necessary to build, well, ATL-based projects, but it turns out that it’s missing some files. On the list are wininet.h and mprapi.h. Curious fact: both import libraries are in fact in the DDK. Just not the headers. Odd. For what it’s worth, the missing headers might be in the Longhorn DDK, but as it is a year or two pre-release at this point, that doesn’t seem like a feasible requirement. Especially due to the NDA. 😉 Anyway, it is obvious that I can’t rely on the DDK for my SDK – this is the first of dozens of projects that I’m trying to convert; who’s to say what else will be missing? Besides, I can’t exactly constrain my development team to using only headers in the DDK’s version of the SDK down the road.

Problem #2: Missing STL headers. We make heavy use of the C++ STL in our code. In fact, one of our developers actually branched out a bit and used hash_map, which is not quite part of the C++ standard, but is present in SGI’s STL implementation and recent Microsoft CRT header sets. However, much to my chagrin, there is no hash_map in the 3790 DDK. How odd. Furthermore, attempts to use the SGI STL led to Internal Compiler Errors. Did I mention that I hate C++? The only hash_map I could get working with the DDK compiler was the one that comes with current Visual Studio. Since one of the objectives that I failed to mention last time is to get the system away from dependence on an IDE for development, I don’t exactly love the idea of requiring Visual Studio for building. And yes, I know about the excellent and free Visual Studio Toolkit 2003, which is a big improvement, but it would still mean that my dev environment would need the PSDK, the visual tools, and the DDK. Not much of a simplification from what I have now.

Problem #3: Brain-dead source layout requirements. This one is my favorite. Yeah, we can work around the goofy SDK with its missing header files, and we can find a workable STL somewhere, but the driving reason behind my wanting to use the DDK for building – the BUILD.EXE utility itself – is broken. Really. I never really appreciated it until this conversion. According to the build utility documentation, source files must be present in either the current directory (i.e. the one with the SOURCES file in it), or in the parent directory, or in a platform-specific subdirectory. Dumb. Really.

The havoc that this creates with source trees is laughable, even for the small library I tried converting. Essentially, in order to use BUILD, you have to dump all of your source files (other than those #include’d from somewhere else) into one of these directories. Look around the DDK for some interesting examples of this. Who would want 200 source files in one directory, if they can be sanely and naturally broken up into subdirectories? For what it’s worth, BUILD is *not* the only utility with this assumption built in – there are others (Perforce’s Jam system, for example; more on that later). I know Microsoft builds lots of stuff with BUILD, and I can’t imagine that they like this. The NT source tree must have some crazy exceptions built into the main makefile (looks like it does, from makefile.new examination).

For better or for worse, this kills BUILD for my project, unless we attempt a major source code re-structuring, or do something dumb like generate a static library at every directory.

Problem #4: Platform support problems. I alluded to this one last time, but was willing to accept it as a limitation, assuming everything else worked as designed. There is no great way to run BUILD.EXE on linux (WINE might do it, of course, but I haven’t tried it). There is absolutely no way on Mac, that I’m aware of, due to different CPU architectures.

So, the moral of the story is that BUILD is really great for the smallish projects I typically use it on, but it presents a major scalability problem as the complexity of the project increases. Now, with that said, it’s entirely possible (likely?) that I don’t know what the heck I’m doing, and hopefully some bright reader will leave feedback pointing out my stupidity. A couple of readers offered their experiences; I’d love to hear what you guys came up with when you moved your projects.

So, the search for the ultimate build system continues.