Document explorer versus useful data

There is a trend in the tools coming out of Microsoft that is driving me nuts and in my opinion significantly hurting productivity.  This trend is the shrinking of the amount of data that appears on the display and requiring more mouse clicks to get there.

For driver writers this trend is most obvious in Document Explorer 8 which is used to display the Vista WDK documentation.   Comparing it to HTML Help Control 5 that was used for Windows Server 2003 SP1 DDK the previous version will show you what I mean. 

On opening the DDK documentation you have two major panes, the pane on the right goes from the toolbar to the bottom of the window, giving you a large amount of information, on my display (1280×1024) about 60 lines or a full page.  When you open the WDK you see an extra pane on the right, called index results that takes away about 12 lines on my display or roughly 20% less useful data.  

Now the second assault on data occurs when you search, with the DDK I get 36 terms on the screen, with the WDK I get 27 or a 25% drop in data.  Worse yet, the search display is now a tabbed window in the data area making toggling between several elements of the search more cumbersome.

Of course index results mentioned previously also make thing more painful, since a large portion of the time you click on a kernel API in the index, instead of seeing the API in the data pane, you see some page about a use of the API, so you have to go down and click the index results to get the API.  As an example go into the Vista DDK documentation, and choose ZwCreateFile in the index, I doubt most of us are looking for TDI Kernel-Mode Client Interactions” when we look up ZwCreateFile but that is what we get.

Microsoft is doing all this at the same time that displays are going to wide screen models with 13% less vertical pixels.  All of this adds up to taking longer to get the information needed to develop or check a driver, making it more likely that the implementation will be rushed and unreliable. 

I don’t blame the WDK documentation group, the tool is a corporate standard.  Perhaps the folks who designed and approved this trend have become so used to Power Point that more that six lines of data at a time is too much for them.   Hopefully, Microsoft will wake up to the problem, at the recent WinHEC a feedback session was asked “How do you search for WDK help?” and the overwhelming answer was Google!


Crossing the Undocumented Line

As a consultant who has more than once taken on projects Microsoft has said are impossible, many people assume I often use undocumented calls in Windows in my work.  In fact, I try to avoid them if at all possible, and am extremely careful in crossing the undocumented line.

A developer should ask a set of questions when considering using an undocumented technique; these are:

  1. Is it really needed?  Before anything else, ask yourself if there is any other way to do this.  Be sure to not constrain your design when you ask this question.  For instance, requiring something be done in the kernel may force an undocumented technique where building a helper application could allow a documented approach.

  2. How undocumented is it?  There is a lot of variation here; there are plenty of undocumented calls that are never mentioned by Microsoft.  These I consider pretty undocumented even if there are examples on the web. 
There are calls that have been in the Microsoft DDK include files for years, but are not documented.  For example, a number of the ZwXXX calls have options that are not described in the documentation, but are in the includes with all the needed data structures.  

Another variant of undocumented is the call that is documented in later OS’s but works for your driver in previous versions.  In the early days of the DDK, there were calls in the samples that were not documented.  These calls are not completely safe, but they are better than the totally undocumented calls.

  1. Which OS’s require the undocumented stuff?  If you are looking at an undocumented call for an older OS, the function you need in later OS’s is not present, so you are probably safer than expecting a call to be there in the future.

  2. What is the scope of the input and usage of the undocumented stuff?  If you are using the undocumented calls in a constrained environment you are in a safer position than a having a widespread and flexible use.  It is hard to test any piece of code, but if the inputs or patterns of use vary, the testing of this area just got harder.

  3. What is the fallback plan? If you do go ahead and do this, what will you do if it breaks tomorrow?  Has your design kept the undocumented stuff limited so it can be replaced in the code?  Is there likely to be an alternative for the replacement?  What is the plan if there is no replacement?

If you decide you have to cross the line and use an undocumented call, there are some things you should be doing:

·         Let the customer know – Document the call and the answers to the above questions to the customer or your management.  Crossing the line should be a decision made by more than just the developer.

·         Plan for a long testing cycle – You have chosen to use something you cannot rely on, so your testing pain just went up.  You really need to test all the likely OS configurations the end user could have.  This means not just testing on the current version of a Windows OS, but instead the base, all service packs, and the current version , i.e., all the latest hotfixes and downloads.  For 32-bit Windows, that is sixteen full tests if you want to support Windows 2000 up to Vista!

There are times to cross the line, but be sure you have a plan and a strong justification before you enter the undocumented territory.


Bleeding edge and far from it

I am just back from WinHEC and while there I realized that many people including a number from Microsoft don’t distinguish developing for the leading edge from living there. I am known as a guy who has done a number of things that Redmond had said “Windows is not capable of doing” and technologies that Microsoft was later proud to show off once they were working. 

When it comes to my tools, I am far from the bleeding edge.  For instance, though I recently started using the latest Visual Studio, most of my work is still done with VS6.   I like the new Studio, but I trust VS6 and that version produces code that for the most part needs no libraries other than the standard system lib’s to support it.  I use Office XP and probably will not move for a long time, since I once had a terrible experience of not being able to send a customer a promised document, since the conversion back to the format the client was using did not work well.


I am not alone in this attitude; many of the developers I respect the most stay far away from the leading edge for their work environments.  In this way we are like Seymour Cray, who required that the parts for his designs were in production and testable even though the systems he was designing were years off.


The challenge here is that Microsoft does not seem to understand this.   The WDK team was proudly pointing out the features and fast updates with online MSDN, when most developers use the local documents.  One of the new features with the MSDN stuff is a wiki capability that is supposed to be monitored by using an RSS feed.   Most of the senior developers I know do not monitor this, since Microsoft’s previous generation of tools do not support RSS!  Another RSS problem is that when Microsoft “improved” its blogging site, it dropped email notification of blog postings, because RSS was available in all the brand new tools.


Another example from WinHEC was the Windows Driver Testing Framework.  This tool assumes you are using the latest Visual Studio to develop with managed languages using the full strength of COM.  I know of almost no respected driver writers who know how to do this, and even fewer who would want to. 


It has to be confusing for the WDK team because their core product is the exception that proves the rule.  Over the years, the development community has found that BETA and just released versions of the DDK are excellent so we trust using these in our day to day work.  Unfortunately, few other teams in Redmond inspire this confidence.


So, Microsoft, if you want us to keep developing on the bleeding edge, give us tools that do not force us to live there.


Never stop learning

One thing I always recommend to my customers is to keep their developers up to speed on the latest Windows driver technologies.  I’m thinking about this right now, since last week saw the release of Developing Drivers with Windows Driver Foundation and we are less than two weeks away from the start of this year’s WinHEC. 

It is scary how many managers and developers believe that they don’t need to know this new stuff.  Even if you are still required to do drivers for NT 4.0 (hopefully not), learning the new stuff can improve your code.  Just seeing where Microsoft has put its effort to improve things may suggest to you places in your code you need to review and improve.

Also, just because you can’t use the latest Windows versions does not mean you cannot use the latest tools.  I had a customer a few years ago who had been chasing a bug for around six months.   I used PREfast on the code and found the bug in the legacy driver in a couple of minutes!

Yes, you can in theory learn this stuff from the Windows Driver Kit documentation, but that is the hard way to do it.  There are a lot of great papers on Windows Driver and Hardware Central and at OSR Online .   I check them weekly to see what is new.  The challenge for papers and documentation is to know how to approach all the information.  That is where a book, WinHEC, or one of the excellent classes offered on Windows drivers can help.

This year will be my 10th WinHEC, and at every one of them I have learned something new that I now apply to my work.  More importantly, WinHEC is a chance to meet many of the experts in the field. You don’t want to abuse this, but it is extremely useful to be able to drop an email to an expert and ask a question.   

People ask on forums, “What book should I get to learn device drivers? “  The real answer is, all of them.  Looking up at my bookshelf, I see 17 books on Windows internals and drivers, including each version of some books.  This does not count all the class notes, the booklets from WinHEC, and many years of OSR’s NT Insider.  It also does not count the draft of the excellent WDF book I mentioned in the first paragraph.  I was a technical reviewer of the book and cannot wait to get my hands on the production version.

I’ve been writing drivers and system software for 35 years, and it surprises some of my clients when I mention I’ve taken five classes on Windows driver development.  These clients think if you’ve written drivers before, you only have to tweak what you know.  Nope.  Windows is an incredibly rich development environment for drivers, and there is a lot to learn.  Right now if I had the time, I would like to sign up for a couple of classes from OSR and a couple from Azius, about Windows driver development, because there are areas I know I could learn more about.

Never stop the learning process for yourself or your developers.   If you do, you are crippling your product’s capabilities and/or time to market, and your future employability as a developer.