Why Windows doesn't let you create an USB MS-DOS startup disk

Today, I'll provide an answer to the following question, which, if you're like me, must have irked you some when formatting an USB flash drive:

Yes why? Why is is that Windows has no problems creating an MS-DOS startup disk for the now useless floppy technology, but does not allow you to do so for the ubiquitous USB memory stick one?
It's not really like this is an impossible feat on Windows, as there exist quite a few tools that allow you to create one already (and I have now added my MUCH BETTER ONE to the mix). So why doesn't Microsoft do it?

If it ain't broke, don't fix it... But don't upgrade it either.

Ah, conservatism. I'm not talking about the political kind here, but rather its no less sinister computer equivalent: "Hey, we worked hard enough to get there. We don't see why we should take the risk of changing existing code, just so newcomers can reap the benefits."

You see, the way Windows creates an MS-DOS startup disk is not by doing anything smart, but by doing something lazy —which some also say is the second smartest thing to do after "smart"— such as including a 1:1 image of a 1.4MB floppy disk in a DLL (diskcopy.dll) and just splashing it wholesale onto a floppy when you request a startup disk.
Oh and yes, that's L-A-Z-Y-lazy since, if you actually have a look at the image file, you'll see that Microsoft didn't even spend much time creating a well thought-out one, but instead picked up the first Windows Millenium Edition (Me) startup disk they had lying around and just proceeded to "delete" stuff until it looked OK enough. The proof of it is that the image still contains much of the "deleted" content from the original disk and for a good read on how exactly the Windows Me Startup disk was "emasculated", you are very much invited to read Daniel Sedory's entry on the subject.

Of course, this whole technique of imaging a whole floppy does fall apart as soon as you're trying to create an unknown size USB bootdisk and this, my little children, is why Windows won't let you create DOS bootable USB flash drives to this day. End of story. Good night.

But wait, there's more!

Alright, alright, I hear you: that was a pretty underwhelming bedtime story. So of course there's more. And to answer your follow up question on why Microsoft didn't just compensate for the issue above by dropping the floppy image from diskcopy.dll and simply include the files themselves, we'll press on with a little bit of history.

As stated previously, the "DOS" files included in the image file that Windows uses when creating an "MS-DOS" startup disk actually come from a Windows Me boot disk, which, amongst its many disappointments, is also well known for crippling the DOS version it came with. As Wikipedia indicates, "One of the most publicized changes from Windows 98 is that Windows Me does not include real mode MS-DOS". Furthermore, departing from any MS-DOS that came prior, "the Autoexec.bat and Config.sys files are used only to set global environment variables". Thus DOS 8.0, which is the one included with Windows Millenium, is indeed seen by many as crippled, whereas DOS 7.1, a.k.a. the one from Windows 98 SE, is often labelled the "last good DOS".

But much more relevant to the USB boot issue is that the Windows Me DOS is not actually able to produce a command prompt on anything else but a floppy media. That is to say, even if you were to create all the required boot records on an USB drive, and then copy the unmodified COMMAND.COM and IO.SYS from the diskcopy.dll floppy image, your drive will never boot. You can try this for yourself, if you have an existing bootable stick, as 7-zip will happily extract the files from diskcopy.dll if you want to copy them over.

That still doesn't explain anything...

"OK fine," you say, "Windows Me is unusable but Microsoft should have the rights to their own DOS stuff anyway, so why don't they either just use the DOS files from Windows 98, or produce an uncrippled new version of the WinMe DOS, and go with that?". That's actually a very valid question, which people have only been able to speculate onto this day. Technically and most likely legally, there doesn't exist a reason why Microsoft shouldn't be able to provide a proper set of DOS files to boot an USB stick with. Furthermore, and this is the trick people use when they want to legally provide MS-DOS files for USB boot on Windows, uncrippling the WinMe DOS files from the diskcopy.dll to get them working on USB is actually trivial and documented here. For the record, even the well known HPUSBFW tool, from HP will patch the COMMAND.COM and IO.SYS files according to the above, when provided with WinMe DOS files. Yet, Microsoft went as far as explicitly discouraging people from doing so...

Thus, the moral of our story is that, while there is no real technical limitation to it, there is no satisfying explanation as to why Microsoft decided to inconvenience millions of their users by not allowing to create a DOS bootable USB drive. However, if you keep watching this blog, you will soon be provided with a 100% Open Source tool that does just that (and more!), so stay tuned.

Bonus

By the way, if you get your hands on an older version of the HPUSBFW tool --the one that is about 400 KB in size, you should be advised that it contains, as a resource, a self extracting UPX compressed executable that will extract the Windows 98 DOS files. The utility itself is designed to only enable extraction of these files if specific HP USB hardware (VID:PID 03F0:0023) is detected, however that can easily be patched to enable any device (F0 03 75 15 → F0 03 90 90 & 23 00 75 0C → 23 90 90 0C) and one could use HPUSBFW as a standalone USB DOS boot utility.