Building and running Clang static-analyzer on Windows/MinGW

With the Clang static-analyzer becoming more and more popular these days, MinGW users on Windows might be looking for some way to also bring the Clang goodness to their shores.

However, well, let's just say that the LLVM documentation isn't that intuitive for newcomers, especially if you were expecting to be able to download a nice Windows binary package and roll. This quick recipe on setting up a complete Clang static-analyzer environment for MinGW attempts to remedy that. As usual, what is presented below worked on my environment at the time of the post - I make no guarantee that it will work elsewhere, especially if you choose to deviate from the guide.

Also, before you start, please be aware that this whole process will send you back about 11 GB in terms of disk space, so make sure you have adequate room.

1. Install MinGW (32 bit!) using the latest msys-get-inst (just run the GUI installer) picking up the very latest packages.
   For the installation directory, I used D:\Clang, which I will assume to also be yours for for the rest of the guide - please modify accordingly. With regards to the installation packages, select MinGW Compiler Suite → C compiler (which should be selected by default), MinGW Compiler Suite → C++ compiler, and MinGW Developer Toolkit to ensure that binutils and all the necessary tools to build clang, except Python, are available. You should also select MSYS Basic System, so that a shell is available. You probably also want to play it safe by picking up the pre-packaged repository catalog for MinGW. The retrieval and installation of the packages will take a few minutes.
2. Download Python v2.x (32 bit installer) from http://www.python.org/download/.
   Use D:\Clang\bin as the installation directory and unselect everything but Python Interpreter and Libraries, as this is all you really need from Python.
   IMPORTANT: Using python 3.2.2 produced an error on my machine during the build, so make sure you pick v2.x.
3. [OBSOLETE: You can skip to the next step as per comments below]
   Download the latest (Experimental) Clang Binaries for Mingw32/x86 from http://llvm.org/releases/download.html. Extract its content (bin/, include/ ...) into D:\Clang, overwriting any existing files. We do not care about headers being overwritten as this setup will be used for Clang and Clang only. Always make sure llvm-gcc is installed in the /bin directory of MinGW, unless you want to see a CreateProcess() error when running it (or want to have to duplicate the binutils files which we already got installed from picking up the MinGW Developer Toolkit).
4. After installing an Subversion client such as Tortoise-SVN if needed:
   
   • Checkout llvm from http://llvm.org/svn/llvm-project/llvm/trunk (this is a Subversion URI), into D:\Clang\msys\1.0\src\llvm
   • Checkout clang from http://llvm.org/svn/llvm-project/cfe/trunk, into D:\Clang\msys\1.0\src\llvm\tools\clang
     NB: you may have to checkout clang outside of llvm first, then move it to the right location if your Subversion client doesn't let you checkout a repository inside an existing repository.
5. Open an msys shell by launching D:\Clang\msys\1.0\msys.bat and issue the following commands:

   cd /src
   mkdir build
   cd build
   export CC=gcc
   export CXX=g++
   ../llvm/configure --disable-docs --enable-optimized --enable-targets=x86,x86_64 --prefix=/mingw
   make -j2

   The --prefix ensures that our files will be installed in the bin directory of MinGW and the two export lines avoid the need to install the Clang binaries to be able to recompile (As per now obsolete Step 3). Note that, in the msys shell, /mingw always point to the root directory where you installed MinGW (here D:\clang) regardless of the name you used. I would strongly advise against using a different directory, or not specifying the --prefix option (in which case /usr/local will be used as default), as I ran into issues when I tried (the static-analyzer failed to locate <stdio.h> when parsing sources). As usual, the -j2, which specifies the number of jobs to run in parallel, is there to make use of as many cores as we have (here two). Adjust according to your platform.
   Oh, and if you want to know why we use --enable-optimized, see the very end of this post...
6. Go for a walk or something. Some of the libraries and executables are huge (hundreds of MB in size) so this whole process will take a while. During the various compilation runs I conducted, I chanced in one that froze (one make process using 100% of the CPU and never completing) and another where Windows reported "llvm-tblgen has stopped working", so don't be surprised if the compilation doesn't work quite right first time 'round -- it's fairly heavy on resources.
7. If the compilation completed, then you can run:

   make install -j2

   Note that this operation will take some time too!
8. Copy all the content of D:\Clang\msys\1.0\src\llvm\tools\clang\tools\scan-build, except c++-analyzer to D:\Clang\bin.
   Copy all the content of D:\Clang\msys\1.0\src\llvm\tools\clang\tools\scan-view to D:\Clang\bin.
9. Issue the command:

   ln -s /mingw/bin/ccc-analyzer /mingw/bin/c++-analyzer

   The reason we're re-creating this symbolic link (which on Windows ends up being a duplicate rather than a link anyway) is that the one from the file provided by LLVM only works on UNIX systems.
10. Edit D:\clang\bin\scan-build, line 100, and remove the /bin there to have:
    my $ClangSB = Cwd::realpath("$RealBin/clang");
    If you don't do that, scan-build will look for clang in /mingw/bin/bin and obviously fail to locate it.
11. Finally, since we had to override them, you shouldn't forget to undefine the CC and CXX environmental variables:

    export -n CC CXX
    unset CC CXX

Success looks like what?

$ cd /d/libusb-pbatard
$ scan-build ./autogen.sh
(...)
$ scan-build make
make  all-recursive
make[1]: Entering directory `/d/libusb-pbatard'
Making all in libusb
make[2]: Entering directory `/d/libusb-pbatard/libusb'
  CC     libusb_1_0_la-core.lo
core.c:1626:3: warning: Value stored to 'r' is never read
                r = 0;
                ^   ~
1 warning generated.
  CC     libusb_1_0_la-descriptor.lo
  CC     libusb_1_0_la-io.lo
io.c:1819:8: warning: Call to 'malloc' has an allocation size of 0 bytes
        fds = malloc(sizeof(*fds) * nfds);
              ^      ~~~~~~~~~~~~~~~~~~~
1 warning generated.
  CC     libusb_1_0_la-sync.lo
  CC     libusb_1_0_la-poll_windows.lo
  CC     libusb_1_0_la-windows_usb.lo
os/windows_usb.c:1218:26: warning: Value stored to 'discdevs' during its initialization is never read
        struct discovered_devs *discdevs = *_discdevs;
                                ^          ~~~~~~~~~~
os/windows_usb.c:1297:4: warning: Value stored to 'session_id' is never read
                        session_id = 0;
                        ^            ~
2 warnings generated.
  RC     libusb-1.0.lo
  CC     libusb_1_0_la-threads_windows.lo
os/threads_windows.c:115:2: warning: Value stored to 'prev_pos' is never read
        prev_pos = pos = NULL;
        ^          ~~~~~~~~~~
1 warning generated.
  CCLD   libusb-1.0.la
Creating library file: .libs/libusb-1.0.dll.a
make[2]: Leaving directory `/d/libusb-pbatard/libusb'
Making all in doc
make[2]: Entering directory `/d/libusb-pbatard/doc'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/d/libusb-pbatard/doc'
Making all in examples
make[2]: Entering directory `/d/libusb-pbatard/examples'
  CC     xusb.o
xusb.c: In function 'test_mass_storage':
xusb.c:443:9: warning: variable 'junk' set but not used [-Wunused-but-set-variable]
xusb.c:520:4: warning: Value stored to 'junk' is never read
                        junk = fwrite(data, 1, size, fd);
                        ^      ~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
  CCLD   xusb.exe
./.libs/lt-xusb.c:692:11: warning: Value stored to 'len' during its initialization is never read
      int len = strlen (new_value);
          ^     ~~~~~~~~~~~~~~~~~~
1 warning generated.
  CC     lsusb.o
  CCLD   lsusb.exe
./.libs/lt-lsusb.c:692:11: warning: Value stored to 'len' during its initialization is never read
      int len = strlen (new_value);
          ^     ~~~~~~~~~~~~~~~~~~
1 warning generated.
make[2]: Leaving directory `/d/libusb-pbatard/examples'
make[2]: Entering directory `/d/libusb-pbatard'
make[2]: Leaving directory `/d/libusb-pbatard'
make[1]: Leaving directory `/d/libusb-pbatard'
scan-build: 8 bugs found.
scan-build: Run 'scan-view /tmp/scan-build-2011-10-24-1' to examine bug reports.

This is a typical result that Clang will (eventually - the analysis takes some time!) churn through. One false positive (that 'junk' variable is meant to be ignored, and only serves to eliminate a warning about not reading the return value from fwrite) and apart from that malloc code, everything looks fairly minor. Even on its very first report from clang, which of course we will act upon, libusb-1.0 on Windows doesn't appear to fare too bad!

Bonus:
If you followed the guide above, chances are you'll be using separate installations for the standard MinGW/MinGW-w64 and the Clangified ones. As such, you may want a title on the msys shell window that reflects that you are in a Clang environment, to identify which is which more easily.
To change the msys commandline window title, just edit your D:\clang\msys\1.0\etc\profile, locate the line:

export PS1='\[\033]0;$MSYSTEM:\w\007

and replace $MSYSTEM with the title you want.

Bonus 2:
Displayed after spending more than one hour of compilation, using gcc:

llvm[0]: ***** Completed Debug+Asserts Build
llvm[0]: ***** Note: Debug build can be 10 times slower than an
llvm[0]: ***** optimized build. Use make ENABLE_OPTIMIZED=1 to
llvm[0]: ***** make an optimized build. Alternatively you can
llvm[0]: ***** configure with --enable-optimized.

Now you tell me!?! Shouldn't that be the kind of info you want to display after someone issues configure?

On Windows 8 and WCID devices

With the 1.2.0 release of libwdi, which, for those who don't know, is a library aimed at making the installation of Windows USB drivers as simple and as non intrusive for end-users as possible, one of the most exciting features brought forward is WCID device driver support.

WCID, which was brought to the limelight with Windows 8, and which stands for Windows Compatible ID, is a set of additional USB Descriptors one can add to a device firmware, to make the OS automatically identify which type of driver should be used, and potentially automatically install it. In other words, what WCID does is bring the Plug-and-Play functionality that exists for HID or Mass Storage, to any USB device.

Thus, as upcoming Windows 8 comes with a WinUSB WCID driver, if you currently produce USB devices that require driver installation, you should seriously consider converting them to WCID as it'll make life a lot easier for everybody.

This is also good news for libusb-1.0 users, as libusb-1.0 relies on WinUSB for generic access on Windows, which means that you can actually produce libusb-1.0 applications for Windows 8, that do not require you or your users to take care of any driver installation. And for earlier versions of Windows, up to Windows XP SP2, libwdi and its friendly installer application Zadig can make it very easy to proceed to the one-off installation of the WinUSB (or libusb-win32 or libusbK) WCID driver, to make those platforms behave in the same way as Windows 8.

More information:

• WCID Devices
• Microsoft WinUSB Device

Avoiding Program Compatibility Assistant warnings on Windows

If you're developing applications on Windows Vista or later, you may end up in a situation where this rather annoying little fellow pops up after your program execution:

"What gives?, you say, my program's not an installer and it doesn't have anything like 'setup' or 'install' in the executable name! Why does Windows pester me and my users with such a warning?" The answer: have a look at your resource file (RC, etc) and especially your versioning strings, such as "FileDescription" or "ProductName". If there's any or part of the taboo keywords in there, such as "setup", "install", "update", Windows assumes that the executable is an installer, and also assumes the worst. Thus, the solution is to never use any or part of these keywords in any of the location that Windows will look for.

For more information on how Windows "heuristically" detects installation programs, have a look at the Installer Detection Technology paragraph from the following TechNet note.

Flashing a NEC/RENESAS USB 3.0 uPD720200/uPD720200A firmware

It looks like NEC/Renesas are releasing steady releases of their popular USB 3.0 controllers drivers and firmwares. In case you have such a controller, you probably want to check the following page for the latest version.

One of the problems I found with the latest firmware update (4015) is that the GUI Renesas upgrade utility just didn't work for my controller, on Windows 7 x64. Even worse, it created commandline processes ad-infinitum, which, apart from obviously clogging up the system, is the last thing you want to see, as repeated erase/flash cycles can really wear out a flash chip.

The good thing however is that the firmware upgrade comes with a commandline flashing utility (W200FW35.exe), so you might as well use that, and flash the firmware manually. Having a commandline utility may also help with a flashrom implementation in the future, since it should be fairly straightforward to reverse engineer. The one thing you have to be careful of however, when using W200FW35.exe, is that you need to select the right SPI-Flash-ROM device type for flashing, and that the included batch files, which are set to a specific type, may not match the one you have.

Issuing W200FW35.exe /? will provide more information. First, in an elevated command prompt, start by issuing /srom ? to identify your chip and check that the flashing utility can access it:

C:\uPD720200_uPD720200A_FW_Updater>W200FW35.exe /srom ?
Bus:0x04 Device:0x00 Function:0x00
This Device is uPD720200A(Revision 4).
    W25X10BV/20BV/40BV(WINBOND)/EN25F05/10/20/40/(EON)/A25L512/010/020/040(AMIC) Type : 3, PageSize = 0x100, Chip Erase = 0xC7

Then, if you have a uPD720200A based controller, you probably want to issue:

W200FW35.exe /srom 0 /dump backup.mem
W200FW35.exe /srom 0 /write F401502.MEM cfg.ini

/srom 0 above ensures that the flash type will be autoselected. Or, in this case, we could have used /srom 3. If you use an srom parameter other than 0, be mindful that using the wrong type will result in both garbage in and garbage out.
As digitaldiatribe also points out, if you have more than one Renesas chip on your system, you will also need to add the /address switch to select the right one. Update: As pointed out by Eluder, the latest firmwares from Renesas no longer come with the W200FW35.exe commandline utility. It can still be obtained from the "3.0.2.8.0.8 (uPD720200) & 4.0.1.5.0.2 (uPD720200a)" archive available on the link provided above.