2011-10-25

Building and running Clang static-analyzer on Windows/MinGW

With the Clang static-analyzer becoming more and more popular these days, MinGW users on Windows might be looking for some way to also bring the Clang goodness to their shores.

However, well, let's just say that the LLVM documentation isn't that intuitive for newcomers, especially if you were expecting to be able to download a nice Windows binary package and roll. This quick recipe on setting up a complete Clang static-analyzer environment for MinGW attempts to remedy that. As usual, what is presented below worked on my environment at the time of the post - I make no guarantee that it will work elsewhere, especially if you choose to deviate from the guide.

Also, before you start, please be aware that this whole process will send you back about 11 GB in terms of disk space, so make sure you have adequate room.

  1. Install MinGW (32 bit!) using the latest msys-get-inst (just run the GUI installer) picking up the very latest packages.
    For the installation directory, I used D:\Clang, which I will assume to also be yours for for the rest of the guide - please modify accordingly. With regards to the installation packages, select MinGW Compiler SuiteC compiler (which should be selected by default), MinGW Compiler SuiteC++ compiler, and MinGW Developer Toolkit to ensure that binutils and all the necessary tools to build clang, except Python, are available. You should also select MSYS Basic System, so that a shell is available. You probably also want to play it safe by picking up the pre-packaged repository catalog for MinGW. The retrieval and installation of the packages will take a few minutes.
  2. Download Python v2.x (32 bit installer) from http://www.python.org/download/.
    Use D:\Clang\bin as the installation directory and unselect everything but Python Interpreter and Libraries, as this is all you really need from Python.
    IMPORTANT: Using python 3.2.2 produced an error on my machine during the build, so make sure you pick v2.x.
  3. [OBSOLETE: You can skip to the next step as per comments below]
    Download the latest (Experimental) Clang Binaries for Mingw32/x86 from http://llvm.org/releases/download.html. Extract its content (bin/, include/ ...) into D:\Clang, overwriting any existing files. We do not care about headers being overwritten as this setup will be used for Clang and Clang only. Always make sure llvm-gcc is installed in the /bin directory of MinGW, unless you want to see a CreateProcess() error when running it (or want to have to duplicate the binutils files which we already got installed from picking up the MinGW Developer Toolkit).
  4. After installing an Subversion client such as Tortoise-SVN if needed:
    • Checkout llvm from http://llvm.org/svn/llvm-project/llvm/trunk (this is a Subversion URI), into D:\Clang\msys\1.0\src\llvm
    • Checkout clang from http://llvm.org/svn/llvm-project/cfe/trunk, into D:\Clang\msys\1.0\src\llvm\tools\clang
      NB: you may have to checkout clang outside of llvm first, then move it to the right location if your Subversion client doesn't let you checkout a repository inside an existing repository.
  5. Open an msys shell by launching D:\Clang\msys\1.0\msys.bat and issue the following commands:
    cd /src
    mkdir build
    cd build
    export CC=gcc
    export CXX=g++
    ../llvm/configure --disable-docs --enable-optimized --enable-targets=x86,x86_64 --prefix=/mingw
    make -j2
    The --prefix ensures that our files will be installed in the bin directory of MinGW and the two export lines avoid the need to install the Clang binaries to be able to recompile (As per now obsolete Step 3). Note that, in the msys shell, /mingw always point to the root directory where you installed MinGW (here D:\clang) regardless of the name you used. I would strongly advise against using a different directory, or not specifying the --prefix option (in which case /usr/local will be used as default), as I ran into issues when I tried (the static-analyzer failed to locate <stdio.h> when parsing sources). As usual, the -j2, which specifies the number of jobs to run in parallel, is there to make use of as many cores as we have (here two). Adjust according to your platform.
    Oh, and if you want to know why we use --enable-optimized, see the very end of this post...
  6. Go for a walk or something. Some of the libraries and executables are huge (hundreds of MB in size) so this whole process will take a while. During the various compilation runs I conducted, I chanced in one that froze (one make process using 100% of the CPU and never completing) and another where Windows reported "llvm-tblgen has stopped working", so don't be surprised if the compilation doesn't work quite right first time 'round -- it's fairly heavy on resources.
  7. If the compilation completed, then you can run:
    make install -j2
    Note that this operation will take some time too!
  8. Copy all the content of D:\Clang\msys\1.0\src\llvm\tools\clang\tools\scan-build, except c++-analyzer to D:\Clang\bin.
    Copy all the content of D:\Clang\msys\1.0\src\llvm\tools\clang\tools\scan-view to D:\Clang\bin.
  9. Issue the command:
    ln -s /mingw/bin/ccc-analyzer /mingw/bin/c++-analyzer
    The reason we're re-creating this symbolic link (which on Windows ends up being a duplicate rather than a link anyway) is that the one from the file provided by LLVM only works on UNIX systems.
  10. Edit D:\clang\bin\scan-build, line 100, and remove the /bin there to have:
    my $ClangSB = Cwd::realpath("$RealBin/clang");
    If you don't do that, scan-build will look for clang in /mingw/bin/bin and obviously fail to locate it.
  11. Finally, since we had to override them, you shouldn't forget to undefine the CC and CXX environmental variables:
    export -n CC CXX
    unset CC CXX

Success looks like what?

$ cd /d/libusb-pbatard
$ scan-build ./autogen.sh
(...)
$ scan-build make
make  all-recursive
make[1]: Entering directory `/d/libusb-pbatard'
Making all in libusb
make[2]: Entering directory `/d/libusb-pbatard/libusb'
  CC     libusb_1_0_la-core.lo
core.c:1626:3: warning: Value stored to 'r' is never read
                r = 0;
                ^   ~
1 warning generated.
  CC     libusb_1_0_la-descriptor.lo
  CC     libusb_1_0_la-io.lo
io.c:1819:8: warning: Call to 'malloc' has an allocation size of 0 bytes
        fds = malloc(sizeof(*fds) * nfds);
              ^      ~~~~~~~~~~~~~~~~~~~
1 warning generated.
  CC     libusb_1_0_la-sync.lo
  CC     libusb_1_0_la-poll_windows.lo
  CC     libusb_1_0_la-windows_usb.lo
os/windows_usb.c:1218:26: warning: Value stored to 'discdevs' during its initialization is never read
        struct discovered_devs *discdevs = *_discdevs;
                                ^          ~~~~~~~~~~
os/windows_usb.c:1297:4: warning: Value stored to 'session_id' is never read
                        session_id = 0;
                        ^            ~
2 warnings generated.
  RC     libusb-1.0.lo
  CC     libusb_1_0_la-threads_windows.lo
os/threads_windows.c:115:2: warning: Value stored to 'prev_pos' is never read
        prev_pos = pos = NULL;
        ^          ~~~~~~~~~~
1 warning generated.
  CCLD   libusb-1.0.la
Creating library file: .libs/libusb-1.0.dll.a
make[2]: Leaving directory `/d/libusb-pbatard/libusb'
Making all in doc
make[2]: Entering directory `/d/libusb-pbatard/doc'
make[2]: Nothing to be done for `all'.
make[2]: Leaving directory `/d/libusb-pbatard/doc'
Making all in examples
make[2]: Entering directory `/d/libusb-pbatard/examples'
  CC     xusb.o
xusb.c: In function 'test_mass_storage':
xusb.c:443:9: warning: variable 'junk' set but not used [-Wunused-but-set-variable]
xusb.c:520:4: warning: Value stored to 'junk' is never read
                        junk = fwrite(data, 1, size, fd);
                        ^      ~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
  CCLD   xusb.exe
./.libs/lt-xusb.c:692:11: warning: Value stored to 'len' during its initialization is never read
      int len = strlen (new_value);
          ^     ~~~~~~~~~~~~~~~~~~
1 warning generated.
  CC     lsusb.o
  CCLD   lsusb.exe
./.libs/lt-lsusb.c:692:11: warning: Value stored to 'len' during its initialization is never read
      int len = strlen (new_value);
          ^     ~~~~~~~~~~~~~~~~~~
1 warning generated.
make[2]: Leaving directory `/d/libusb-pbatard/examples'
make[2]: Entering directory `/d/libusb-pbatard'
make[2]: Leaving directory `/d/libusb-pbatard'
make[1]: Leaving directory `/d/libusb-pbatard'
scan-build: 8 bugs found.
scan-build: Run 'scan-view /tmp/scan-build-2011-10-24-1' to examine bug reports.
This is a typical result that Clang will (eventually - the analysis takes some time!) churn through. One false positive (that 'junk' variable is meant to be ignored, and only serves to eliminate a warning about not reading the return value from fwrite) and apart from that malloc code, everything looks fairly minor. Even on its very first report from clang, which of course we will act upon, libusb-1.0 on Windows doesn't appear to fare too bad!

Bonus:
If you followed the guide above, chances are you'll be using separate installations for the standard MinGW/MinGW-w64 and the Clangified ones. As such, you may want a title on the msys shell window that reflects that you are in a Clang environment, to identify which is which more easily.
To change the msys commandline window title, just edit your D:\clang\msys\1.0\etc\profile, locate the line:
export PS1='\[\033]0;$MSYSTEM:\w\007
and replace $MSYSTEM with the title you want.

Bonus 2:
Displayed after spending more than one hour of compilation, using gcc:
llvm[0]: ***** Completed Debug+Asserts Build
llvm[0]: ***** Note: Debug build can be 10 times slower than an
llvm[0]: ***** optimized build. Use make ENABLE_OPTIMIZED=1 to
llvm[0]: ***** make an optimized build. Alternatively you can
llvm[0]: ***** configure with --enable-optimized.
Now you tell me!?! Shouldn't that be the kind of info you want to display after someone issues configure?

8 comments:

  1. awesome - saved me a bunch of time - using the clang front end binaries for mingw32 x86 is not well documented. I was able to get clang working on an existing MINGW install by copying the content of the clang (bin/, include/ ...) to the mingw "folders". example C:\MinGW\lib\clang\2.9\include

    $ clang -o hello hel.c

    $ ./hello
    hello clang from mingw

    ReplyDelete
  2. I am unable to fix the bug "tblgen.exe stopped working". What should i do?
    I am using vista OS.

    ReplyDelete
  3. I get up to the last command in step 5 (make -j2) and get errors saying that 'cmath' file not found. Any suggestions?

    ReplyDelete
  4. See http://lists.cs.uiuc.edu/pipermail/llvmbugs/2012-March/022344.html

    I got the error as well, and it looks like, since we're recompiling and reinstalling Clang anyway, the better approach is to skip the installation of the MinGW32 Clang binaries (step 3), and instead set CC to gcc and CXX to g++.

    I have now amended the guide to do just that.

    ReplyDelete
  5. Excellent instructions. Thank you for helping me to go a long way towards a long term aspiration to run Clang static analyser over our code (with the bonus of compiling it under Clang for an alternative set ofwarnings to g++). Apart from a few svn oddities for the Clang code all went smoothly and I even managed to integrate it with my NetBeans IDE making the makefile a bit more project wise than my original knocked up batch version for the command line

    ReplyDelete
  6. Your step 11 does not do what you seem to think: it leaves both CC and CXX both defined and exported, but simply with a value equal to the empty string. To do it properly, you need:

    $ export -n CC CXX
    $ unset CC CXX

    Better still, don't export them in the first place, at step 5; if you really do need to override what configure will discover for itself, just define them within the configure command itself:

    $ ../llvm/configure CC=gcc CXX=g++ --disable-docs --enable-optimized --enable-targets=x86,x86_64 --prefix=/mingw

    is the preferred format, when configure has been generated with a version of GNU autoconf from about 2.50 onwards, or

    $ CC=gcc CXX=g++ ../llvm/configure --disable-docs --enable-optimized --enable-targets=x86,x86_64 --prefix=/mingw

    to temporarily export them, for configure generated otherwise.

    ReplyDelete
    Replies
    1. Thanks! I have now updated the step.

      Delete
  7. I am getting the same error described here: http://sourceforge.net/p/mingw/bugs/2043/ (error: 'FE_ALL_EXCEPT' was not declared in this scope ... error: 'feclearexcept' was not declared in this scope ... error: 'FE_INEXACT' was not declared in this scope)

    If I comment out the few lines in that file, the build goes much farther, but after much work, still dies while linking:

    llvm[2]: ======= Finished Linking Release+Asserts Executable llvm-profdata (without symbols)
    make[2]: Leaving directory `/src/build/tools/llvm-profdata'
    make[2]: Entering directory `/src/build/tools/llvm-symbolizer'
    llvm[2]: Compiling LLVMSymbolize.cpp for Release+Asserts build
    llvm[2]: Compiling llvm-symbolizer.cpp for Release+Asserts build
    llvm[2]: Linking Release+Asserts executable llvm-symbolizer (without symbols)
    C:/MinGW/msys/1.0/src/build/Release+Asserts/lib/libLLVMSupport.a(COM.o):COM.cpp:(.text+0x24): undefined refere
    nce to `CoInitializeEx@8'
    C:/MinGW/msys/1.0/src/build/Release+Asserts/lib/libLLVMSupport.a(COM.o):COM.cpp:(.text+0x31): undefined refere
    nce to `CoUninitialize@0'
    c:/mingw/bin/../lib/gcc/mingw32/4.8.1/../../../../mingw32/bin/ld.exe: C:/MinGW/msys/1.0/src/build/Release+Asse
    rts/lib/libLLVMSupport.a(COM.o): bad reloc address 0x20 in section `.eh_frame'
    collect2.exe: error: ld returned 1 exit status
    make[2]: *** [/usr/src/build/Release+Asserts/bin/llvm-symbolizer.exe] Error 1
    make[2]: Leaving directory `/src/build/tools/llvm-symbolizer'
    make[1]: *** [llvm-symbolizer/.makeall] Error 2
    make[1]: Leaving directory `/src/build/tools'
    make: *** [all] Error 1

    How do I get past these errors?

    ReplyDelete