2011-06-05

Extracting and using a modified VMWare Player BIOS or UEFI firmware

(updated 2016.05.17 for VMWare UEFI firmware extraction)

One day or another, you may want to play with BIOS/UEFI firmware modification. But before jumping into physical motherboard flash alteration, where the consequences of a mishap can be difficult to salvage, experimenting with the BIOS/UEFI in a virtual environment sounds like a sound first step. You have of course the opportunity to do so using bochs, however the ubiquitous nature and ease of use of the (freely available) VMware Player can turn the latter in a much better candidate for the job if you don't really care about the extra features that bochs brings in. This post details how one can extract the BIOS/UEFI firmwares used by the VMWare Player, and how to setup the player to use a modified one.
  1. Download and install VMware Player. The current version of VMware Player is 12, and the download can be obtained from this page (freely, but after e-mail registration). Binaries are available for Windows and Linux, in both 32 and 64 bit versions.
  2. Extract the BIOS from the VMware executables by following the instructions below:

    • Windows: Download and install 7-zip (which any reasonable Windows user should have installed anyway). Then navigate to your VMware Player installation directory and locate the vmware-vmx.exe application (notice the -vmx here). It should reside in the same directory for 32 bit, or in the x64\ directory for 64 bit. Open it in 7-Zip and go to the .rsrc\BINRES\ directory.
      For the BIOS (internally called bios440, as it emulates an intel 440BX chipset), you need to look for a file exactly 524 288 bytes (512 KB) in size. On current versions, there should be only one, called '6006'. This is the BIOS file we are after, so just extract it to a directory of your choice.
      For the UEFI firmware, you should look for a file that is 2 097 152 bytes (2 MB) in size. Because VMWare can emulate both 32 and 64-bit platforms, you will find that there exists 2 of these. At the time of writing this article, just know that '6020' is the IA32 UEFI firmware and '6021' is the X64 UEFI firmware.
    • Linux: From a terminal, navigate to the directory vmware-vmx binary resides (default is /usr/lib/vmware/bin. Issue the following set of commands (copied from the Arch Linux VMware page):
      $ objcopy /usr/lib/vmware/bin/vmware-vmx -O binary -j bios440 --set-section-flags bios440=a bios440.rom.Z
      $ perl -e 'use Compress::Zlib; my $v; read STDIN, $v, '$(stat -c%s "./bios440.rom.Z")'; $v = uncompress($v); print $v;' < bios440.rom.Z > bios440.rom
      Remember that you can always use objdump -h to find the various sections before using objcopy, in case the BIOS is no longer called bios440.rom
  3. Edit the BIOS or UEFI firmware as you see fit. Note that the BIOS VMware uses is of type Phoenix.
    If you are on Windows, you can download a full version of Phoenix Bios Editor Pro v2.1.0.0 from Intel under the name BIOS Logo Change Utility (or simply search for "Phoenix" on the Intel Download Center). If you install this tool on Vista or later, you will have to run Phoenix Bios Editor Pro as Administrator, and possibly in a 32 bit environment, for it to work.
  4. Once your BIOS/UEFI file is modified according to your needs, locate the Virtual Machine you plan to test your firmware with and copy your modified file into its directory. Then, edit the .vmx file to add one of these lines, according to the type of platform you want to run.
    • For BIOS, you only need to add:
      bios440.filename = "bios440.rom"
    • For 32-bit UEFI, you will need to have something like:
      firmware = "efi"
      efi32.filename = "efi32.rom"
    • For 64-bit UEFI, you will need:
      firmware = "efi"
      efi32.filename = "efi64.rom"
  5. Run the image, and it should use your modified firmware. If you have simply modded the original BIOS using Phoenix Bios Editor, a good way to confirm that the VM is using your custom BIOS is to change the 'Quiet Boot Logo' section, which contains the VMware logos you see during BIOS execution.
Also note that, as pointed out on this page, if you are usingVMWare Fusion, you may actually be able to extract the various embedded binaries using the -e switch (the page linked also provides a list of all the binaries you can extract). Sadly however, this switch doesn't apply for VMWare Player on Windows...

2011-05-05

Installing the drivers for an HP LaserJet 6P on Windows

From the time where printer manufacturers were still producing near indestructible models. Plus, 5 years on, and I'm still using the toner cartridge that came with the printer. Add a USB → IEEE1284 adapter, plug into a NAS or plug computer, and enjoy your network printer.

Now, I did spend some time getting cups going, but I found it a bit flaky, so I reverted to direct samba access to the raw device. But that means the need for a driver on Windows, and Windows 7 considers that the HP printing dinosaurs are too old to be supported by default. On the other hand, that probably means you'll be able to find a cheap one on ebay.

Now, for those of us who know better and still want to do our Windows printing through a trusted HP LaserJet, the procedure to obtain the drivers is as follows (should work for the whole range of old HP printers, and not just the 6P):

  1. Open IE (won't work in Firefox or Chrome)
  2. Go to http://catalog.update.microsoft.com
  3. Search your model, e.g. "LaserJet 6P". This should list a set of drivers for Windows 7, Windows Server 2008, etc. that are also of course compatible with Windows 10. Gotta wonder what the "search for drivers online" feature of Windows is really used for.
  4. Select the most relevant driver, e.g. "Microsoft driver update for HP LaserJet 6P"
  5. Click "View Basket"
  6. Click "Download"
  7. You'll get something like X86-all-XXXX.cab or AMD64-all-XXXX.cab file that you need to save. Be mindful that if you try to use the AMD64 package on a 32 bit version of Windows, or X86 on 64 bit, it will not work, so make sure you get the right one.
  8. Open the .cab file you just saved using 7-Zip, and extract all the files into some directory.
  9. Install your printer and select "Have Disk" when prompted for a driver. Then point to the directory where you extracted the files.
  10. Enjoy another many years use of your LaserJet printer. In all fairness, the early LaserJet printers are so indestructible that you'll probably die long before your printer does.

2011-04-08

Quick and dirty script to compile binutils with all targets

Yeah, yeah, I know you're not supposed to compile binutils in situ and whatnot, but I need a fully functional objdump/objcopy for testing stuff and I don't have all day.

The following script, when ran from the binutils-x.yz directory will compile binutils with all the targets. Worked 8 years ago, works today. Who'd want more?
#!/bin/sh

ROOT=`pwd`

for i in libiberty intl bfd opcodes binutils
do
cd $ROOT/$i
./configure --enable-targets=all --disable-nls
make -j4
done
You'll get a bunch of warnings about configure: WARNING: unrecognized options: --enable-targets, --disable-nls for some of the subprojects, but you can ignore them. Once you're done, you should have a nice objdump executable in the binutils/ directory, which, when poked with option -i should report something like this:
/usr/src/binutils-2.23.51-1/binutils/objdump: supported targets: pe-i386 a.out.adobe a.out-zero-big a.out-mips-little epoc-pe-arm-big epoc-pe-arm-little epoc-pei-arm-big epoc-pei-arm-little pe-arm-wince-big pe-arm-wince-little pei-arm-wince-big pei-arm-wince-little coff-arm-big coff-arm-little a.out-arm-netbsd pe-arm-big pe-arm-little pei-arm-big pei-arm-little b.out.big b.out.little elf32-avr elf32-bfin elf32-bfinfdpic elf32-big elf32-bigarc elf32-bigarm elf32-bigarm-symbian elf32-bigarm-vxworks elf32-bigmips elf32-bigmips-vxworks elf32-bigmoxie elf32-bignios2 elf32-cr16 elf32-cr16c elf32-cris elf32-crx elf32-d10v elf32-d30v elf32-dlx elf32-epiphany elf32-fr30 elf32-frv elf32-frvfdpic elf32-h8300 elf32-hppa-linux elf32-hppa-netbsd elf32-hppa elf32-i370 elf32-i386-freebsd elf32-i386-nacl elf32-i386-sol2 elf32-i386-vxworks elf32-i386 elf32-i860-little elf32-i860 elf32-i960 elf32-ip2k elf32-iq2000 elf32-lm32 elf32-little elf32-littlearc elf32-littlearm elf32-littlearm-symbian elf32-littlearm-vxworks elf32-littlemips elf32-littlemips-vxworks elf32-littlemoxie elf32-littlenios2 elf32-m32c elf32-m32r elf32-m32rle elf32-m32r-linux elf32-m32rle-linux elf32-m68hc11 elf32-m68hc12 elf32-m68k elf32-m88k elf32-mcore-big elf32-mcore-little elf32-mep elf32-metag elf32-microblaze elf32-mn10200 elf32-mn10300 elf32-mt elf32-msp430 elf32-openrisc elf32-or32 elf32-pj elf32-pjl elf32-powerpc elf32-powerpc-vxworks elf32-powerpcle elf32-powerpc-freebsd elf32-rl78 elf32-rx-be elf32-rx-be-ns elf32-rx-le elf32-s390 elf32-sh elf32-shbig-fdpic elf32-shbig-linux elf32-sh-fdpic elf32-shl elf32-shl-symbian elf32-sh-linux elf32-shl-nbsd elf32-shl-vxworks elf32-sh-nbsd elf32-sh-vxworks elf32-sparc elf32-sparc-sol2 elf32-sparc-vxworks elf32-spu elf32-tic6x-be elf32-tic6x-le elf32-tilegx-be elf32-tilegx-le elf32-tilepro elf32-tradbigmips elf32-tradlittlemips elf32-tradbigmips-freebsd elf32-tradlittlemips-freebsd elf32-us-cris elf32-v850 elf32-v850-rh850 elf32-vax elf32-xc16x elf32-xgate elf32-xstormy16 elf32-xtensa-be elf32-xtensa-le pe-powerpc pei-powerpc pe-powerpcle pei-powerpcle a.out-cris ecoff-bigmips ecoff-biglittlemips ecoff-littlemips coff-go32 coff-go32-exe coff-h8300 coff-h8500 a.out-hp300hpux a.out-i386 a.out-i386-bsd coff-i386 a.out-i386-freebsd a.out-i386-lynx coff-i386-lynx msdos a.out-i386-netbsd i386os9k pei-i386 coff-i860 coff-Intel-big coff-Intel-little ieee coff-m68k coff-m68k-un a.out-m68k-netbsd coff-m68k-sysv coff-m88kbcs a.out-m88k-mach3 a.out-m88k-openbsd mach-o-be mach-o-le mach-o-fat mach-o-i386 pe-mcore-big pe-mcore-little pei-mcore-big pei-mcore-little pe-mips pei-mips a.out-newsos3 nlm32-i386 nlm32-powerpc nlm32-sparc coff-or32-big a.out-pc532-mach a.out-ns32k-netbsd a.out-pdp11 pef pef-xlib ppcboot aixcoff-rs6000 coff-sh-small coff-sh coff-shl-small coff-shl pe-shl pei-shl som coff-sparc a.out-sparc-little a.out-sparc-linux a.out-sparc-lynx coff-sparc-lynx a.out-sparc-netbsd a.out-sunos-big sym a.out-tic30 coff-tic30 coff0-beh-c54x coff0-c54x coff1-beh-c54x coff1-c54x coff2-beh-c54x coff2-c54x coff-tic80 a.out-vax-bsd a.out-vax-netbsd a.out-vax1k-netbsd versados vms-libtxt coff-w65 coff-we32k coff-z80 coff-z8k elf32-am33lin srec symbolsrec verilog tekhex binary ihex
/usr/src/binutils-2.23.51-1/binutils/objdump: supported architectures: aarch64 alpha alpha:ev4 alpha:ev5 alpha:ev6 arc arc5 base arc6 arc7 arc8 arm armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5t armv5te xscale ep9312 iwmmxt iwmmxt2 avr avr:1 avr:2 avr:25 avr:3 avr:31 avr:35 avr:4 avr:5 avr:51 avr:6 avr:101 avr:102 avr:103 avr:104 avr:105 avr:106 avr:107 bfin cr16 cr16c cris crisv32 cris:common_v10_v32 crx d10v d10v:ts2 d10v:ts3 d30v dlx epiphany32 epiphany16 fr30 frv tomcat simple fr550 fr500 fr450 fr400 fr300 h8300 h8300h h8300s h8300hn h8300sn h8300sx h8300sxn h8500 hppa1.1 hppa2.0w hppa2.0 hppa1.0 i370:common i370:360 i370:370 i386 i386:x86-64 i386:x64-32 i8086 i386:intel i386:x86-64:intel i386:x64-32:intel i860 i960:core i960:ka_sa i960:kb_sb i960:mc i960:xa i960:ca i960:jx i960:hx ia64-elf64 ia64-elf32 ip2022ext ip2022 iq2000 iq10 k1om k1om:intel l1om l1om:intel lm32 m16c m32c m32r m32rx m32r2 m68hc11 m68hc12 m68hc12 m9s12x m9s12xg m68k m68k:68000m68k:68008 m68k:68010 m68k:68020 m68k:68030 m68k:68040 m68k:68060 m68k:cpu32 m68k:fido m68k:isa-a:nodiv m68k:isa-a m68k:isa-a:mac m68k:isa-a:emac m68k:isa-aplus m68k:isa-aplus:mac m68k:isa-aplus:emac m68k:isa-b:nousp m68k:isa-b:nousp:mac m68k:isa-b:nousp:emac m68k:isa-b m68k:isa-b:mac m68k:isa-b:emac m68k:isa-b:float m68k:isa-b:float:mac m68k:isa-b:float:emac m68k:isa-c m68k:isa-c:mac m68k:isa-c:emac m68k:isa-c:nodiv m68k:isa-c:nodiv:mac m68k:isa-c:nodiv:emac m68k:5200 m68k:5206e m68k:5307 m68k:5407 m68k:528x m68k:521x m68k:5249 m68k:547x m68k:548x m68k:cfv4e m88k:88100 MCore mep h1 c5 metag MicroBlaze mips mips:3000 mips:3900 mips:4000 mips:4010 mips:4100 mips:4111 mips:4120 mips:4300 mips:4400 mips:4600 mips:4650 mips:5000 mips:5400 mips:5500 mips:5900 mips:6000 mips:7000 mips:8000 mips:9000 mips:10000 mips:12000 mips:14000 mips:16000 mips:16 mips:mips5 mips:isa32 mips:isa32r2 mips:isa64 mips:isa64r2 mips:sb1 mips:loongson_2e mips:loongson_2f mips:loongson_3a mips:octeon mips:octeon+ mips:octeon2 mips:xlr mips:micromips mmix mn10200 mn10300 am33 am33-2 moxie msp:14 msp:11 msp:110 msp:12 msp:13 msp:14msp:15 msp:16 msp:21 msp:31 msp:32 msp:33 msp:41 msp:42 msp:43 msp:44 ms1 ms1-003 ms2 nios2 ns32k:32032 ns32k:32532 openrisc or32 pdp11 powerpc:common powerpc:common64 powerpc:603 powerpc:EC603e powerpc:604 powerpc:403 powerpc:601 powerpc:620 powerpc:630 powerpc:a35 powerpc:rs64ii powerpc:rs64iii powerpc:7400 powerpc:e500 powerpc:e500mc powerpc:e500mc64 powerpc:MPC8XX powerpc:750 powerpc:titan powerpc:vle powerpc:e5500 powerpc:e6500 rs6000:6000 rs6000:rs1 rs6000:rsc rs6000:rs2 rl78 rx rx s390:31-bit s390:64-bit score7 score3 sh sh2 sh2e sh-dsp sh3 sh3-nommu sh3-dsp sh3e sh4 sh4a sh4al-dsp sh4-nofpu sh4-nommu-nofpu sh4a-nofpu sh2a sh2a-nofpu sh2a-nofpu-or-sh4-nommu-nofpu sh2a-nofpu-or-sh3-nommu sh2a-or-sh4 sh2a-or-sh3e sh5 sparc sparc:sparclet sparc:sparclite sparc:v8plus sparc:v8plusa sparc:sparclite_le sparc:v9 sparc:v9a sparc:v8plusb sparc:v9b spu:256K tms320c30 tms320c4x tms320c3x tms320c54x tic6x tic80 tilegx tilegx32 tilepro v850 (using old gcc ABI) v850e3v5 (using old gcc ABI) v850e2v4 (using old gcc ABI) v850e2v3 (using old gcc ABI) v850e2 (using old gcc ABI) v850e1 (using old gcc ABI) v850e (using old gcc ABI) v850-rh850 v850e3v5 v850e2v4 v850e2v3 v850e2 v850e1 v850e vax w65 we32k:32000 xstormy16 xtensa xc16x xc16xl xc16xs xgate z80-any z80-strict z80 z80-full r800 z8001 z8002

2011-04-06

32 bit apps, Windows x64 and the System32 directory

While adding a hidden magic key feature to Zadig, that deletes the libusb-1.0 DLLs that are installed by the libusbK driver in System32\ and SysWOW64\ (when you are developing for both libwdi and libusb, the multiplication of libusb DLLs becomes a major issue), I found that a 32 bit Zadig application running on x64 Windows would not see the libusb-1.0.dll residing in System32\.

The reason for that is that, from a 32 bit app perspective, Windows maps SysWOW64\ (the directory that, as its name does NOT indicate, contains 32 bit binaries) to System32\, and therefore trying to access the actual System32\ (the directory that contains 64 bit binaries) using %WINDIR%\System32.

If for any reason you want to access the actual System32\ directory from a 32 bit application running on Windows x64, you have to use Sysnative (eg: C:\Windows\Sysnative).

2011-04-01

Enabling the Issuer Statement button on a Windows certificate

You may have seen a few of these certificates where clicking on the "Issuer Statement" button brings you to a specific webpage, usually a "Certification Practice Statement", or CPS, on the issuer's webserver. Maybe you've wondered how you could achieve the same on certificates you generate? Well, wonder no more.

It basically all boils down to adding a CPS Policy Identifier in the Certificate Policies field, and the Windows PKI API provides everything you need to do so:

  DWORD dwSize;
  CERT_POLICY_QUALIFIER_INFO certPolicyQualifier;
  CERT_POLICY_INFO certPolicyInfo = { "1.3.6.1.5.5.7.2.1", 1, &certPolicyQualifier };
  CERT_POLICIES_INFO certPolicyInfoArray = { 1, &certPolicyInfo };
  CHAR szCPSName[] = "http://cps-page.yourserver.com";
  CERT_NAME_VALUE certCPSValue;
  CERT_EXTENSION certExtension;
  CERT_EXTENSIONS certExtensionsArray = { 1, &certExtension };

  // Set the CPS Certificate Policies field - this enables the "Issuer Statement" button on the cert
  certCPSValue.dwValueType = CERT_RDN_IA5_STRING;
  certCPSValue.Value.cbData = sizeof(szCPSName);
  certCPSValue.Value.pbData = (BYTE*)szCPSName;
  if ( (!CryptEncodeObject(X509_ASN_ENCODING, X509_NAME_VALUE, (LPVOID)&certCPSValue, NULL, &dwSize))
    || ((pbCPSNotice = (BYTE*)malloc(dwSize)) == NULL)
    || (!CryptEncodeObject(X509_ASN_ENCODING, X509_NAME_VALUE, (LPVOID)&certCPSValue, pbCPSNotice, &dwSize)) ) {
    printf("could not setup CPS\n");
    goto out;
  }

  certPolicyQualifier.pszPolicyQualifierId = szOID_PKIX_POLICY_QUALIFIER_CPS;
  certPolicyQualifier.Qualifier.cbData = dwSize;
  certPolicyQualifier.Qualifier.pbData = pbCPSNotice;
  if ( (!CryptEncodeObject(X509_ASN_ENCODING, X509_CERT_POLICIES, (LPVOID)&certPolicyInfoArray, NULL, &dwSize))
    || ((pbPolicyInfo = (BYTE*)malloc(dwSize)) == NULL)
    || (!CryptEncodeObject(X509_ASN_ENCODING, X509_CERT_POLICIES, (LPVOID)&certPolicyInfoArray, pbPolicyInfo, &dwSize)) ) {
    printf("could not setup Certificate Policies\n");
    goto out;
  }

  certExtension.pszObjId = szOID_CERT_POLICIES;
  certExtension.fCritical = FALSE;
  certExtension.Value.cbData = dwSize;
  certExtension.Value.pbData = pbPolicyInfo;

  // Call CertCreateSelfSignCertificate() with certExtensionsArray as the last parameter or something...
  (...)

out:
  // Cleanup...
  if (pbCPSNotice != NULL) free(pbCPSNotice);
  if (pbPolicyInfo != NULL) free(pbPolicyInfo);
For a real-life example of the above, as well as the setting of other certificate properties such as the Enhanced Key Usage or an URL in the Subject Alternative Name, you may want to have a look at the LGPL pki.c from libwdi, and our CreateSelfSignedCert() function there.

Oh, and we pretty much rewrote basic LPGL redistributable versions of the MakeCat/Inf2Cat, MakeCert, CertMgr and Signtool WDK utilities, so if you need them, there's here as well. For additional details, look here

2011-03-24

Error 800B0003 - The form specified for the subject is not one supported or known by the specified trust provider

You're trying to digitally sign a file, aren't you?
And until this post, you weren't aware that only a specific set of files, such as executables, DLLs or cat files could be signed on Windows, but that random blurb, such as text files couldn't.

Well, now you know. If you see this error, it simply means that Windows does not recognize the input file as something it is able to affix a digital signature on.

2011-03-21

Programatically setting and applying Local Group Policies on Windows

One of the most annoying aspects of device driver installation on Windows, and one of the main gripe people are expected to have with the current libwdi (which has since been fixed programmatically using the technique exposed below), is the Windows default of creating a system restore point each time it installs a new driver. Having seen restore point creations that take more than 5 minutes, and ending up in failure as a result, since 5 minutes is the default timeout for a point creation, I have been looking at ways to change this default behaviour.

Enters the Local Group Policy. If you're lucky enough to run a version of Windows that enables it, you might be aware of the gpedit mmc tool (launch the command gpedit.msc to access it), which can be used to alter various Windows settings, a.k.a. Group Policies. In general, Group Policies are settings that are applied by Active Directory administrators in a networked environment, but the non networked part, called Local Group Policy can also be used by the end users to control the behaviour of their local machine or user account, should the network administrator decide not to override those. Of special interest to us is the Administrative Templates → System → Device Installation → "Prevent creation of a system restore point during device activity..." setting, which you can access after launching gpedit.msc. It works, and is exactly what we have been looking for.

But then again, we don't want to rely on end users to change this setting themselves, especially if they are running the Basic or Home edition of Windows 7 or Vista where gpedit is not available. We want to do it programmatically.

If you use the RegFormApp tool from NirSoft, to monitor what gpedit actually does when enabling this setting, you find that it simply creates a new DisableSystemRestore DWORD in the registry at:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Group Policy Objects\{<some-guid>}Machine\Software\Policies\Microsoft\Windows\DeviceInstall\Settings

Brilliant. The much decried Registry might actually prove its worth, if all you have to do to toggle system restore point creation on driver installation, is modify a key there... except, we're talking about a Microsoft way of implementing a solution, and it would be way too simple if Windows' own Local Group Policy settings were stored in the registry. The key created by gpedit there is in fact useless for the programmer, because it's just a some temporary storage for gpedit and not at all where the system looks when deciding whether it should create a restore point. As a matter of fact, the GUID changes every time you run gpedit.

So now we need to figure out a way to set our Local Group Policy programmatically, as gpedit does, and you can trust that it's not going to be as simple as what we were hoping for. Thankfully, if you look hard enough, you may end up on the Microsoft's USGCB Tech Blog page dealing with Utilities for automating Local Group Policy management. These tools are precisely the kind of sample we are looking for, as the ImportRegPol there can pick up a .pol file and apply the LPGO associated with it, and they come with the source.

To cut a long story short, you will need to use an IGroupPolicyObject Interface object, and below is the C source (with no error checking whatsoever) that one can use to disable the creation of a restore point during driver installation on Windows Vista and Windows 7:
#include <gpedit.h>

DWORD val, val_size=sizeof(DWORD);
HRESULT hr;
IGroupPolicyObject* pLGPO;
HKEY machine_key, dsrkey;
// MSVC is finicky about these ones => redefine them
const IID my_IID_IGroupPolicyObject = 
 { 0xea502723, 0xa23d, 0x11d1, {0xa7, 0xd3, 0x0, 0x0, 0xf8, 0x75, 0x71, 0xe3} };
const IID my_CLSID_GroupPolicyObject = 
 { 0xea502722, 0xa23d, 0x11d1, {0xa7, 0xd3, 0x0, 0x0, 0xf8, 0x75, 0x71, 0xe3} };
GUID ext_guid = REGISTRY_EXTENSION_GUID;
// This next one can be any GUID you want
GUID snap_guid = { 0x3d271cfc, 0x2bc6, 0x4ac2, {0xb6, 0x33, 0x3b, 0xdf, 0xf5, 0xbd, 0xab, 0x2a} };

// Create an instance of the IGroupPolicyObject class
CoInitializeEx(NULL, COINIT_APARTMENTTHREADED);
CoCreateInstance(&my_CLSID_GroupPolicyObject, NULL, CLSCTX_INPROC_SERVER,
 &my_IID_IGroupPolicyObject, (LPVOID*)&pLGPO);

// We need the machine LGPO (if C++, no need to go through the lpVtbl table)
pLGPO->lpVtbl->OpenLocalMachineGPO(pLGPO, GPO_OPEN_LOAD_REGISTRY);
pLGPO->lpVtbl->GetRegistryKey(pLGPO, GPO_SECTION_MACHINE, &machine_key);

// The disable System Restore is a DWORD value of Policies\Microsoft\Windows\DeviceInstall\Settings
RegCreateKeyEx(machine_key, "Software\\Policies\\Microsoft\\Windows\\DeviceInstall\\Settings",
 0, NULL, 0, KEY_SET_VALUE | KEY_QUERY_VALUE, NULL, &dsrkey, NULL);
 
// Create the value
val = 1;
RegSetKeyValue(dsrkey, NULL, "DisableSystemRestore", REG_DWORD, &val, sizeof(val));
RegCloseKey(dsrkey);

// Apply policy and free resources
pLGPO->lpVtbl->Save(pLGPO, TRUE, TRUE, &ext_guid, &snap_guid);
RegCloseKey(machine_key);
pLGPO->lpVtbl->Release(pLGPO);
Disabling of system restore during driver installation - coming soon to a libwdi powered application near you!

2011-03-15

AVR development with LUFA - TILTHW

  1. If you're going to switch between .USBSpecification = VERSION_BCD(01.10) and .USBSpecification = VERSION_BCD(02.00) and you're using a specific Windows driver (i.e. not HID, Mass Storage, etc.), then your device will only get installed properly until you reinstall that driver. Windows does not like having the USB specs of an existing device changed.

  2. The API changes quite a lot from release to release. Some would make a big deal out of it, but it really isn't. Just be careful that if you see an old LUFA guide, some of its calls might be obsolete. For instance some calls like EVENT_USB_Device_UnhandledControlRequest have been obsoleted.

  3. Stream serial debug output (printf) is easy once
    • you use the following in your source:
      #include <LUFA/Drivers/Peripheral/Serial.h>
      (...)
          Serial_Init(115200, true);
          Serial_CreateStream(NULL);
    • you make sure that $(LUFA_SRC_SERIAL) is referenced in your makefile sources (SRC = ...) along with $(LUFA_SRC_USB), as it is not included automatically.

  4. If you're getting undefined references to Endpoint_Read_Stream_LE and friends, you probably have LUFA_OPTS += -D CONTROL_ONLY_DEVICE defined in your makefile. Comment it out.

  5. On an AVR90USB162, there's only so much space that can be used for printf strings apparently, regardless of how much flash is being used. If you have too many serial debug statements, you may find that serial output is garbled until you eliminate some of them. Make sure you only use serial debug sparingly.

  6. EVENT_USB_Device_ConfigurationChanged() is NOT a good place to do serial debug output statement.

  7. If you are using multiple interfaces on Windows, you DO want to add a delay in EVENT_USB_Device_ConfigurationChanged()

  8. Use NULL as last parameter in the EP R/W calls (eg. Endpoint_Read_Stream_LE), unless you want to open a big bag of hurt!

  9. If you're using the same endpoint number for IN and OUT (eg. 0x01 & 0x81), you need to use Endpoint_SetEndpointDirection() to toggle the direction after selecting the EP. Oh, and don't use ENDPOINT_DESCRIPTOR_DIR_IN or ENDPOINT_DESCRIPTOR_DIR_OUT, use ENDPOINT_DIR_IN and ENDPOINT_DIR_OUT instead.

2011-03-14

Thunderbird and inline patch attachments

If you're running Thunderbird, depending on how you ever dealt with some types of attachments in the past, there is a chance that, according to how you told Thunderbird to handle some of the types of attachments, you now get an undesirable attachment behaviour.

For instance, if you ever told Thunderbird how it should open a .patch file (eg. with TortoiseGit), you might find that, patches you send are no longer sent as inline text (text/plain), but instead are sent as non-inline (application/force-download). This can be a problem for people reading those patches with dumb e-mail clients, so you might be looking for a way to restore the inline behaviour.

Well, this is usually all handled in the mimeTypes.rdf in your profile. For instance, to get a .patch file to always be sent as inline, you could ensure that you have the following entry:
  <RDF:Description RDF:about="urn:mimetype:text/plain"
                   NC:fileExtensions="patch"
                   NC:description="Patch File"
                   NC:value="text/plain"
                   NC:editable="true">
    <NC:handlerProp RDF:resource="urn:mimetype:handler:text/plain"/>
  </RDF:Description>

In case this isn't enough, another parameter which you want to check is Tools → Advanced → Options → General Tab → Config Editor and then look for mail.content_disposition_type. From what I understand, a value of 0 means that, anything that is not overridden in mimeTypes.rdf will be sent as inline. A value of 1 means that everything is sent as attachment, and a value of 2 means that everything, except images, is sent as attachment, so you probably want to have 0 there.

2011-03-13

Troubleshooting USB device development...

...or how I learned to stop worrying and love Windows' Code 10 ("This device cannot start.")

Let's face it, when it comes to USB device development, the USB stack on Windows is a lot more unforgiving than its Linux counterpart. Now, the problem of course is, Windows being Windows, it does its hardest at hiding what the technical details of a problem are, but instead will spew generic error codes that are only just a notch above useless.

Today's error: the dreaded "This device cannot start. (Code 10)" from the device manager (or CM_PROB_FAILED_START if you're using a WDK related app)  whenever it sees an USB device that it doesn't like. As Tim Roberts puts it here,
CM_PROB_FAILED_START means that your driver loaded, passed through DriverEntry, passed through AddDevice, and then failed during IRP_MN_START_DEVICE.
In layman's terms, this means that the issue occurs during the USB transfers that Windows issues with the device, after the driver has been setup and the device instance has been (re)created. Indeed, Windows does not simply stop communicating with the USB device once it's been instantiated internally; there are a few additional commands that are issued, such as retrieving the String Descriptors, as well setting the Configuration or setting the interface, if Alt Interfaces are provided. Officially of course, Windows does not offer a user API to control the latter, but it still needs to issue these control commands to the device, in order to place it in a known state. And if your device firmware is improperly setup (eg. stalls on any of these commands), Windows will fail the device altogether.

So now of course the question becomes, how can I tell if any of the later Control Transfers are failing?


LogMan and NetMon

To do that, you want an USB analyzer (or you can try the USB consortium's USB Command Verifier, however, despite asking for $2000 a pop for a VID, the USB IF are cheapskate enough not to provide an app that comes with signed Windows drivers, so you have to run a pretty intrusive EHCI driver in test mode). If you're on a 64 bit system, or a recent version of Windows, and you want a free solution, this can be problematic, as SnoopyPro doesn't seem to work that well there. However, in recent versions of Windows, you can get some decent snooping of the USB transfers through NetMon and the CodePlex USB Parsers. The whole process is detailed in this post.

Once you have these tools, run the following from an elevated command prompt to start a trace (NB: you can dump hub traffic as well, as per the Microsoft post, but if you're troubleshooting a standard USB device, you probably don't care much about that part):
Logman start Usbtrace -p Microsoft-Windows-USB-USBPORT -ets -nb 128 640 -bs 128 -o dump.etl
Then just plug in your device, so that you get the Code 10 / CM_PROB_FAILED_START error, and then stop the trace with:
Logman stop Usbtrace -ets


Analysing the trace

And now you have a trace in NetMon. Now what? Well, a couple of things.
First, you might want to isolate the Control Transfers that deal with your device only. If you're using a device with VID 0x1234, this can be done as follows (There's probably a way to simplify this - haven't found it yet...):
USBPort.USBPORT_ETW_EVENT_DEVICE_INITIALIZE.fid_USBPORT_Device.idVendor == 0x1234 OR
USBPort.USBPORT_ETW_EVENT_DISPATCH_URB_FUNCTION_CONTROL_TRANSFER.fid_USBPORT_Device.idVendor == 0x1234 OR
USBPort.USBPORT_ETW_EVENT_COMPLETE_URB_FUNCTION_CONTROL_TRANSFER_DATA.fid_USBPORT_Device.idVendor == 0x1234 OR
USBPort.USBPORT_ETW_EVENT_DISPATCH_URB_FUNCTION_CONTROL_TRANSFER_EX.fid_USBPORT_Device.idVendor == 0x1234 OR
USBPort.USBPORT_ETW_EVENT_COMPLETE_URB_FUNCTION_CONTROL_TRANSFER_EX_DATA.fid_USBPORT_Device.idVendor == 0x1234 OR
USBPort.USBPORT_ETW_EVENT_COMPLETE_INTERNAL_URB_FUNCTION_CONTROL_TRANSFER.fid_USBPORT_Device.idVendor == 0x1234 OR
USBPort.USBPORT_ETW_EVENT_ENDPOINT_OPEN.fid_USBPORT_Device.idVendor == 0x1234
That's better. But more importantly, you want to check for errors, which could be done like this for a Pipe Stall:

USBPort.USBPORT_ETW_EVENT_COMPLETE_INTERNAL_URB_FUNCTION_CONTROL_TRANSFER.ControlTransfer.Urb.fid_URB_Hdr_Status == 0xc0000004

Or by the much more generic error filter from Load Filters -> Standard Filters -> USB -> USB Error Events, which translates to the following:
(USBPort AND NetEvent.Header.Descriptor.Opcode == 34) OR
(USBHub AND NetEvent.Header.Descriptor.Opcode == 11) OR
(NetEvent.Header.Descriptor.Level == 0x2) OR
(USBHub AND NetEvent.Header.Descriptor.Id == 210) OR
(Property.USBUrbStatus != "success" && Property.USBUrbStatus != "")

If you're lucky, then you might find that your device has raised something like USBD_STATUS_STALL_PID during one of the post Device_Initialize Control transfers. If so, then you're halfway there: find the corresponding section of code in your firmware, fix it, and Windows should be a lot happier about your device...

2011-03-11

Atmel AVR USB - Using LUFA with the AVR Studio 5 IDE

OK, so now we have our working makefile that builds our LUFA app using nothing but a DOS prompt and the AVR Studio 5 toolchain.

But how about being able to do both the development and compilation from the AVR Studio UI, to have something that resembles a serious development environment at last?

Supposedly the only step, since Atmel were kind enough to provide a conversion tool in AVRS5, is to convert the .aps file we got from LUFA and use the result. I'm still going to use the Joystick demo from the previous post to demonstrate that, since its makefile was fixed to work with AVRS5.


idiotic idiosyncrasies

So File -> Import AVR Studio 4 Project, then browse to the Demos\Device\ClassDriver\Joystick\Joystick.aps.
Now, if you don't select the right directory, Microsoft (this one's an MS idiotic idiosyncrasy, as they do the same in regular Visual Studio) will duplicate the files into a new location, instead of simply reusing the existing one. There's enough data duplication in the world already, so let's avoid that. The original source location is the location we want to use for the project.
So you thought that selecting the Demos\Device\ClassDriver\Joystick\ directory for the Solution Folder would do the trick? Wrong. Whatever you pick, Microsoft will create an extra subdirectory using the solution name in there, so if you use Demos\Device\ClassDriver\Joystick\, all the files will be duplicated into Demos\Device\ClassDriver\Joystick\Joystick\.
No bother then, you say, I'll just use Demos\Device\ClassDriver\ and AVR Studio will then pick up the existing Joystick\ directory and reuse the original files... My young little Padawan, you have a lot to learn about how people are hell bent on making sure convenience is unreachable in this world!

If you try that, you'll get the error:
Project Import Failed : The process cannot access the file 'D:\lufa-lib\Demos\Device\ClassDriver\Joystick\makefile' because it is being used by another process. [Activity : Converting AVR Studio 4 ExternalMakeFileSettings]

Yup, the whole process fails because the file that the process wants to modify is being accessed by the process itself. The thing is, either Atmel or MS is so adamant about duplicating files all over the place, for no good reason, that they haven't considered this really wild idea that someone would ever want not to duplicate the bloody file, and therefore fail to consider that if source and dest are the same, they shouldn't try to copy the files. This is so utterly idiotic!

The workaround? Actually use Demos\Device\ClassDriver\Joystick\ for the Solution Folder, so that the files are duplicated into a Joystick\ subdirectory, close AVR Studio, move the Joystick.avrgccproj file from Joystick\Joystick\ to Joystick\, and then manually edit the Joystick.avrsln file with a text editor and replace the "Joystick\Joystick.avrgccproj" reference there with "Joystick.avrgccproj".
Then delete the Joystick\ subdirectory, and you should be set as Nature (but not Microsoft) intended, i.e. with all of the solution, the project files and the source in the same single directory.

Once you've done that, you should be able to open your LUFA solution and get it to compile straight from AVRS5. Neat!

The only small issue that seems to exist is that when issuing "Rebuild Solution" in AVRS5, which calls "make clean all" behind the scenes, we run into the "opening dependency file .dep/Joystick.o.d: No such file or directory" error again. This seems to be a makefile issue, with the -include $(shell mkdir .dep) $(wildcard .dep/*) line being executed before the clean command, and thus not being able to re-create the .dep\ directory.

Atmel AVR USB - Starting with LUFA, using AVR Studio 5 and the commandline

That's right, we're going to do it the hard way (i.e. sans WinAVR) because we like it better when things don't work straight out of the box. If you don't want to patch stuff, just download WinAVR and be done with it.


I for one welcome our new LUFA overlords

LUFA is so easy to get started with (at least when using WinAVR), it should almost be illegal. I wish libusb would get there, but given our slow rate of getting anything that is convenient for users actually integrated, I doubt we ever will... Anyway, first order of the day is to get the latest LUFA, which we'll pick from SVN. Or you can just pick the latest zip archive from the official website but if you're using LUFA_101122 or earlier, please be aware that there is a bug with the Minimus implementation with regards to the HWB button. To clone from SVN, just have a look here.

Now, the interesting thing of course is, if you look in LUFA\Drivers\Board, you'll see that there is a MINIMUS directory. That means our current test board is supported straight out of the box. Yay!

There are a bunch of demos and projects as well, which is also where you measure the usefulness of a library (once again, unlike libusb, which chose to remove proposed samples, despite having a very limited supply of 'em included...). To test LUFA with the Minimus, we're going to pick the Joystick demo from Demos\Device\ClassDriver. Of course, we don't have a joystick on our board, and just one usable button, so that'll be a pretty limited "joystick", but Windows should still list the device, and if everything works as expected, we should be able to see our button presses in real-time in Windows device properties, so it's not that bad of a trade-off.


Minimus's minimal reconfiguration

The way you set things up for your target board in LUFA is by editing the relevant variables at the top of the makefile. If you go to the Demos\Device\ClassDriver\Joystick directory, you'll see that, apart from the source, you only have one makefile as well as Joystick.aps, which is the AVR Studio 4 project file that actually calls the makefile for compilation.

There's only very little we need to modify here. If you're using a Minimus board, here's what you want:
# MCU name
MCU = at90usb162

# Target architecture (see library "Board Types" documentation).
ARCH = AVR8

# Target board
BOARD = MINIMUS

# Processor frequency.
F_CPU = 16000000


Compiling with AVR Studio 5 from the commandline

As pointed out earlier, we're not going to install WinAVR, because that would be too easy. We got a perfectly good avr-gcc toolchain installed with AVR Studio 5, so we might as well use it. Oh, and we're not going to use the AVR Studio 4 → AVR Studio 5 conversion process on the .aps just yet, because:
  1. You might be interested in using AVR Studio 5 for compilation outside of the GUI
  2. We will run into issues with the default makefile, that are better fixed first from the commandline, before moving to AVR Studio
Using AVR Studio from DOS prompt is actually very easy. You just need to add the following to your PATH System environmental variable (provided you used the default path during AVR Studio installation:
C:\Program Files (x86)\Atmel\AVR Studio 5.0\extensions\Application\AVR Toolchain\bin;
Once that's done, just open a prompt and navigate to the Joystick directory. Then run make, and...
D:\lufa-lib\Demos\Device\ClassDriver\Joystick>make
The system cannot find the path specified.
The system cannot find the path specified.
ECHO is off.
-------- begin --------
avr-gcc (AVR_8_bit_GNU_Toolchain_3.2.0_253) 4.5.1
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The system cannot find the path specified.
The system cannot find the path specified.
-f was unexpected at this time.
make: *** [sizebefore] Error 255
Dang!

I'll just cut to the chase here:
The "The system cannot find the path specified." occur whenever a "/dev/null" redirection is done in the Makefile. As to the "-f was unexpected at this time" (now, aren't you glad you found this blog), it occurs from the if test -f $(TARGET).elf statement that is done in "sizebefore:". That's right Dorothy, we're not using a UNIX shell anymore...
A quick glance at the "sizebefore:" and "sizeafter:" statements tell us that they're completely optional. So let's just remove them, which means that you must change the line:
# Default target.
all: begin gccversion sizebefore build sizeafter end
into:
# Default target.
all: begin gccversion build end
As well as remove these references in the:
# Listing of phony targets.
.PHONY : all begin finish end sizebefore sizeafter gccversion \
section at the end of the file.


The Joystick's in the mind, man...

Let's try that again, shall we?
D:\lufa-lib\Demos\Device\ClassDriver\Joystick>make
The system cannot find the path specified.
The system cannot find the path specified.
ECHO is off.
-------- begin --------
avr-gcc (AVR_8_bit_GNU_Toolchain_3.2.0_253) 4.5.1
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The system cannot find the path specified.
The system cannot find the path specified.
-f was unexpected at this time.
make: *** [sizebefore] Error 255

D:\lufa-lib\Demos\Device\ClassDriver\Joystick>make
The system cannot find the path specified.
The system cannot find the path specified.
ECHO is off.
-------- begin --------
avr-gcc (AVR_8_bit_GNU_Toolchain_3.2.0_253) 4.5.1
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ECHO is off.
Compiling C: Joystick.c
avr-gcc -c -mmcu=at90usb162 -I. -gdwarf-2 -DF_CPU=16000000UL -DF_CLOCK=16000000UL -DBOARD=BOARD_MINIMUS -DARCH=ARCH_AVR8 -D USB_DEVICE_ONLY -D FIXED_CONTROL_ENDPOINT_SIZE=8 -D FIXED_NUM_CONFIGURATIONS=1 -D USE_FLASH_DESCRIPTORS -D USE_STATIC_OPTIONS="(USB_DEVICE_OPT_FULLSPEED | USB_OPT_REG_ENABLED | USB_OPT_AUTO_PLL)" -Os -funsigned-char -funsigned-bitfields -ffunction-sections -fno-inline-small-functions -fpack-struct -fshort-enums -fno-strict-aliasing -Wall -Wstrict-prototypes -Wa,-adhlns=./Joystick.lst -I../../../../ -std=c99 -MMD -MP -MF .dep/Joystick.o.d Joystick.c -o Joystick.o
In file included from Joystick.h:49:0,
                 from Joystick.c:37:
../../../../LUFA/Drivers/Board/Joystick.h:120:31: fatal error: Board/Joystick.h: No such file or directory
compilation terminated.
make: *** [Joystick.o] Error 1

That's a bit better. But yeah, the Board does indeed not define a joystick, so let's add a virtual one. The Joystick.h from LUFA\Drivers\Board\BUMBLEB will actually do just fine (it's using Port D but doesn't conflict with our LEDs or switches), so we just copy that file over to LUFA\Drivers\Board\MINIMUS. Ideally, you want to edit the __JOYSTICK_BUMBLEB_H__ macro in there to __JOYSTICK_MINIMUS_H__. Then you also need to edit LUFA\Drivers\Board\Joystick.h to add the lines:
        #elif (BOARD == BOARD_MINIMUS)
            #include "MINIMUS/Joystick.h"
Alrighty, let's try again...


"Come on makefile, you used to be cool!"
D:\lufa-lib\Demos\Device\ClassDriver\Joystick>make
The system cannot find the path specified.
The system cannot find the path specified.
ECHO is off.
-------- begin --------
avr-gcc (AVR_8_bit_GNU_Toolchain_3.2.0_253) 4.5.1
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

ECHO is off.
Compiling C: Joystick.c
avr-gcc -c -mmcu=at90usb162 -I. -gdwarf-2 -DF_CPU=16000000UL -DF_CLOCK=16000000UL -DBOARD=BOARD_MINIMUS -DARCH=ARCH_AVR8 -D USB_DEVICE_ONLY -D FIXED_CONTROL_ENDPOINT_SIZE=8 -D FIXED_NUM_CONFIGURATIONS=1 -D USE_FLASH_DESCRIPTORS -D USE_STATIC_OPTIONS="(USB_DEVICE_OPT_FULLSPEED | USB_OPT_REG_ENABLED | USB_OPT_AUTO_PLL)" -Os -funsigned-char -funsigned-bitfields -ffunction-sections -fno-inline-small-functions -fpack-struct -fshort-enums -fno-strict-aliasing -Wall -Wstrict-prototypes -Wa,-adhlns=./Joystick.lst -I../../../../ -std=c99 -MMD -MP -MF .dep/Joystick.o.d Joystick.c -o Joystick.o
Joystick.c:188:1: fatal error: opening dependency file .dep/Joystick.o.d: No such file or directory
compilation terminated.
make: *** [Joystick.o] Error 1
Ah, that dreaded "fatal error: opening dependency file .dep/Joystick.o.d: No such file or directory" error...
Once again, to cut to the chase, you'll notice that we haven't fixed the "The system cannot find the path specified." /dev/null errors. Well, what do you know, the dependency problems actually comes from the .dep directory creation step not being able to execute because it references /dev/null.
The fix is simple (though output polluting): In the makefile, change the following:
# Create object files directory
$(shell mkdir $(OBJDIR) 2>/dev/null)

# Include the dependency files.
-include $(shell mkdir .dep 2>/dev/null) $(wildcard .dep/*)
into:
# Create object files directory
$(shell mkdir $(OBJDIR))

# Include the dependency files.
-include $(shell mkdir .dep) $(wildcard .dep/*)

Try again.
(...)
ECHO is off.
Creating load file for Flash: Joystick.hex
avr-objcopy -O ihex -R .eeprom -R .fuse -R .lock Joystick.elf Joystick.hex
ECHO is off.
Creating load file for EEPROM: Joystick.eep
avr-objcopy -j .eeprom --set-section-flags=.eeprom="alloc,load" \
        --change-section-lma .eeprom=0 --no-change-warnings -O ihex Joystick.elf Joystick.eep || exit 0
ECHO is off.
Creating Extended Listing: Joystick.lss
avr-objdump -h -S -z Joystick.elf > Joystick.lss
ECHO is off.
Creating Symbol Table: Joystick.sym
avr-nm -n Joystick.elf > Joystick.sym
--------  end  --------
Yes, we have our hex file!! The system works!
If you flash the hex, you should now get a nice little "LUFA Joystick Demo" in your devices, and if you go to the Game controller settings → Properties, you should be able to see the status of the HWB button in realtime.

Now, using AVR Studio 5 with LUFA wasn't that difficult after all was it?
In the next post, I'll show you how to run everything from the AVR Studio UI, so that you're all set for LUFA development.

2011-03-10

Atmel AVR USB - First app

Time to get our toolchain installed then. You'll see plenty of posts mentioning the need to install WinAVR around, but with the latest toolchain from Atmel, it looks like a separate installation of WinAVR has become a thing of the past, and we won't actually need to install anything but latest official Atmel toolchain, which is called AVR Studio.

Now, Atmel did the smart thing here: They picked up the Visual Studio 2010 base from Microsoft which they wrapped around avr-gcc and avr-libc. We've come a long way from substandard proprietary toolchains...

So off you go to download AVR Studio 5 (free, but registration required). Installation takes a little while but is a no brainer. Sadly, no Flip integration is provided yet, so we'll have to generate a .hex and still use Flip manually to transfer the files. On the other hand, it should be possible to port the libusb 1.0 and commandline based dfu-programmer to Windows, to provide a flashing tool that integrates with AVR Studio.


Creating a project for Minimus USB

Alrighty, let's create our first LED blinking app then. Creatively, I'm going to call it 'Blinky'.
In AVR Studio, click "New Project" and use the "AVR GCC", "Empty AVR GCC Project" template (which should be selected by default). The Minimus USB, or any AT90USB162 based board don't seem to exist in the Atmel/User Boards lists, so don't bother trying to pick a template there. Change the default Name to 'Blinky' and click OK.
On the next screen, in the Device Family, select "megaAVR, 8-bit", and pick up AT90USB162 (or any other Atmel chip you might be using). Click OK and you're done. AVR Studio will even create a source template for you, how nice!

One thing you'll notice when compiling a new project is that the default settings are not to generate a .hex. To fix that, simply right click on the project, in the solution explorer, and in the Build tab, on the "Generate Files" line make sure the ".hex" checkbox is set (for both Release and Debug).


Blinking code

Armed with the knowledge of where the LEDs and HWB button are connected (PD5 and PD6 for the LEDs, PD7 for HWB), let's write us some code to blink the LEDs. We might as well look at the samples for inspiration.

First thing we'll do, since we need to add some delay when toggling the LEDs, is use some of the handy functions from the avr-libc. util/delay.h provides a _delay_ms() call that should do the trick. The only thing is you need to define F_CPU to your board's frequency, before the #include, for the delay calls to be accurate. The Minimus USB uses a 16 MHz Xtal, therefore our blinking code can be:

#include <avr/io.h>
#define F_CPU 16.000E6
#include <util/delay.h>

void init_ports(void)
{
    PORTB = 0b00000000;
    DDRB = 0b00000000;

    PORTC = 0b00000000;
    DDRC = 0b00000000;

    PORTD = 0b00000000;
    DDRD = 0b01100000;      // Set LEDs as output
}

int main(void)
{
    init_ports();
    
    while(1)
    {
        if (PIND & 0b10000000) {
            PORTD = PORTD & ~0b01000000;
            PORTD = PORTD |  0b00100000;
            _delay_ms(200);
        
            PORTD = PORTD & ~0b00100000;
            PORTD = PORTD |  0b01000000;
            _delay_ms(200);
        } else {
            PORTD = PORTD | 0b01100000;
        }
    }
}


The (loose) PIND & 0b10000000 test is added to turn off both LEDS when HWB is pressed.This should compile alright.
Note that you might want to pay attention to the warnings generated when compiling (on the "Errors" tab), especially the one that says, when using delay.h in Debug mode:
"Compiler optimizations disabled; functions from won't work as designed". Using delay.h will be fine for Release, but if you find that your delays are off in Debug, don't be surprised.


That friggin' watchdog timer

OK then, let's use Flip to transfer the hex, then click "Start Application" and have a satisfied look at our red and blue LEDs alternating... except they don't! Only the red LED blinks, and definitely not at 200 ms. AVR Dude, what the hell?!?

Oh, and it gets better, if you unplug the board, and plug it back, then suddenly everything works. What's going on here?

Part of the answer can be found in the Flip user guide:


4. What is the difference between starting the application with the Reset box checked off or not?
• If you start the application with the Reset box checked, the device will start after a Watchdog reset and this watchdog will remain active. You need to disable the Watchdog in your application. Resetting the device, ensures there is no side effect from resources used by the boot-loader before starting the user application.

Lo and behold, if you uncheck the Reset box and reflash/restart the app, everything works as expected. Of course, if you're unaware that the Flip Reset box is the culprit, you're bound to run into this issue, eventually figure out that it has to do with the WatchDog, and want to disable it in your program, so we might as well take care of that little bugger.

Now, I found that the code samples from all the LED blinkers out there, that are supposed to disable the watchdog, as well as the watchdog disable code from the AT90USB162 datasheet didn't work at all. Even including <avr/wdt.h> and calling wdt_disable() alone didn't work (but at least you could do something like wdt_enable(WDTO_8S); to make the delay before reset much longer, in order to give you a chance to issue a wdt_reset() before it expires).
There have been a few discussions on the AVRFreaks forums related to this, and the first way I found to disable watchdog and make it work with a Minimus USB, for programs transferred using Flip with Reset enabled, was the one provided by curtvm's from this post.

If you include wd.h and call WD_DISABLE(), the Watchdog timer finally gets disabled. But then, looking at what the LUFA samples were doing, it appears that using avr/wdt.h also works, provided that you issue and MCUSR &= ~(1 << WDRF) before calling wdt_disable().
Thus, our final code, which now works regardless of whether the Watchdog is initially enabled or not, becomes:
#include <avr/io.h>
#define F_CPU 16.000E6
#include <util/delay.h>
#include <avr/wdt.h>

void init_ports(void)
{
    PORTB = 0b00000000;
    DDRB = 0b00000000;

    PORTC = 0b00000000;
    DDRC = 0b00000000;

    PORTD = 0b00000000;
    DDRD = 0b01100000;      // Set LEDs as output
}

int main(void)
{
    MCUSR &= ~(1 << WDRF);
    wdt_disable();
    init_ports();
    
    while(1)
    {
        if (PIND & 0b10000000) {
            PORTD = PORTD & ~0b01000000;
            PORTD = PORTD |  0b00100000;
            _delay_ms(200);
        
            PORTD = PORTD & ~0b00100000;
            PORTD = PORTD |  0b01000000;
            _delay_ms(200);
        } else {
            PORTD = PORTD | 0b01100000;
        }
    }
}

Tested and confirmed. Great, now we're all set to produce more interesting AVR USB apps, like ones using LUFA for instance...

Atmel AVR USB - Let's get cracking!

Recently purchased a Minimus USB clone as these have gotten relatively cheap thanks to all the current PS3 hacking (Disclaimer: I don't have a PS3 and don't care much about what other non-development use these boards are also being targeted at).

All this to say that getting an Atmel AT90USB162 based board, or similar, has now gotten cheap enough that you might as well get one, if you're planning to play with USB, using Atmel AVR chips.

The following post assumes that you're completely new to AVR development, and aims at helping you getting started. I'll assume the use of a Minimus USB board and Windows, for the rest of this guide.


Flip... flops?

Except for the bootloader, the flash should be pretty much blank on arrival, yet you probably found a test program that tests the LEDs or something, that you want to run to confirm that the board is good. There's an USB bootloader in there, to take care of the flashing, but to be able to use it you'll need the following:
  1. the flashing application itself, which is Java based and called "Flip"
  2. an USB driver for the application to use
If you head to the Atmel website you'll find the Flip 3.4.2 downloads. If you don't have Java installed already, you should obviously pick the one that has the JRE included, and install the application. That takes care of step 1, but we still need some drivers to have the app communicate with the device. These drivers are located in "C:\Program Files (x86)\Atmel\Flip 3.4.2\usb" (If you used the default installation path).
Now the good news here is that they are using the libusb-win32 Windows driver (smart choice!). The bad news however is that, as per Flip 3.4.2, they are not using the signed libusb0.sys drivers, or anything WHQL. Therefore, if you're running on Windows 7 or Vista x64, the driver installation will fail. Ouch!
Thankfully, you don't have to use the Atmel driver files from the Flip application, and the latest libusb-win32 ones, which are signed, will work just fine. The easiest way to install them then is to download the latest libusb-win32 binary release, extract the zip and then run the inf-wizard.exe you'll find in there, with your device plugged in. Or you can also download the latest Zadig.exe, select Options->Advanced Mode, and change the target driver from WinUSB. which is the default, to libusb0.

Once the driver is successfully installed, you can run Flip, and if you press Ctrl+U to bring the USB communications menu (or go to Settings -> Communication -> USB), you should now be able to open the device with Flip properly identifying your chip. Groovy.

I also should point out that a more detailed guide (with pictures) on how to use Flip with AT90USB chips is also available from Atmel.


Bootloader mode or not bootloader mode: that is the question

One item that might be of interest to you, if you're completely new to using bootloaders to reprogram the flash, is that once you successfully transfer an application to the flash and get it to run, the bootloader will obviously no longer be accessible, and if your app does not do anything with USB, it will be innaccessible from your computer.

So how do you get back to USB bootloader mode for reflashing? Simple:
Just make sure the HWB button is pressed when you press the RST button.

So in essence the method to enter the bootloader is 1. Press RST, 2. press HWB, 3. release RST, 4. release HWB - easy.
This of course assumes that the Hardware Boot Enable Fuse (HWBE) has been set on the board and that PD7 is connected to a HWB button. For more info, see chapter 23.5 of the AT90USB162 datasheet.


Is it Blinky? Or is it just Clyde?

"Enough of these preliminaries, let's get these LEDs blinking already!"

Alright, I hear ya. But if you assume that any LED blinking app listed on the Minimus USB website would work, you assumed wrong. The only one that actually successfully blinks the LEDs is the one you will find in the minimus_led_test.rar, and even that one, depending on how you run it, might not actually blink the LEDs because of how its program, but because of the watchdog timer (more about this in the next post).

So, provided you extracted the minimus_led_test.hex, and you have Flip in programming mode, the way to go is:
  1. File -> Load Hex file and pick up the .hex. The FLASH Buffer information will now be populated
  2. Click the 'Run' button, which will erase, program and verify the flash (default options). Note that if you try to flash without erasing first, you will get a "Device protection is set" error. You always need to erase the flash first
  3. Click the "Start Application" button
Your LEDs blinking yet? Good. Now let's get ourselves sorted to write our own LED blinker app.
We'll setup a toolchain that generates Intel .hex files, and then flash these files using the same method above.

2011-02-11

Denying non-fast-forward and SourceForge

Well, I have it from good source that deleting commits, on a public source, is actually fine if you have a good reason to do so. But what if that public source happens to be a git repositoty on SourceForge?

Let's say you wanted to push some code modifications, so you added a source (or so you thought), but realize after commit & push, that what you really added was the binary rather than the .c. Don't want your public repo to be polluted by unneeded binaries.

So you tried:
git reset --hard HEAD~1
git push origin +master:master
or
git commit --amend
but every time you push you get the error:
error: denying non-fast-forward refs/heads/master (you should pull first)
To ssh://USER@PROJECT.git.sourceforge.net/gitroot/PROJECT
 ! [remote rejected] master -> master (non-fast-forward)
error: failed to push some refs to 'ssh://USER@PROJECT.git.sourceforge.net/gitroot/PROJECT'
What gives?

A good search will of course lead you to ticket #9187 and receive.denyNonFastforwards being set to true by default on the remote git repository provided by SourceForge.

How can you edit your remote config to unset denyNonFastforwards? Easy:
ssh -t USER,PROJECT@shell.sourceforge.net create
[USER@shell-24010 ~]$ sf-help
(...) 
Your project DSCM repositories are here (use adminrepo for CVS and SVN):

    /home/scm_git/U/US/USER.u
    /home/scm_git/P/PR/PROJECT
(...)
[USER@shell-24010 ~]$ cd /home/scm_git/P/PR/PROJECT
[USER@shell-24010 PROJECT]$ ls -alFh
total 12K
drwxrwsr-x 3 dummy PROJECT   80 Nov 29 19:15 ./
drwxr-xr-x 3 root  root    4.0K Feb 10 21:58 ../
drwxrwsr-x 7 dummy PROJECT 1.0K Nov 29 19:15 PROJECT/
[USER@shell-24010 PROJECT]$ cd PROJECT
[USER@shell-24010 PROJECT]$ ls -alFh
total 48K
drwxrwsr-x   7 dummy PROJECT 1.0K Nov 29 19:15 ./
drwxrwsr-x   3 dummy PROJECT   80 Nov 29 19:15 ../
-rw-rw-r--   1 dummy PROJECT   23 Nov 29 19:15 HEAD
drwxrwsr-x   2 dummy PROJECT   80 Nov 29 19:15 branches/
-rw-rw-r--   1 dummy PROJECT  126 Nov 29 19:15 config
-rw-rw-r--   1 dummy PROJECT    5 Nov 29 19:15 description
drwxrwsr-x   2 dummy PROJECT 1.0K Nov 29 19:15 hooks/
drwxrwsr-x   2 dummy PROJECT   80 Nov 29 19:15 info/
drwxrwsr-x 133 dummy PROJECT 5.0K Feb 10 21:41 objects/
drwxrwsr-x   4 dummy PROJECT   80 Nov 29 19:15 refs/
Yay, there's a config there:
[USER@shell-24010 PROJECT]$ vi config
Then remove/unset that pesky denyNonFastforwards and you're good to go.