2019-07-24

Installing Debian ARM64 on a Raspberry Pi 3 in UEFI mode


That's right baby, we're talking vanilla Debian ARM64 (not Raspbian, not Armbian) in pure UEFI mode, where you'll get a GRUB UEFI prompt allowing you to change options at boot and everything.

At long last, the Raspberry Pi can be used to install vanilla GNU/Linux distributions in the same manner as you can do on a UEFI PC. Isn't that nice?


Not that I don't like Raspbian or Armbian (as a matter of fact I am impressed by the very fine job the Armbian maintainers are doing with their distro), but I have now spent enough time helping with the UEFI Raspberry Pi 3 effort not to push this whole endeavour to its logical conclusion: Install vanilla ARM64 GNU/Linux distros. That's because, in terms of long term support and features, nothing beats a vanilla distro. I mean, what's the point of having an 64-bit CPU if the distro you're going to install forces you to use 32-bit?

Prerequisites

Hardware:

  • A micro SD card with sufficient space (16 GB or more recommended). You may also manage with a USB Flash Drive, but this guide is geared primarily towards SD card installation
  • A Raspberry Pi 3 (Model B or Model B+) with a proper power source. If you're ever seeing a lightning bolt on the top left of your display during install, please invest into a power supply that can deliver more wattage.
Note that our goal here is to install the system on a SD card, through netinstall, using a single media for the whole installation process.

In other words, there is no need to use an additional USB Flash Drive, as we could do, to boot the Debian installer and then install from USB to SD. This is mostly because it's inconvenient to have to use two drives when one can most certainly do, and also because while USB to SD may look easier on paper (no need to fiddle with the "CD-ROM" device for instance) it's actually more difficult to complete properly.

Thus, while I'll give you some pointers on how to perform a USB based installation in Appendix D, I can also tell you, from experience, that you are better off not trying to use a separate USB as your installation media and instead performing the installation from a single SD, as described in this guide.

Software:

  • The latest Raspberry Pi 3 UEFI firmware binary, along with the relevant Broadcom bootloader support files (i.e. bootcode.bin, config.txt, fixup.dat, start.elf).

    You can find a ready-to-use archive with all of the above at https://github.com/pftf/RPi3/releases
    (RPi3_UEFI_Firmware_v#.##.zip, 3 MB).

    Note that this firmware archive works for both the Raspberry Pi 3 Model B and the Raspberry Pi 3 Model B+ (as the relevant Device Tree is automatically selected during boot).
  • (Optional) The non-free WLAN firmware binaries that are needed if you want to use Wifi for the installation.
    Note that, if you picked up the archive above then you don't need to do anything as the WLAN firmware binaries are included in it too.

Preparation


Note: a complete example of how to achieve the first 3 steps below using DISKPART on Windows or fdisk + mkfs on Linux is provided in Appendix A at the end of this post.
  • Partition your SD media as MBR and create a single partition of 300 MB of type 0x0e (FAT16 with LBA).
    Do not be tempted to use GPT as the partition scheme or 0xef (ESP) for the partition type, as the ondie Broadcom bootloader does not support any of those. It must be MBR and type 0x0e. You can use the command line utilities fdisk on Linux or DISKPART on Windows to do that.
  • Set the partition as active/bootable. This is very important as, otherwise, the Debian partition manager will not automatically detect it as ESP (EFI System Partition) which will create problems that you have to manually resolve (See Appendix C).
    If using fdisk on Linux, you can use a to set the partition as active.
    If using Windows, you can use DISKPART and then type the command active after selecting the relevant disk and partition.
  • Format the partition as FAT16. It MUST be FAT16 and not FAT32, as the Debian partition manager will not detect it as ESP otherwise and, again, you will have to perform extra steps to salvage your system before reboot (Appendix C).
    The Linux and Windows base utilities should be smart enough to use FAT16 and not FAT32 for a 300 MB partition, so you should simply be able to use mkfs.vfat /dev/<yourdevice> (Linux) or format fs=fat quick in Windows' DISKPART. The Windows Disk Manager should also be smart enough to use FAT16 instead of FAT32 if you decide to use it to format the partition.
  • Extract the UEFI bootloader support files mentioned above to the newly formatted FAT partition. If you downloaded the Raspberry Pi 3 UEFI firmware binary from the link above, you just have to uncompress the zip file onto the root of your media, and everything will be set as it should be.
  • Extract the content of the Debian ISO you downloaded to the root of the FAT partition. On Windows you can use a utility such as 7-zip to do just that (or you can mount the ISO in File Explorer then copy the files).
Once you have completed the steps above, eject your SD card, insert it in your Pi 3 and power it up. Make sure no other media is plugged in besides the SD card. Especially, make sure that there aren't any USB Flash Drives or USB HDDs connected.

Initial Boot


Unless you did something wrong, you should see the multicoloured boot screen, which indicates that the Raspberry Pi properly detected your SD media and is loading the low level CPU bootloader from it.

Then you should see the black and white Raspberry logo, which indicates that the Raspberry Pi UEFI firmware is running.



Wait for the GNU GRUB menu to appear (which it should do by default after the Pi logo disappears) and choose *Install (which should already be the default) and let the Debian installer process start.

Debian Installer


Note: In case anything goes wrong during install, remember that you can use Alt-F4 to check the current installation log for details about the error.
  • Select your Language, Country and Keyboard and let the installer proceed until it reports that No Common CD-ROM drive was detected.
  • At this stage, on Load CD-ROM drivers from removable media select No.
  • On Manually select a CD-ROM module and device select Yes.
  • On Module needed for accessing the CD-ROM select none.
  • On Device file for accessing the CD-ROM type exactly the following:

    -t vfat -o rw /dev/mmcblk0p1

    For the reasons why you need to type this, see Appendix B below.
  • With the "CD-ROM" device set, let the installation process proceed and retrieve the base packages from the media until it asks you for the non-free firmware files on the network hardware detection. If you plan to use the wired connection, you can skip the (Optional) step below.
  • (Optional) If you plan to use WLAN for the installation, choose Yes for Load missing firmware from removable media. If you created the media from that Raspberry Pi 3 firmware archive linked above, the relevant firmware files will be detected under the firmware/ directory.

    Note 1: Because there are multiple files to load, you will be prompted multiple times for different firmware files (look closely at their names, you will see that they are actually different). This is normal. Just select Yes for each new file.

    Note 2: Though they are included in the UEFI firmware zip archive we linked above, it is most likely okay not to provide the .clm_blob if you don't have it (the Wifi drivers should work without that file), so don't be afraid to select No here if needed.
  • Set up your network as requested by the installer by (optionally) choosing the network interface you want to use for installation and (also optionally) setting up your access point and credentials if you use Wifi.
  • Go through the hostname, user/password set up and customize those as you see fit.
  • Let the installer continue until you get to the Partition disks screen. There, for Partitioning method select Manual. You should see something like this:

    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
         #1  primary  314.6 MB  B  K  ESP
             pri/log                  FREE SPACE

    If, instead, you see something like this:

    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
         #1  primary  314.6 MB  B   fat16
             pri/log                FREE SPACE

    In other words, if you don't see B K ESP for the first partition, then it means that you didn't partition or format your drive as explained above and you will need to reference Appendix C (Help, I screwed up my partitioning!) to sort you out.
  • From there select the FREE SPACE partition and use the partition manager's menu to create two new primary partitions (one for swap and one for the root file system), until you have something like this:

    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
         #1  primary  314.6 MB  B  K  ESP
         #2  primary    1.0 GB     f  swap    swap
         #3  primary   14.7 GB     f  ext4    /
    
  • Select Finish partitioning and write changes to disk and then Yes on Write the changes to disks? and let the installer continue with the base system installation.
  • After a while, the installer will produce a big red ominous message that says:

    [!!] Configure the package manager
      
    apt-configuration problem
    An attempt to configure apt to install additional packages from the CD failed.

    This, however, is actually a completely benign message and you can safely ignore it by selecting Continue . That's because, since we are conducting a net install, we couldn't care less about no longer being to access the "CD-ROM" files after install...
  • Once you have dimissed the message above, pick the mirror closest to your geographical location and let the installer proceed with some more software installation (this time, the software will be picked from that network mirror rather than the media).
    When prompted for the "package usage survey" pick whichever option you like.
  • Finally, at the Software selection screen, select any additional software package you wish to install. Note that the "Debian desktop environment" should work out of the box if you decide to install it (though I have only tested Xfce so far). It's probably a good idea to install at least "SSH server".
  • Let the process finalize the software and GRUB bootloader installation and, provided you didn't screw up your partitioning (i.e. you saw B K ESP when you entered the partition manager, otherwise see Appendix C) select Continue to reboot your machine on the Installation complete prompt.

If everything worked properly, your system will now boot into your brand new vanilla Debian ARM64 system. Enjoy!

Post install fixes


Here are a few things that you might want to fix post install:
  1. You may find a cdrom0 drive on your desktop, which can't seem to be accessible. This is a leftover from the installer process not knowing how to handle the installation media device. You should edit /etc/fstab to remove it.
     
  2. If you installed the cups package, you may get an error while loading modules (systemctl --failed will report that systemd-modules-load.service is in failed state). This is all due to the current cups package trying to load IBM PC kernel modules... on a non PC device. To fix this, simply delete /etc/modules-load.d/cups-filters.conf and reboot.
  3. If using UEFI firmware v1.6 or later, you can enable the serial console by editing /etc/default/grub and changing GRUB_CMDLINE_LINUX="" to GRUB_CMDLINE_LINUX="console=ttyS0,115200", and then running update-grub.
    You may also enable serial console access for GRUB by adding the following in the same file:
    GRUB_TERMINAL=serial
    GRUB_SERIAL_COMMAND="serial --unit=0 --speed=115200 --stop=1" 

Appendix A: How to create and format the SD partition for installation


IMPORTANT NOTE 1: Please make sure to select the disk that matches your SD media before issuing any of these commands. Using the wrong disk will irremediably destroy your data!

IMPORTANT NOTE 2: Do not be tempted to "force" FAT32 in DISKPART or mkfs and do not forget to set the bootable/active flag, else you will afoul of the issue described in Appendix C. 

Windows

C:>diskpart

Microsoft DiskPart version 10.0.18362.1

Copyright (C) Microsoft Corporation.
On computer: ########

DISKPART> list disk

  Disk ###  Status         Size     Free     Dyn  Gpt
  --------  -------------  -------  -------  ---  ---
  Disk 0    Online          238 GB      0 B        *
  Disk 1    Online          465 GB  1024 KB        *
  Disk 4    Online         4657 GB  1024 KB        *
  Disk 5    Online         4649 GB      0 B        *
  Disk 6    Online           14 GB    14 GB

DISKPART> select disk 6

Disk 6 is now the selected disk.

DISKPART> clean

DiskPart succeeded in cleaning the disk.

DISKPART> convert mbr

DiskPart successfully converted the selected disk to MBR format.

DISKPART> create partition primary size=300

DiskPart succeeded in creating the specified partition.

DISKPART> active

DiskPart marked the current partition as active.

DISKPART> format fs=fat quick

  100 percent completed

DiskPart successfully formatted the volume.

DISKPART> exit

Leaving DiskPart...

C:>

Note, if needed you can also force a specific partition type (e.g. set id=0e to force FAT16 LBA), but that shouldn't be needed as DISKPART should set the appropriate type accordingly.

Linux


The following assumes /dev/sdf is your SD/MMC device. Change it in all the commands below to use your actual device.

(Optional) If your drive was partitioned as GPT, or if you're not sure, you may want to issue the two following commands first. If it's MBR you can skip this step:

# Delete the primary GPT:
dd if=/dev/zero of=/dev/sdf bs=512 count=34
# Delete the backup GPT.:
dd if=/dev/zero of=/dev/sdf bs=512 count=34 seek=$((`blockdev --getsz /dev/sdf` - 34))

Now use fdisk and mkfs to partition the drive:

root@debian:~# fdisk /dev/sdf

Welcome to fdisk (util-linux 2.34).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x7d188929.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-31291391, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-31291391, default 31291391): +300M

Created a new partition 1 of type 'Linux' and of size 300 MiB.

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): e
Changed type of partition 'Linux' to 'W95 FAT16 (LBA)'.

Command (m for help): a
Selected partition 1
The bootable flag on partition 1 is enabled now.

Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.

root@debian:~# mkfs.vfat -F 16 /dev/sdf1
mkfs.fat 4.1 (2017-01-24)
root@debian:~#

Appendix B: Why do we need to use -t vfat -o rw /dev/mmcblk0p1 as the CD-ROM device?

  • Why this weird device name with options? Because these are actually mount command line parameters and the Debian installer actually calls mount behind the scenes and feeds it exactly what we write here. This means we can hijack the device name field to invoke the additional mount parameters we need.
  • Why /dev/mmcblk0p1? That's simply name of the device for the first partition (p1) on the SD/MMC media (mmcblk0) as seen by the Linux kernel on a Raspberry Pi.
  • Why -t vfat? Because the Debian installer appends fstype=iso9660 to the mount option which prevents automount and forces us to override the file system type.
  • Why -o rw? Because the Debian installer won't be able to use the first partition for /boot/efi otherwise or load the WLAN firmware from the media (you get a device or resource busy when trying to remount the media).

Appendix C: Help I screwed up my partitioning!


Of course you did. You thought you knew better, and now you are paying the price...

The problem in  a nutshell is that:
  1. You can't use a regular ESP on a Raspberry Pi, on account that GPT or an MBR partition with type 0xef are not handled by the Broadcom CPU bootloader. And there is nothing you can do about this, because this is a behaviour that's hardcoded in the CPU silicon itself.
     
  2. The Debian installer's partition manager is very temperamental about what it will recognize as an ESP. In other words, if you don't use the perfect combination of boot flag, partition type and file system, it will fail to see it as an ESP.
Now the good news is that this is recoverable, but you need to know what you're doing.
  • The first thing you should do in the Debian partition manager is set the first partition to be used as ESP. In other words, you will need to edit the first partition until you get this:
    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
         #1  primary  314.6 MB  B  K  ESP
             pri/log                  FREE SPACE
  • Then you can proceed as the guide describe, but you need to bear in mind that, as soon as you choose to write the partition changes, the partition manager will have changed your first partition type to 0xef, which, as we have seen is ubootable by the CPU. Therefore, DO NOT PROCEED WITH THE SYSTEM REBOOT AT THE END UNTIL YOU HAVE CHANGED THE PARTITION TYPE BACK.
  • To do that, once you get to the Installation complete prompt that asks you to select Continue to reboot, you need to press Alt-F2 then Enter to activate a console.
  • Then type exactly the following command:
    chroot /target fdisk /dev/mmcblk0
    Then press the keys t, 1, e, w
  • No you can go back to the installer console (Alt-F1) and select Continue to reboot

Appendix D: Installing to a SD from an USB Flash Drive


As I explained above, and though it may seem simpler, I would discourage to use this method to install Debian on a Raspberry Pi. But I can understand that, if you don't have a card reader, you may be constrained to using this method.

For the most part, this should work fine out of the box. As a matter of fact, if you do it this way, you won't have to fiddle with the "CD-ROM" media detection. However, I will now list some of the caveat you'll face if you proceed like:

Caveat 1: If you use guided partitioning your SD/MMC media will be formatted as GPT (because this is a UEFI system after all) which the Broadcom CPU used in the Raspberry Pi can not boot. It has to be MBR. How you are supposed to force MBR over GPT in the Debian partition manager, I'll let you figure out.

Caveat 2: Similarly, you need to go through the 0xef to 0x0e conversion of your ESP, as the Pi won't boot from that partition otherwise.

Caveat 3: Of course you will also need to duplicate all the bootcode.bin, fixup.bat and so on from your USB boot media onto the SD ESP partition if you want it to boot (which is the reason why is is much more convenient to just set the ESP and Debian installer on the SD right of the bat, so you don't risk forgetting to copy a file).

Caveat 4: When I tried USB to SD install, I found that the GRUB installer somehow didn't seem to create an efi/boot/bootaa64.efi, which, if left uncorrected, will prevent the system from booting automatically.

43 comments:

  1. Thanks a lot for the clear guide!! I just finished with this on my Raspberry Pi 3b+ with a 128GB Sandisk SD. Everything just worked! I did not even get the red error about the issue with the CD.

    I chose XFCE as the desktop and it indeed booted straight into it.

    Some things I'm still missing:
    - accelerated graphics, the VC4 support is not there, OpenGL is rendered in software
    - audio, at least aplay doesn't work, maybe also related to missing VC4 drivers

    ReplyDelete
  2. Just loading the 'vc4' module, doesn't do much. Searching online I see references to a specific device tree overlay being required for that.

    The instructions online all suggest this can be done through config.txt by adding a line 'dtoverlay=vc4-fkms-v3d'. This would trigger Raspberry Pi's boot loader to add this to the Linux arguments so that it loads this overlay.

    Of course now this loader is loading the UEFI binary instead which loads Grub which loads Linux.

    So my suspicion would be that I would need to provide this vc4-fkms-v3d overlay as an argument through Grub and make sure the file itself is inside the initrd so it can be loaded at boot time.

    I searched around, but could not find any details on tweaking DT overlays in Debian or what arguments Linux itself uses to apply a DT overlay.

    Could you confirm whether my suspicion is correct? Any pointers to help me further would be really appreciated!

    ReplyDelete
    Replies
    1. DT overlays *are* supported by the UEFI firmware using the 'dtoverlay' parameter in 'config.txt'. But you also need to provide the device tree binary you want the overlay to apply on (i.e. you can't just provide the overlay). See https://github.com/tianocore/edk2-platforms/tree/master/Platform/RaspberryPi/RPi3#Custom-Device-Tree

      In other words, something like this should work in 'config.txt':

      device_tree_address=0x10000
      device_tree_end=0x20000
      device_tree=bcm2710-rpi-3-b-plus.dtb
      dtoverlay=vc4-fkms-v3d

      Of course, both the 'bcm2710-rpi-3-b-plus.dtb' and 'vc4-fkms-v3d' need to be present on your SD card as well.

      Doing this definitely seems to have an effect on my device as, with the above, I get a blank screen instead of the X environment on startup and I also see the following extra data in 'dmesg':

      [ +0.001566] [drm] Initialized vc4 0.0.0 20140616 for soc:gpu on minor 0
      [ +0.000011] [drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
      [ +0.000002] [drm] Driver supports precise vblank timestamp query.

      Hope that helps!

      Delete
    2. With dtoverlay=vc4-fkms-v3d,cma-256 I also get a blank screen. With dtoverlay=vc4-fkms-v3d,cma-256 it boots up with graphics, but USB does not work then.

      Delete
  3. Hi Pete,

    thank you for your guide. Debian 10 is running without any issues on a RPi 3 B.

    But anything related to GPIO is not. I had issues to install the raspi3-firmware package first. It expects a tmpfs in /boot/firmware which I added to /etc/fstab and mounted it.
    After that I could install it. What I expected are dtb files in /boot but not in a temporary mount point which is empty after reboot. Is this a bug or did I miss something?
    Additionally I do not understand the devicetree thing here: https://www.raspberrypi.org/documentation/configuration/device-tree.md because this seems to be completely different now.
    They are talking about tools like dtoverlay or dtparam commands which are not available in Debian buster. How do I get the device tree address amd the device tree end?
    The definition of a firmware in Raspbian speech was the bootloader, Kernel, firmware, X server which has changed now. I assume that only the .dtb files have to be moved to /boot or is it /boot/efi or /boot/efi/firmware?
    Previously I had following relevant entries in the /boot/config.txt:
    dtparam=i2c_arm=on
    dtparam=spi=on
    dtoverlay=w1-gpio
    dtparam=i2c=on

    This was previously nedded for 1-wire, OLED display and other hardware which are not working anymore. There is no /dev/i2c-1. GPIO is not working.
    I assume that the addresses are wrong for a bcm2710-rpi-3-b.dtb file?
    The current entries are:
    ...
    device_tree_address=0x10000
    device_tree_end=0x20000
    device_tree=bcm2710-rpi-3-b.dtb
    dtparam=spi=on
    dtparam=i2c=on
    dtparam=i2c_arm=on
    dtoverlay=i2c1
    dtoverlay=w1-gpio

    Maybe you could give me some hints?


    ReplyDelete
  4. 1. /boot/efi is what you need to use instead of /boot/firmware on a UEFI system, because this is where the ESP (EFI System Partition) is mounted. In our case, the ESP is the FAT Raspberry Pi boot partition, that contains the boot files, config.txt and (optionally) the .dtb's. Unfortunately, the raspi people have not yet acknowledged that not everybody may use /boot/firmware to mount the FAT partition (because, unlike what is the case for /boot/efi, there isn't really a de-facto convention that mandates the use of /boot/firmware, so they are taking a gamble that the files they need are always going to be there). But then again, I don't think you should be using raspi3-firmware, which clearly isn't designed for UEFI systems, when all you need is copy the files you need to /boot/efi.
    2. The device tree configuration and use of overlay hasn't changed. The UEFI firmware supports them, but you need to understand that it is all set up in the global config.txt. These commands (dtoverlay, dtparam) are NOT Linux commands, but they apply directly to the Broadcom VideoCore to let it know what components it should pick to create the DeviceTree binary that it will then pass to the Linux Kernel. I think you may want to review your understanding of Device Trees and how the Raspberry Pi is meant to use them, because it all needs to be set in config.txt (i.e. /boot/efi/config.txt in our case, but you can also edit this file independently since it resides on the root of the FAT partition), and the .dtb you need, along with any potential overlays, need to also exist in /boot/efi (i.e. the FAT partition). Then once you have that, you should be able to set everything as you see fit (no need to invoke raspi3-firmware).

    In other words, you will need to pick all the binary files you need from https://github.com/raspberrypi/firmware/tree/master/boot (including the overlays), copy them over to the FAT partition, either through /boot/efi or by mounting the SD card on another computer, and then you should be able to apply the settings you pointed out to your config.txt.

    ReplyDelete
  5. Hello Pete,

    thank you for your response.
    I aggree that there was a knowledge gap in UEFI and additionally in the dtoverlay stuff.
    The most things are clear now. What I`m missing is the address part of the device tree thing.
    This is something magic. Is there any good knowledge source out there to understand how to calculate that?
    Thank you in advance.

    Regards
    Ralf

    ReplyDelete
    Replies
    1. There's nothing magic. https://github.com/tianocore/edk2-platforms/tree/master/Platform/RaspberryPi/RPi3#Custom-Device-Tree tells you explicitly: "The address range must be [0x10000:0x20000]". So that's what you need to use. You don't have to "guess" anything here, just follow what every single example you've seen tells you to use.

      Delete
  6. Hi Pete,

    no chance...I followed all of your instructions, downloaded the firmware from github, cleaned up the unnecessary files.
    There is still no /dev/i2c/1 or /dev/i2c-1.
    It seems to be that there is still something missing which is unrelated to the firmware and bootloader config/modules in Debian 10. I definiteley have a RPi 3 B, config is correct, nothing interesting in dmesg...it is not working without any reason.

    #cat /proc/device-tree/model
    Chip is a bcm2835

    => Raspberry Pi 3 Model B, newer chinese hardware release with bcm2835

    Do you have the /dev/i2c* device available in your setup?
    Any further ideas how to debug that?

    This drives me crazy and as result I would have to go back to raspbian if this cannot be fixed in a acceptable time base. I would like to avoid that because Debian ist the better choice in the closed Broadcom world. At the moment I feel that I should throw all the Raspberries out of the window now and go the BeagleBone way, which is completely open and very well documented...

    Ralf

    ReplyDelete
    Replies
    1. The I2C devices are there:
      root@pi3:/proc/device-tree/soc# ls /proc/device-tree/soc/ | grep i2c
      i2c@7e205000
      i2c@7e804000
      i2c@7e805000

      Now, whether a *VANILLA* kernel will actually do anything with these (as opposed to the custom Raspbian kernel that is tailored for the platform) and create a device is another story. You may have to load kernel modules (if they exist for your kernel) to get the device nodes created. Or you may have to work with the Debian kernel maintainers so that they add support for the Raspberry Pi I2C in the *VANILLA* kernel. There seems to exist a i2c-bcm2835.ko kernel module on Debian, which I tried to load (along with i2c-dev) by adding an entry for those in /etc/modules, and while I can see them successfully loaded, they didn't seem to create the expected devices or anything under /sys/class/i2c-adapter/.

      Please bear in mind that the only reason the devices you want work pretty much out of the box under Raspbian is because people like you invested the time to make sure that they did, by ensuring that the kernel and distro have all the elements required. So, of course, if nobody wants to carry out the same work with mainline distros (which I would argue is what the Raspberry Pi Foundation should have done from the get go, instead of crafting their own distro, but that's another story), then you'll find that support for the hardware you want will continue to be subpar at best. Therefore I would strongly encourage you to work with Debian to figure out what the issue is and get it sorted. If not, then you're probably better off using Raspbian indeed...

      Delete
  7. Hi Pete,

    this is what I have done already with same result. It seems to be a kernel driver issue.
    Maybe there is a time slot to download both sources from Raspbian and Debian kernel and run a diff over both kernel drivers.

    Ralf

    ReplyDelete
  8. Replies
    1. No it won't. Since I am involved in the Pi 4 UEFI firmware development effort (which is one of the many elements required for a similar installation to work on the Pi 4) I will post an update on this blog about Debian Pi 4 installation in UEFI mode *WHEN* that becomes viable. But at the moment that is not possible due to multiple issues, such as Debian not integrating a working network driver for the Pi 4 adapter (requires this, which the Debian folks don't seem to want to integrate yet https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=950578), no SD card driver for ACPI mode (as far as I can tell, *NOBODY* is working on this right now), and other things so that Debian can access the full 4 GB of RAM for Pi 4's with 4 GB instead of being limited to 3 GB. You can find a Raspberry Pi 4 UEFI firmware at https://github.com/pftf/RPi4 but as CLEARLY mentioned in the README, it comes with the limitations mentioned above which makes it unsuitable to install Debian in the same manner as this guide's. I expect it will be MONTHS at best before I can update this guide for Pi 4. You can however help speed this up by telling the Debian maintainers that you need the Genet patch integrated in the next release of Debian, coz I don't think they've understood how critical it is to have it, since they could have integrated it for the version they released a couple weeks ago but didn't... As long as Debian doesn't have that network driver in their installation kernel, a guide such as the one above is NO_GO for the Raspberry Pi 4.

      Delete
    2. fedora 32 netinstall image has kernel 5.7 which already has bcmgenet
      i can install successfully with rpi4 eeprom 06/07

      Delete
  9. Hi,

    Thank you very much for this post. I waited very long hoping that some day this installation method would exist.

    I wanted to install from a usb key exactly as I do on my computer, it's just easier to not do any tricks with the installer. However, as I do with my PC, I always format the disk before installation. I will explain exactly what I did:

    1- Erase the SD card (sda is the SD card)
    sudo dd if=/dev/zero bs=1MiB status=progress of=/dev/sda

    2- Partition the SD:
    sudo fdisk /dev/sda

    Here I created a 1023 MiB partition of type 0x0c (FAT32), a partition of 32 GiB of default Linux type to be used as /, and a partition of 16 GiB of type 0x07 (NTFS) to be used as data.

    3- Format ESP
    sudo mkfs.fat -F32 -n ESP -v /dev/sda1

    4- Copy the EFI firmware (not the installer) into the FAT32 ESP partition
    I did this one in the graphical XFCE file manager

    5- Copy the Debian testing netinst ISO into a USB drive (sdb)
    sudo dd if=Downloads/debian-bullseye-DI-alpha2-arm64-netinst.iso status=progress of=/dev/sdb

    6- Put both the SD card and the USB drive in the RPi3b+

    7- Boot
    No problems here. The installer runs as on any PC :)

    8- Installation
    I didn't use WiFi; don't know if any problems would have appeared.
    Installed EXACTLY as on any PC. No tricks needed. I chose manual partitioning as I always do on my PC.
    The installer doesn't recognize the ESP partition, so I manually told the installer to use partition 1 as FAT32 and mount it in /boot/efi. I put all the options I use for my SSD (nodiratime, relatime, ...) and one very important: readonly. And don't format partition; keep contents.
    After installation is finished remove installation media and reboot.

    9- First attempt to boot
    Although installation doesn't report any errors, it won't boot. We need to put again the USB and reboot. But first run the EFI setup and configure the CPU speed to max :)

    10- Run recovery mode and run a shell in the / partition (for me it was the second one: mmcblk0p1 or something like that).
    Commands (the first one I don't know what it does or if it is necessary, but I used it (I read this: https://feeding.cloud.geek.nz/posts/reinstalling-grub-on-unbootable-debian/)):
    grub-mkdevicemap
    grub-install /dev/mmcblk0
    update-grub
    exit

    11- Reboot (and remove USB)

    12- First succesful boot!
    Not it should run :)

    ReplyDelete
    Replies
    1. 12: edit: Now it should run :)

      Delete
    2. 10: Edit: The first step in the recovery mode is to remount the EFI partition as rw:
      mount -o remount,rw /dev/mmcblk0p1 /boot/efi/

      Delete
    3. 4: Alternatively, from the terminal:

      $ sudo mount /dev/sdb1 /mnt/hd
      $ cd /mnt/hd/
      $ sudo unzip ~/Downloads/RPi4_UEFI_Firmware_v1.32.zip
      $ cd -
      $ sudo umount /mnt/hd

      Delete
  10. Prerequisites: missing display and keyboard or can it be done using ssh?

    ReplyDelete
    Replies
    1. You can configure Debian with preseed for automated installation. If you do that properly, there's no need for display or keyboard. See https://www.debian.org/releases/stable/example-preseed.txt

      Delete
  11. ok thx

    I'll try and then report.

    appreciate your help

    ReplyDelete
    Replies
    1. You may want to try this then:

      1. Edit /boot/grub/grub.cfg on the bootable media and add a set timeout=5 line under:
      set menu_color_highlight=white/blue
      (This is needed because the default of GRUB is to wait for keypress forever)

      2. Still in /boot/grub/grub.cfg locate the menuentry 'Install' { section (first one) and change:
      linux /install.a64/vmlinuz --- quiet
      to
      linux /install.a64/vmlinuz auto=true file=/cdrom/preseed.txt --- quiet
      This way, automated install is triggered and that preseed.txt from the media will be used.

      3. At the top level of the media, create a preseed.txt with the following content:

      d-i debian-installer/locale string en_IE.UTF-8
      d-i keyboard-configuration/xkb-keymap select irish
      d-i netcfg/get_hostname string unassigned-hostname
      d-i netcfg/get_domain string unassigned-domain
      d-i netcfg/hostname string pi3
      d-i anna/choose_modules string network-console
      d-i network-console/password password r00tme
      d-i network-console/password-again password r00tme

      Note that I'm using an Irish locale, so you'll have to change for yours. But if you do that
      you should be able to connect to the installer as 'pi3' through ssh and proceed from there.

      Delete
  12. Thought I was on my way to success, but then reality struck again! After I type device file the way it is posted above the install fails. Seems can't read or access for some reason. From a shell access I see mmcblk0p1 listed as a device. I partitioned in fat on a windows 10 machine using a partition app- EaseUS Partition Master.

    ReplyDelete
    Replies
    1. You may have a defective SD card (SD cards are a lot less reliable than people think!). Or you may want to ditch using EaseUS Partition Manager and follow Appendix A.

      Delete
  13. Thanks for getting back to me, I wasn't sure if anybody would... the instructions in appendix A is that for Windows?

    ReplyDelete
    Replies
    1. Yes. The first part is titled "Windows", so it is clearly for Windows.

      Delete
  14. Hey I want to thank you for your patience, I've not been bringing my A-game. I'm currently installing Cinnamon. I like the look and feel, but it's taking a long time to install. I'm mostly interested in web server option is there a smaller install that you could recommend? Lastly I have two odroid boards XU4 and C4 and a rock 64 will these instructions work for any of those??? Thanks again fot your tutorial.

    ReplyDelete
    Replies
    1. Sorry but you'll have to do your own research. And, no, these instructions are designed SPECIFICALLY for the Raspberry Pi 3. If you use something else, you are 100% on your own.

      Delete
  15. Hello. I followed the steps and it seems to be working fine up to the point of grub installation. I get an error that goes like "grub-install dummy failed". my disk layout is the same/similar as this one

    MMC/SD card #1 (mmcblk0) - 16.0 GB SD 2WCGO
    #1 primary 314.6 MB B K ESP
    #2 primary 1.0 GB f swap swap
    #3 primary 14.7 GB f ext4 /

    the only difference is that I'm using a much larger microSD. any ideas?

    ReplyDelete
    Replies
    1. Having the exact GRUB error might help. The first thing I would do, when that error happens, is that the boot/efi partition is properly mounted read/write. The size of the microSD shouldn't matter.

      Delete
  16. Any success with getting onboard sound to work or maybe an add-on HAT? Also I found that it will hand on a reboot, but a shut down and start up work just fine. Thanks for your development effort and making this available.

    ReplyDelete
  17. Hi, Pete. playing with my boards, all having uboot with ... [sigh] incomplete UEFI support, I realized, that I need yet one ARM SBC, but this time, with decent UEFI onboard. And I found out, that it's RPis with your work put there. Could you please tell me what is better RPi to pick, if I want the most complete, stable, compliant UEFI? In other words RPi 3 B/B+ or RPi 4 UEFI is already mature enough to use it without frustration? the use case is writing a UEFI OS Loader (not linux related).

    and yet, a couple of notes. you wrote: "it MUST be FAT16 and not FAT32, as the Debian partition manager will not detect it as ESP otherwise". I understand, this is to overcome quirks of a particular program (debian installer), but just out of curiosity - why they did so? UEFI spec. clearly says, that ESP is FAT32. always. and also, you mentioned several times the CPU ROM code inability to understand GPT/ESP. I am wondering, why not using an ordinary, non-ESP FAT volume then? UEFI happily loads OSLs from such volumes. And even Load Options, pointing to such volumes, could be created and will be recognized and managed by the UEFI Boot Manager. is this something again, related to the quirky "debian installer"? or it's just purism ("there should be ESP!")? :)

    ReplyDelete
  18. The best Pi to pick for UEFI would be the Pi 4, not because it's most complete but because, it's the most up to date platform, and therefore the one UEFI developers are focusing on. As to whether it will feat your need, it's up to you to read the firmware notes and see if that is likely to be the case, as noone can answer that question but you.

    "Why they did so?" I'm not the right person to ask. I'm just reporting what I found empirically here. I tried various things, and found that FAT16 led to better detection as ESP by the Debian Installer than FAT32. So the whole point is not "You can't use FAT32" but "If you deviate from what I am advising above, then you are likely to have to perform extra steps to make it work, so, if you don't want to have to waste your time going through extra actions, just follow the guide".

    ReplyDelete
    Replies
    1. thanks. and yet one question, maybe you can hint. what is the most guaranteed way of finding the Simple Text Output Protocol (STOP) on the UART/serial? When searching the ConOut variable, is searching for the node Type 3 Subtype 14 (Messaging, UART) a guaranteed way of finding a STOP instance on the serial port? Because, there is yet Type 2 (ACPI Device Path) SubType: 1 (ACPI Device Path) HID=PNP0501, "Serial" node. Or maybe searching for Type 3, Subtype 10, (Vendor) and checking for any of the defined GUIDs for all kinds of terminals is the best? I mean, if it's a serial port, then Type 3, Subtype 14 (UART) always should be there and is thus the most guaranteed sign of the serial console DP?

      Delete
    2. https://uefi.org/sites/default/files/resources/UEFI%20Spec%202.8B%20May%202020.pdf

      That's way too specific a question for me to invest the time trying to provide an answer, without being contracted for it. Sorry.

      Delete
  19. I liked the way this article is composed. I have been using various linux distros on PC (x86)... and this article contains all the details that are needed for reader with that background. All the hardwork that goes in it, and level of details, is highly appreciated.

    I have stumbled across a very peculiar problem. is this article also applicable to raspberry pi 400 ? As I read, https://github.com/pftf/RPi4 is still in experimental. Also pi 4B vs pi 400 have different processors so wonder if firmware would be different or not. Are they both referred as aarch64 ?

    ReplyDelete
    Replies
    1. "is this article also applicable to raspberry pi 400?" No it isn't. This article is ONLY for Pi 3. For Pi 4 and derivatives, you may find something like https://www.raspberrypi.org/forums/viewtopic.php?f=50&t=282839 useful, but that was written for Pi 4B, so your mileage with Pi 400 may vary.
      Also, the firmwares are different. You can't use the Pi 3 UEFI firmware and expect it to work on the Pi 4.

      Delete
  20. Hi Pete,

    I've used your tutorial many times now, and it works perfectly! It's nice to have a plain debian x64 variant for management purposes, and in my use case, the 64 bit variant seems to be also slightly faster, maybe because I'm using Python a fair amount.
    My diskpart command is 'create partition primary align=1024 size=511', since the newer isos are slightly over 300MB, and it aligns the sectors nicely at powers of 2. No idea if the latter helps, I didn't notice any performance influence but well, with SD-card sizes nowadays..
    I wanted to thank you for this work. I don't have any RPI4s, but I just saw the Github also has RPI4 firmware, so that's interesting for when I do.
    Oh, when trying this method using the iso for Debian 11 bullseye, it keeps asking for the same firmware during the 'Load missing firmware from removable media' step, and it'll never be able to get any (both wired and wifi) network interface to work on both 3B and 3B+. I just install 10, and then immediately update through changing '/etc/apt/sources.list'. It's not a major issue, and seeing as there's no difference in the firmware used, it's probably due to a difference in included drivers on the iso.
    Anyway, thanks for the tutorial!

    ReplyDelete
    Replies
    1. The guide for RPi4 bullseye installation can be found at https://www.raspberrypi.org/forums/viewtopic.php?f=50&t=282839. Also, the reason it's asking for the same firmware is because, when using USB media, it appears that the Debian installer assumes the firmware not to already be present when it asks for it, and is looking for a USB plug event to detect content changes. So you need to unplug-replug the USB for each firmware blob you are asked if you find it appears to be stuck in a loop.

      Delete
  21. i'm frustrating to get this to work... I choose 1st partition to be 500 cause newer images are bigger. At first dhcp or manual setup didn't work from eth interface. Then I changed to wireless and setup was completed. Then at boot after firmware logo i get a black screen with message Synchronous exception at 0x17FFFF....

    ReplyDelete
  22. Hey ! you have a problem with the bootloader, above they wrote maybe the boot files are missing on your sd. But you can easily boot using any usb flash card. insert it into a free port and upload the raspberry

    ReplyDelete
  23. Hi! After successful installation I get an error "Synchronous Exception at 0x17FFFFFA9402E831" on black screen. Looks like UEFI produce the error. Any ideas?

    ReplyDelete