Reverse Engineering PCI Drivers

by Vikram Ambrose -- Vambrose 01:44, 22 February 2010 (UTC)

This is a short guide demonstrating how I used the PCI Passthrough framework in KVM to reverse engineer atiflash.exe – a video BIOS flashing tool for DOS.

atiflash.exe is a real dos application that flashes the video BIOS on ATi graphics cards. There are numerous ATi video BIOS flashing tools for Windows, but none for Linux as the code for doing this has not been made publicly available by AMD.

This technique allowed me to trace all PCI configuration space and memory mapped i/o (mmio) accesses to the PCI-E device from the closed source flashing tool.

Tools needed:

• CPU with virtualisation support (eg, AMD-V, Intel VT-x, an IOMMU is not necessary)
• Newer-the-better Linux kernel with KVM support
• Special KVM branch of QEMU (git://git.kernel.org/pub/scm/virt/kvm/qemu-kvm.git)
• Guest operating system (I used FreeDOS from UBCD - http://www.ultimatebootcd.com/)
• Secondary video card to boot Linux with while the Radeon is used in the virtual machine

In order to give the virtual machine direct pci access, the host operating system cannot register a driver for the device. In my case, this simply meant blacklisting the radeon and drm kernel modules.

All the tracing is done from QEMU. Using the special qemu-kvm branch mentioned above, go into qemu-kvm/hw/device-assignments.c and ensure:

 #define DEVICE_ASSIGNMENT_DEBUG 1 

This will turn on logging for assigned_dev_pci_{read,write}_config() functions (Config space) and slow_bar_read{b,w,l} and slow_bar_write{b,w,l} functions (MMIO). You may also want to add d->e_physbase to the prints to get the physical assigned address.

In order to trace the write commands to the ROM space, you will need to write your own _write{b,w,l} functions. Eg:

 static void slow_rom_writeb(void *opaque, target_phys_addr_t addr, uint32_t val)
 {
     DEBUG(" addr=0x" TARGET_FMT_plx " val=0x%02x\n", addr, val);
 }
 ...
 static CPUWriteMemoryFunc * const slow_rom_write[] = {
    &slow_rom_writeb,
    &slow_rom_writew,
    &slow_rom_writel
 };

Then register your function array in assigned_dev_iomem_map_slow(), Eg:

 ...
    if (region_num == PCI_ROM_SLOT)
 -        m = cpu_register_io_memory(slow_bar_read, NULL, region);
 +        m = cpu_register_io_memory(slow_bar_read, slow_rom_write, region);
 ...

Now modify assigned_dev_register_regions() to get QEMU to use the slow path otherwise KVM will let the guest write directly to the device.

 ...
       if (cur_region->type & IORESOURCE_MEM) {
 -            int slow_map = 0;
 +            int slow_map = 1;
 ...

You can also get the Programmed I/O (PIO) by logging assigned_dev_ioport_{read,write)*(), but otherwise this should give you a good idea of what the flashing tool is doing.

Now use lspci to get the location of the PCI device:

 02:00.0 VGA compatible controller: ATI Technologies Inc RV770 [Radeon HD 4870]
 02:00.1 Audio device: ATI Technologies Inc HD48x0 audio

Pass the device to qemu, along with your virtual machine disk image. Eg;

 qemu-system-x86 -hda disk.img -cdrom ubcd411.iso -boot order=d -pcidevice host=02:00:0,dma=none

Run the flashing tool from the VM and watch the traces appear on the console.

Here are some traces from my trials (graciously decoded by Carl-D):

• http://coreboot.pastebin.com/f73df946c
• http://coreboot.pastebin.com/f6c21856e

Useful links:

• VGA Bios database - http://www.techpowerup.com/vgabios/
• Chapter 12, PCI device drivers of "Linux Device Drivers (3rd Edition)" by J.Corbet et al, downloadable for free from - http://lwn.net/Kernel/LDD3/
• If you would like to use the same technique on a native Linux application/driver, see the mmiotrace project - http://nouveau.freedesktop.org/wiki/MmioTrace

Special thanks go out to Carl-Daniel Hailfinger, Twice#11 and the entire KVM and QEMU team for making this possible.