Flashmap: Difference between revisions

From coreboot
Jump to navigation Jump to search
m (Add some missing cros_bundle_firmware command-line switch--cross references)
m (Update nontrivial CL list)
 
(18 intermediate revisions by the same user not shown)
Line 4: Line 4:


==How it's currently done (how the Chromium OS project presently constructs firmware images)==
==How it's currently done (how the Chromium OS project presently constructs firmware images)==
Most Intel-based Chromium OS devices currently use an 8 MB firmware image that includes---among other things---the Intel ME firmware, a copy of coreboot including the ramstage and depthcharge (bootloader) payload, two additional copies of the ramstage and bootloader payload, and a separate SeaBIOS payload. The primary description of this format exists in board-specific flattened device tree files, which are used by a script called [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/cros_bundle_firmware cros_bundle_firmware] to modify the image produced by the coreboot build system. For instance, the layout of the Panther board's firmware exists at [https://chromium.googlesource.com/chromiumos/platform/depthcharge/+/master/board/panther/fmap.dts https://chromium.googlesource.com/chromiumos/platform/depthcharge/+/master/board/panther/fmap.dts], and results in a final image that looks like this:
Most Intel-based Chromium OS devices currently use an 8 MB firmware image that includes—among other things—the Intel ME firmware, a copy of coreboot including the ramstage and depthcharge (bootloader) payload, two additional copies of the ramstage and bootloader payload, and a separate SeaBIOS payload. The primary description of this format exists in board-specific flattened device tree files, which are used by a script called [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/cros_bundle_firmware cros_bundle_firmware] to modify the image produced by the coreboot build system. For instance, the layout of the Panther board's firmware exists at [https://chromium.googlesource.com/chromiumos/platform/depthcharge/+/master/board/panther/fmap.dts https://chromium.googlesource.com/chromiumos/platform/depthcharge/+/master/board/panther/fmap.dts], and results in a final image that looks like this:
 
===Intel Chromium OS firmware flash layout with component packaging notes===
{| class="wikitable"
{| class="wikitable"
! Section
! Section
! Offset
! Offset
! FMAP name
! Contents
! Contents
! Original source
! Original source
Line 17: Line 20:
|-
|-
| 0x700000
| 0x700000
| Bootstub (coreboot image)
| BOOT_STUB
| Coreboot image
| coreboot.rom (coreboot build system)
| coreboot.rom (coreboot build system)
| --coreboot coreboot.rom
| --coreboot coreboot.rom
--coreboot-elf depthcharge.elf
--coreboot-elf depthcharge.elf
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/bundle_firmware.py#967 adds] depthcharge.elf as a payload (which---md5sums confirm---is exactly equivalent to just adding depthcharge.payload) and [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/fdt.py#693 compiled] (then [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/bundle_firmware.py#1072 mod] [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/cros_bundle_firmware#96 if] [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#780 ied]) version of fmap.dts to the existing CBFS
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/bundle_firmware.py#967 adds] both depthcharge.elf as a payload (which—md5sums confirm—is exactly equivalent to just adding depthcharge.payload) and a [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/fdt.py#693 compiled] (then [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/bundle_firmware.py#1072 mod] [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/cros_bundle_firmware#96 if] [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#780 ied]) version of fmap.dts to the existing CBFS
| <code>CONFIG_CMOS_POST_OFFSET</code>, <code>CONFIG_CBFS_SIZE</code>
| <code>CONFIG_CBFS_SIZE</code>, <code>CONFIG_HAVE_MRC</code>, <code>CONFIG_MRC_FILE</code>, <code>CONFIG_VGA_BIOS</code>, <code>CONFIG_VGA_BIOS_FILE</code>, <code>CONFIG_VGA_BIOS_ID</code>, <code>CONFIG_PAYLOAD_NONE</code>
|-
|-
| 0x611000
| 0x611000
| GBB (Google Binary Block)
| GBB
| Google Binary Block
| /chromeos-config/ entries in fmap.dts
| /chromeos-config/ entries in fmap.dts
| --dt fmap.dts
| --dt fmap.dts
--bmpblk bmpblk.bin
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/bundle_firmware.py#389 generates] and inserts it
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/bundle_firmware.py#389 generates] and inserts it
|-
|-
| 0x610840
| 0x610840
| RO_FRID_PAD
| (Reserved)
| (Reserved)
|-
|-
| 0x610800
| 0x610800
| RO-FWID (Firmware ID)
| RO_FRID
| Firmware ID
| /model entry in fmap.dts concatenated with build number
| /model entry in fmap.dts concatenated with build number
| --dt fmap.dts
| --dt fmap.dts
Line 40: Line 48:
|-
|-
| 0x610000
| 0x610000
| FMAP (Flash MAP)
| FMAP
| Flashmap
| fmap.dts itself
| fmap.dts itself
| --dt fmap.dts
| --dt fmap.dts
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#909 generates] it while assembling the ultimate image file
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#644 reformats] the section names then [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#909 generates] the table while assembling the ultimate image file
| <code>CONFIG_CHROMEOS</code>, <code>CONFIG_FLASHMAP_OFFSET</code>
|-
|-
| 0x604000
| 0x604000
| RO_UNUSED
| (Reserved)
| (Reserved)
|-
|-
| 0x600000
| 0x600000
| RO-VPD (Vital Product Data)
| RO_VPD
| Vital Product Data
|
|
|
|
Line 57: Line 69:
|-
|-
| 0x400000
| 0x400000
| Legacy (SeaBIOS image)
| RW_LEGACY
| SeaBIOS image
| seabios.cbfs
| --seabios seabios.cbfs
| cros_bundle_firmware helper inserts the file verbatim, as confirmed by an md5sum on the final image
|-
|-
| 0x3fa000
| 0x3fa000
| RW_UNUSED
| (Reserved)
| (Reserved)
|-
|-
| 0x3f8000
| 0x3f8000
| RW-VPD (Vital Product Data)
| RW_VPD
| Vital Product Data
|
|
|
|
Line 71: Line 89:
|-
|-
| 0x3f6000
| 0x3f6000
| Vblock-dev (third-party kernel signing keys)
| VBLOCK_DEV
| Third-party kernel signing keys
|
|
|
|
Line 77: Line 96:
|-
|-
| 0x3f4000
| 0x3f4000
| Shared-data (RW firmware calibration data)
| SHARED_DATA
| RW firmware calibration data
|
|
|
|
Line 85: Line 105:
|-
|-
| 0x3f0000
| 0x3f0000
| ELOG (Event LOG)
| RW_ELOG
| Event log
|
|
|
|
| cros_bundle_firmware fills this section with 0xff s
| cros_bundle_firmware fills this section with 0xff s
| <code>CONFIG_ELOG</code>, <code>CONFIG_ELOG_AREA_SIZE</code>
|-
|-
| 0x3e0000
| 0x3e0000
| MRC-cache (Memory Reference Code training data)
| RW_MRC_CACHE
| Memory Reference Code training data
|
|
|
|
Line 99: Line 122:
|-
|-
| 0x3dffc0
| 0x3dffc0
| FWID-B
| RW_FWID_B
| Updateable firmware ID
| colspan="3" style="text-align: center" | (Copy of RO-FWID above)
| colspan="3" style="text-align: center" | (Copy of RO-FWID above)
|-
|-
| 0x300000
| 0x300000
| Main-B (copy of coreboot ramstage and payload)
| FW_MAIN_B
| Copy of coreboot ramstage and payload
| colspan="3" style="text-align: center" | (Same as main-A below)
| colspan="3" style="text-align: center" | (Same as main-A below)
|-
|-
| 0x2f0000
| 0x2f0000
| Vblock-B (signing keys)
| VBLOCK_B
| Signing keys
| colspan="3" style="text-align: center" | (Same as vblock-A below)
| colspan="3" style="text-align: center" | (Same as vblock-A below)
|-
|-
Line 113: Line 139:
|-
|-
| 0x2effc0
| 0x2effc0
| FWID-A
| RW_FWID_A
| Updateable firmware ID
| colspan="3" style="text-align: center" | (Copy of RO-FWID above)
| colspan="3" style="text-align: center" | (Copy of RO-FWID above)
|-
|-
| 0x210000
| 0x210000
| Main-A (copy of coreboot ramstage and payload)
| FW_MAIN_A
| Copy of coreboot ramstage and payload
| depthcharge.payload and ramstage.stage
| depthcharge.payload and ramstage.stage
| --uboot depthcharge.payload
| --uboot depthcharge.payload
--add-blob ramstage ramstage.stage
--add-blob ramstage ramstage.stage
| Because fmap.dts lists its type as "blob boot,ramstage" , cros_bundle_firmware concatenates the blob called ramstage onto the end of coreboot
| Because fmap.dts lists its type as "blob boot,ramstage" , cros_bundle_firmware concatenates the blob called ramstage onto the end of coreboot
| <code>CONFIG_VBOOT_BOOT_LOADER_INDEX</code>, <code>CONFIG_VBOOT_RAMSTAGE_INDEX</code>, <code>CONFIG_RELOCATABLE_RAMSTAGE</code>, <code>CONFIG_CACHE_RELOCATED_RAMSTAGE_OUTSIDE_CBMEM</code>
|-
|-
| 0x200000
| 0x200000
| Vblock-A (signing keys)
| VBLOCK_A
| Signing keys
|
|
|
|
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#377 signs] the main-A section and puts the signature here
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#377 signs] the main-A section and puts the signature here
| <code>CONFIG_VBOOT_VERIFY_FIRMWARE</code>
|-
|-
| rowspan="3" | FW-descriptor
| rowspan="3" | FW-descriptor
|-
|-
| 0x001000
| 0x001000
| ME (Intel Management Engine firmware blob)
| SI_ME
| Intel Management Engine firmware blob
| coreboot.rom
| coreboot.rom
| --coreboot coreboot.rom
| --coreboot coreboot.rom
| Present in the original image generated by the coreboot build system (which simply inserted some Intel blob verbatim)
| Present in the original image generated by the coreboot build system (which simply inserted some Intel blob verbatim)
| <code>CONFIG_USE_BLOBS</code>
|-
|-
| 0x000000
| 0x000000
| FD (Intel Firmware Descriptor header)
| SI_DESC
| IFD (Intel Firmware Descriptor) header
| coreboot.rom
| coreboot.rom
| --coreboot coreboot.rom
| --coreboot coreboot.rom
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#303 invokes] ifdtool to create the final image, using the original coreboot.rom file's ME and "skeleton" (template) IFD, and concatenating on verbatim the rest of the pieces it assembled to form the (Intel-dubbed) "BIOS" section
| cros_bundle_firmware helper [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/lib/pack_firmware.py#303 invokes] ifdtool to create the final image, using the original coreboot.rom file's ME and "skeleton" (template) IFD, and concatenating on verbatim the rest of the pieces it assembled to form the (Intel-dubbed) "BIOS" section
| <code>CONFIG_USE_BLOBS</code>
|}
|}


==What's so bad about that (the pitfalls of this build model that we hope to solve)==
==What's so bad about that (the pitfalls of this build model that we hope to solve)==
While the layout of the flash image is obvious from a board's fmap.dts file, the actual process by which the image is constructed&mdash;and most importantly, the transformation and other processing applied to each input file&mdash;is poorly documented and not widely understood. After examining the overall assembly procedure and distilling it into the above tables, the following shortcomings are apparent:
===The build system organization favors divergence from upstream coreboot===
This implies two subproblems: upstreaming code from Chromium OS to the main coreboot source tree is difficult, and it isn't easy to build upstream coreboot for a Chromium OS device. It's important to note that, while related, these are not identical shortcomings: while the upstreaming difficulty certainly prohibits building some devices, other devices have enough code already pushed that it would be possible to run them if the coreboot build system were able to produce an image that was ready to flash on the hardware.
As can be seen from the above table, there's no technical reason that sufficient functionality to build a complete image can't be upstreamed to coreboot itself: with the exception of ramstage.stage&mdash;which is actually just another copy of a binary generated by the coreboot build system&mdash;everything that needs to be added to the coreboot-generated image is either a binary blob or compiles without depending on any part of coreboot besides libpayload.
===There is error-prone duplication in the image layout configuration===
Because image assembly is currently a multistage process, several offsets and sizes need to be duplicated across multiple configuration files; if these files are not manually kept in step, the process can result in an obviously broken (e.g. part or all of coreboot or the CBFS is overwritten with another binary) or more subtly broken (e.g. coreboot assumes an FMAP exists at a particular flash location, but when it attempts to find it at runtime, it isn't there). To make matters worse, the redundant configuration parameters are stored in separate ''repositories'', so checking out a different version of one boot component can silently result in a mismatch. Adding insult to injury, one of said configurations is stored in ''binary form'' within the IFD, which forces us to check in an IFD blob for each boardmaking it even ''more'' difficult to rearrange the flash layout.


==Why you should care (how this pertains to all coreboot users)==
==Why you should care (how this pertains to all coreboot users)==
Short answer: For owners of certain Chromebooks/Chromeboxes: it would make it easier to build upstream coreboot for their devices, as well as easier to upstream more boards. For everyone else: the Chromium OS firmware includes several features that are generally useful, and landing tools for building Chromebook/Chromebox images would enable other users to easily add modular, precompiled components to their image builds. This would allow them to avoid maintaining separate, last-mile build systems that use separate configuration files, and hence make it easier to apply coreboot to advanced and unconventional setups.
The benefits reach beyond the mere construction of a firmware image, however: on many modern systems, integrating flashmap support will make it easier to flash images onto the board. For example, on recent x86 platforms, the Intel Management Engine reads its firmware from a section of the flash memory. Post-boot, access to this portion of the flash chip from the CPU is often completely disallowed; thus, it is impossible to perform actions such as reflashing that portion of the flash without an external debugger. Unless the coreboot build system is able to extract the flash layout from the existing image, crafting a replacement image that can be flashed from the host itself is very difficult. Including a flashmap in a standard format directly in the image would provide the foundations necessary for the build system to automatically construct images that could be flashed on such systems. However, in order to facilitate this future direction, it's important that whatever tools are added use the image's&mdash;rather than the source tree's&mdash;flashmap representation whenever possible, so that end users can easily construct their own images even if the latter cannot be redistributed to them.


==How do we fix it (the solution being pursued)==
==How do we fix it (the solution being pursued)==
===What new features is coreboot getting?===
We're developing a novel plaintext format for representing flashmap sections in the source tree. Take a look at [http://crbug.com/461875 Chromium bug 461875] for more discussion and details, but to sum it up, we're moving from representing layouts in dts files to using "fmd" files such as the following Panther example:
<nowiki>
HOST_FIRMWARE@0xff800000 8M {
        # Non-BIOS section of the Intel Firmware Descriptor image.
        # This section covers all the parts that are not shown to the CPU right
        # below 4G.
        SI_ALL 2M {
                # Firmware Descriptor section of the Intel Firmware Descriptor
                # image.
                SI_DESC 4K
                # Intel Management Engine section of the Intel Firmware
                # Descriptor image.
                SI_ME
        }
        # "BIOS" section of the Intel Firmware Descriptor image.
        # This section covers the complete image as shown to the CPU right below
        # 4G.
        SI_BIOS {
                RW_SECTION_A 0xf0000 {
                        # Alignment: 4K (for updating) and must be in start of
                        # each RW_SECTION.
                        VBLOCK_A 64K
                        FW_MAIN_A
                        RW_FWID_A 64
                }
                RW_SECTION_B 0xf0000 {
                        # Alignment: 4K (for updating) and must be in start of
                        # each RW_SECTION.
                        VBLOCK_B 64K
                        FW_MAIN_B
                        RW_FWID_B 64
                }
                # Alignment: 4K (for updating).
                RW_MRC_CACHE 64K
                # Alignment: 4K (for updating).
                RW_ELOG 16K
                # Alignment 4K (for updating).
                # Anything in this range may be updated in recovery.
                RW_SHARED 16K {
                        # Alignment: 4K (for random read/write).
                        # RW firmware can put calibration data here.
                        SHARED_DATA 8K
                        # Alignment: 4K (for random read/write).
                        VBLOCK_DEV 8K
                }
                # Alignment: 4K (for updating).
                RW_VPD 8K
                RW_LEGACY@2M 2M
                # This describes the portion of the image that will be
                # write-protected in the factory.
                WP_RO {
                        # Alignment: 4K.
                        # Vital Product Data.
                        RO_VPD 16K
                        RO_SECTION@64K {
                                # We encourage to align FMAP partition in as
                                # large a block as possible so that flashrom can
                                # find it sooner. For example, aligning to 512K
                                # is better than 256K.
                                FMAP 2K
                                RO_FRID 0x40
                                # Alignment: 4K.
                                GBB@4K
                                BOOT_STUB 1M
                        }
                }
        }
}</nowiki>
===What needs to be done?===
# On Chrome OS devices, some offsets and sizes are currently read from an FMAP section in Google's [https://code.google.com/p/flashmap flashmap] format. This is currently controlled by <code>CONFIG_CHROMEOS</code>, but should be migrated to a separate Kconfig key so that it can be used for other purposes as well. To begin with, the build system can just assume that an FMAP region will be manually inserted after the initial build process.
## (landed) https://chromium-review.googlesource.com/#/c/260501: Eliminate <code>CONFIG_ELOG_FULL_THRESHOLD</code> and <code>CONFIG_ELOG_SHRINK_SIZE</code>
## https://chromium-review.googlesource.com/#/c/250010: Create <code>CONFIG_USE_FLASHMAP</code> to control whether flashmap offsets and sizes should be used
# The source tree's representation of the flashmap is currently stored in the [https://chromium.googlesource.com/chromiumos/platform/depthcharge/+/master/board/ board directory of the depthcharge codebase] and parsed by the [https://chromium.googlesource.com/chromiumos/platform/dev-util/+/master/host/cros_bundle_firmware separate cros_bundle_firmware script]. The ability to generate an FMAP region&mdash;and the per-board dts files&mdash;should be moved into the coreboot repository, where it can be used to construct the firmware image. For now, we can use a Kconfig option to explicitly specify the path to the desired "Chrome OS 8MB" dts file. There already exists a [https://chromium-review.googlesource.com/#/c/196513 prototype reimplementation of much of cros_bundle_firmware] that might be a reasonable reference for this work, but unfortunately, much of its code is spent dealing with the Python/C interface.
## (defunct) https://chromium-review.googlesource.com/#/c/251281: Begin <tt>fmaptool</tt>, a C utility for generating FMAP sections given a dts file
## https://chromium-review.googlesource.com/#/c/255031: Introduce the fmd ("flashmap descriptor") language and compiler
# Migrate flash offsets and sizes out of Kconfig in favor of centralizing them in the flashmap description. This will significantly reduce the amount of configuration duplication within each board, as well as preventing many mismatch problems. Some of these numbers (e.g. the CBFS offset&mdash;or, preferably, the FMAP one instead&mdash;because the romstage isn't relocatable on x86) are needed at compile time, so we'll need to generate a header that allows them to be used programmatically.
# Make the coreboot build system capable of inserting blobs into the firmware image, external to the CBFS. One important question here is whether the FMAP deserves to go in a section of its own, or be placed in the CBFS: it's a bit of a chicken-and-egg problem, because the location of the CBFS could be stored in the FMAP so it didn't need to be baked into the coreboot build itself. This may require changes to cbfstool, but will allow the replacement of cros_bundle_firmware.
# Generate per-board IFD headers based on the per-board plaintext dts files and (possibly) some binary template. This should allow us to eliminate this class of blobs from the repositories. This may require some changes to ifdtool.
# Add a default set of dts files for e.g. Intel, AMD, and ARM CPUs with various flash chip sizes. Remove the remnants of the flash offset/size Kconfig keys, and turn <code>CONFIG_USE_FLASHMAP</code> Kconfig key into <code>CONFIG_CUSTOM_FLASHMAP</code>, which then allows you to choose a non-default (e.g. Chrome OS) layout for your particular build.
# Implement the ability to build images with the same layout as an existing image. This will help users to easily build modified images for devices that already run coreboot. Once again, the issue of where the FMAP should be placed is important, because we need to be able to find it in the existing image.
# Add functionality to copy nonredistributable portions of an existing image into a new one. This way, users who own a hardware device that ships with vender blobs will be able to build firmware images for it without violating vendor agreements.

Latest revision as of 17:53, 19 March 2015

or...

Toward a unified representation for the layout of coreboot flash images

N.B. The changes described herein are being made as part of the Chromium OS project; as such, they will initially be committed to the project's own fork of the main coreboot repository, which is available at https://chromium.googlesource.com/chromiumos/third_party/coreboot. Unless otherwise noted, the paths and processes described throughout this page are as they exist(ed) in a checkout of the master branch of the Chromium OS sources as they appeared at the beginning of 2015. One of the guiding design principles is to keep the tools general enough that they will be helpful to others, and the resulting work will be upstreamed to the main repository once it has been regression-tested in the context of Chromium OS hardware.

How it's currently done (how the Chromium OS project presently constructs firmware images)

Most Intel-based Chromium OS devices currently use an 8 MB firmware image that includes—among other things—the Intel ME firmware, a copy of coreboot including the ramstage and depthcharge (bootloader) payload, two additional copies of the ramstage and bootloader payload, and a separate SeaBIOS payload. The primary description of this format exists in board-specific flattened device tree files, which are used by a script called cros_bundle_firmware to modify the image produced by the coreboot build system. For instance, the layout of the Panther board's firmware exists at https://chromium.googlesource.com/chromiumos/platform/depthcharge/+/master/board/panther/fmap.dts, and results in a final image that looks like this:

Intel Chromium OS firmware flash layout with component packaging notes

Section Offset FMAP name Contents Original source cros_bundle_firmware flag(s) Packaging procedure Coreboot Kconfig entr[yi](es)
RO
0x700000 BOOT_STUB Coreboot image coreboot.rom (coreboot build system) --coreboot coreboot.rom

--coreboot-elf depthcharge.elf

cros_bundle_firmware helper adds both depthcharge.elf as a payload (which—md5sums confirm—is exactly equivalent to just adding depthcharge.payload) and a compiled (then mod if ied) version of fmap.dts to the existing CBFS CONFIG_CBFS_SIZE, CONFIG_HAVE_MRC, CONFIG_MRC_FILE, CONFIG_VGA_BIOS, CONFIG_VGA_BIOS_FILE, CONFIG_VGA_BIOS_ID, CONFIG_PAYLOAD_NONE
0x611000 GBB Google Binary Block /chromeos-config/ entries in fmap.dts --dt fmap.dts

--bmpblk bmpblk.bin

cros_bundle_firmware helper generates and inserts it
0x610840 RO_FRID_PAD (Reserved)
0x610800 RO_FRID Firmware ID /model entry in fmap.dts concatenated with build number --dt fmap.dts cros_bundle_firmware helper performs the concatenation
0x610000 FMAP Flashmap fmap.dts itself --dt fmap.dts cros_bundle_firmware helper reformats the section names then generates the table while assembling the ultimate image file CONFIG_CHROMEOS, CONFIG_FLASHMAP_OFFSET
0x604000 RO_UNUSED (Reserved)
0x600000 RO_VPD Vital Product Data cros_bundle_firmware fills this section with 0xff s
0x400000 RW_LEGACY SeaBIOS image seabios.cbfs --seabios seabios.cbfs cros_bundle_firmware helper inserts the file verbatim, as confirmed by an md5sum on the final image
0x3fa000 RW_UNUSED (Reserved)
0x3f8000 RW_VPD Vital Product Data cros_bundle_firmware fills this section with 0xff s
RW-shared
0x3f6000 VBLOCK_DEV Third-party kernel signing keys cros_bundle_firmware fills this section with 0xff s
0x3f4000 SHARED_DATA RW firmware calibration data cros_bundle_firmware fills this section with 0x00 s
0x3f0000 RW_ELOG Event log cros_bundle_firmware fills this section with 0xff s CONFIG_ELOG, CONFIG_ELOG_AREA_SIZE
0x3e0000 RW_MRC_CACHE Memory Reference Code training data cros_bundle_firmware fills this section with 0xff s
RW-B
0x3dffc0 RW_FWID_B Updateable firmware ID (Copy of RO-FWID above)
0x300000 FW_MAIN_B Copy of coreboot ramstage and payload (Same as main-A below)
0x2f0000 VBLOCK_B Signing keys (Same as vblock-A below)
RW-A
0x2effc0 RW_FWID_A Updateable firmware ID (Copy of RO-FWID above)
0x210000 FW_MAIN_A Copy of coreboot ramstage and payload depthcharge.payload and ramstage.stage --uboot depthcharge.payload

--add-blob ramstage ramstage.stage

Because fmap.dts lists its type as "blob boot,ramstage" , cros_bundle_firmware concatenates the blob called ramstage onto the end of coreboot CONFIG_VBOOT_BOOT_LOADER_INDEX, CONFIG_VBOOT_RAMSTAGE_INDEX, CONFIG_RELOCATABLE_RAMSTAGE, CONFIG_CACHE_RELOCATED_RAMSTAGE_OUTSIDE_CBMEM
0x200000 VBLOCK_A Signing keys cros_bundle_firmware helper signs the main-A section and puts the signature here CONFIG_VBOOT_VERIFY_FIRMWARE
FW-descriptor
0x001000 SI_ME Intel Management Engine firmware blob coreboot.rom --coreboot coreboot.rom Present in the original image generated by the coreboot build system (which simply inserted some Intel blob verbatim) CONFIG_USE_BLOBS
0x000000 SI_DESC IFD (Intel Firmware Descriptor) header coreboot.rom --coreboot coreboot.rom cros_bundle_firmware helper invokes ifdtool to create the final image, using the original coreboot.rom file's ME and "skeleton" (template) IFD, and concatenating on verbatim the rest of the pieces it assembled to form the (Intel-dubbed) "BIOS" section CONFIG_USE_BLOBS

What's so bad about that (the pitfalls of this build model that we hope to solve)

While the layout of the flash image is obvious from a board's fmap.dts file, the actual process by which the image is constructed—and most importantly, the transformation and other processing applied to each input file—is poorly documented and not widely understood. After examining the overall assembly procedure and distilling it into the above tables, the following shortcomings are apparent:

The build system organization favors divergence from upstream coreboot

This implies two subproblems: upstreaming code from Chromium OS to the main coreboot source tree is difficult, and it isn't easy to build upstream coreboot for a Chromium OS device. It's important to note that, while related, these are not identical shortcomings: while the upstreaming difficulty certainly prohibits building some devices, other devices have enough code already pushed that it would be possible to run them if the coreboot build system were able to produce an image that was ready to flash on the hardware.

As can be seen from the above table, there's no technical reason that sufficient functionality to build a complete image can't be upstreamed to coreboot itself: with the exception of ramstage.stage—which is actually just another copy of a binary generated by the coreboot build system—everything that needs to be added to the coreboot-generated image is either a binary blob or compiles without depending on any part of coreboot besides libpayload.

There is error-prone duplication in the image layout configuration

Because image assembly is currently a multistage process, several offsets and sizes need to be duplicated across multiple configuration files; if these files are not manually kept in step, the process can result in an obviously broken (e.g. part or all of coreboot or the CBFS is overwritten with another binary) or more subtly broken (e.g. coreboot assumes an FMAP exists at a particular flash location, but when it attempts to find it at runtime, it isn't there). To make matters worse, the redundant configuration parameters are stored in separate repositories, so checking out a different version of one boot component can silently result in a mismatch. Adding insult to injury, one of said configurations is stored in binary form within the IFD, which forces us to check in an IFD blob for each boardmaking it even more difficult to rearrange the flash layout.

Why you should care (how this pertains to all coreboot users)

Short answer: For owners of certain Chromebooks/Chromeboxes: it would make it easier to build upstream coreboot for their devices, as well as easier to upstream more boards. For everyone else: the Chromium OS firmware includes several features that are generally useful, and landing tools for building Chromebook/Chromebox images would enable other users to easily add modular, precompiled components to their image builds. This would allow them to avoid maintaining separate, last-mile build systems that use separate configuration files, and hence make it easier to apply coreboot to advanced and unconventional setups.

The benefits reach beyond the mere construction of a firmware image, however: on many modern systems, integrating flashmap support will make it easier to flash images onto the board. For example, on recent x86 platforms, the Intel Management Engine reads its firmware from a section of the flash memory. Post-boot, access to this portion of the flash chip from the CPU is often completely disallowed; thus, it is impossible to perform actions such as reflashing that portion of the flash without an external debugger. Unless the coreboot build system is able to extract the flash layout from the existing image, crafting a replacement image that can be flashed from the host itself is very difficult. Including a flashmap in a standard format directly in the image would provide the foundations necessary for the build system to automatically construct images that could be flashed on such systems. However, in order to facilitate this future direction, it's important that whatever tools are added use the image's—rather than the source tree's—flashmap representation whenever possible, so that end users can easily construct their own images even if the latter cannot be redistributed to them.

How do we fix it (the solution being pursued)

What new features is coreboot getting?

We're developing a novel plaintext format for representing flashmap sections in the source tree. Take a look at Chromium bug 461875 for more discussion and details, but to sum it up, we're moving from representing layouts in dts files to using "fmd" files such as the following Panther example:

 HOST_FIRMWARE@0xff800000 8M {
        # Non-BIOS section of the Intel Firmware Descriptor image.
        # This section covers all the parts that are not shown to the CPU right
        # below 4G.
        SI_ALL 2M {
                # Firmware Descriptor section of the Intel Firmware Descriptor
                # image.
                SI_DESC 4K
                # Intel Management Engine section of the Intel Firmware
                # Descriptor image.
                SI_ME
        }
        # "BIOS" section of the Intel Firmware Descriptor image.
        # This section covers the complete image as shown to the CPU right below
        # 4G.
        SI_BIOS {
                RW_SECTION_A 0xf0000 {
                        # Alignment: 4K (for updating) and must be in start of
                        # each RW_SECTION.
                        VBLOCK_A 64K
                        FW_MAIN_A
                        RW_FWID_A 64
                }
                RW_SECTION_B 0xf0000 {
                        # Alignment: 4K (for updating) and must be in start of
                        # each RW_SECTION.
                        VBLOCK_B 64K
                        FW_MAIN_B
                        RW_FWID_B 64
                }
                # Alignment: 4K (for updating).
                RW_MRC_CACHE 64K
                # Alignment: 4K (for updating).
                RW_ELOG 16K
                # Alignment 4K (for updating).
                # Anything in this range may be updated in recovery.
                RW_SHARED 16K {
                        # Alignment: 4K (for random read/write).
                        # RW firmware can put calibration data here.
                        SHARED_DATA 8K
                        # Alignment: 4K (for random read/write).
                        VBLOCK_DEV 8K
                }
                # Alignment: 4K (for updating).
                RW_VPD 8K
                RW_LEGACY@2M 2M
                # This describes the portion of the image that will be
                # write-protected in the factory.
                WP_RO {
                        # Alignment: 4K.
                        # Vital Product Data.
                        RO_VPD 16K
                        RO_SECTION@64K {
                                # We encourage to align FMAP partition in as
                                # large a block as possible so that flashrom can
                                # find it sooner. For example, aligning to 512K
                                # is better than 256K.
                                FMAP 2K
                                RO_FRID 0x40
                                # Alignment: 4K.
                                GBB@4K
                                BOOT_STUB 1M
                        }
                }
        }
 }

What needs to be done?

  1. On Chrome OS devices, some offsets and sizes are currently read from an FMAP section in Google's flashmap format. This is currently controlled by CONFIG_CHROMEOS, but should be migrated to a separate Kconfig key so that it can be used for other purposes as well. To begin with, the build system can just assume that an FMAP region will be manually inserted after the initial build process.
    1. (landed) https://chromium-review.googlesource.com/#/c/260501: Eliminate CONFIG_ELOG_FULL_THRESHOLD and CONFIG_ELOG_SHRINK_SIZE
    2. https://chromium-review.googlesource.com/#/c/250010: Create CONFIG_USE_FLASHMAP to control whether flashmap offsets and sizes should be used
  2. The source tree's representation of the flashmap is currently stored in the board directory of the depthcharge codebase and parsed by the separate cros_bundle_firmware script. The ability to generate an FMAP region—and the per-board dts files—should be moved into the coreboot repository, where it can be used to construct the firmware image. For now, we can use a Kconfig option to explicitly specify the path to the desired "Chrome OS 8MB" dts file. There already exists a prototype reimplementation of much of cros_bundle_firmware that might be a reasonable reference for this work, but unfortunately, much of its code is spent dealing with the Python/C interface.
    1. (defunct) https://chromium-review.googlesource.com/#/c/251281: Begin fmaptool, a C utility for generating FMAP sections given a dts file
    2. https://chromium-review.googlesource.com/#/c/255031: Introduce the fmd ("flashmap descriptor") language and compiler
  3. Migrate flash offsets and sizes out of Kconfig in favor of centralizing them in the flashmap description. This will significantly reduce the amount of configuration duplication within each board, as well as preventing many mismatch problems. Some of these numbers (e.g. the CBFS offset—or, preferably, the FMAP one instead—because the romstage isn't relocatable on x86) are needed at compile time, so we'll need to generate a header that allows them to be used programmatically.
  4. Make the coreboot build system capable of inserting blobs into the firmware image, external to the CBFS. One important question here is whether the FMAP deserves to go in a section of its own, or be placed in the CBFS: it's a bit of a chicken-and-egg problem, because the location of the CBFS could be stored in the FMAP so it didn't need to be baked into the coreboot build itself. This may require changes to cbfstool, but will allow the replacement of cros_bundle_firmware.
  5. Generate per-board IFD headers based on the per-board plaintext dts files and (possibly) some binary template. This should allow us to eliminate this class of blobs from the repositories. This may require some changes to ifdtool.
  6. Add a default set of dts files for e.g. Intel, AMD, and ARM CPUs with various flash chip sizes. Remove the remnants of the flash offset/size Kconfig keys, and turn CONFIG_USE_FLASHMAP Kconfig key into CONFIG_CUSTOM_FLASHMAP, which then allows you to choose a non-default (e.g. Chrome OS) layout for your particular build.
  7. Implement the ability to build images with the same layout as an existing image. This will help users to easily build modified images for devices that already run coreboot. Once again, the issue of where the FMAP should be placed is important, because we need to be able to find it in the existing image.
  8. Add functionality to copy nonredistributable portions of an existing image into a new one. This way, users who own a hardware device that ships with vender blobs will be able to build firmware images for it without violating vendor agreements.