# x86_64 architecture documentation

This section documents coreboot’s x86_64 support. When enabled, every coreboot stage is built for x86_64, contrary to UEFI’s implementation that only runs some stages in x86_64. On UEFI the PEI phase, which is x86_32, brings up DRAM and installs page tables for the x86_64 DXE and BDS phases.

## Toolchain requirements for x86_64 support

- The compiler must support generating code for the large memory model (-mcmodel=large). It’s supported since GCC 4.4.

  Page tables can be used to provide security benefits, such as by marking memory as non-executable or removing it entirely. This could be useful for SMM to mark regular DRAM as NX.

  The large memory model causes the compiler to emit 64bit addressing instructions, which increases code size. In theory, this is roughly made up for by the faster execution of the x86_64 code.

- All x86 coreboot stages and payloads must be loaded below 4GiB in physical memory. When jumping to the payload coreboot will drop from long mode back to protected mode to keep compatibility with these payloads.

## Comparison to UEFI

On UEFI the SEC and PEI phases (similar to coreboot’s bootblock and romstage) are run in x86_32 mode. The following (guessed) reasons are likely:

- There’s no need for x86_64 as memory hasn’t been trained yet. The whole 4GiB address space, including CAR, memory mapped SPI flash and PCI BARs, are accessible in x86_32.

- When the EFI specification was written compilers did not support large memory model, required in CAR when using a 1:1 page mapping

- Code is 20% bigger in x86_64 due to large memory model where pointers and function calls always use 8 byte addressing. However flash size was very limited, compared to today’s flash chips, when the EFI spec was written.

## Current software constraints for x86_64 support

The following constraints are coreboot limitations as it was intended to run in protected mode only. The code still assumes 32bit pointers in some places and thus:

- The high dword of pointers must always be zero.

- All memory returned by malloc must be below 4GiB in physical memory.

- All code that is to be run must be below 4GiB in physical memory.

- CBMEM must reside below 4GiB in physical memory.

Any software within coreboot must not access memory resources above 4GiB until end of BS_DEV_RESOURCES in ramstage. Only at that point the full memory map is known and identity mapped.

## Supported boards

On the supported boards you can enable x86_64 compilation by setting the Kconfig `USE_X86_64_SUPPORT`. This config option is enabled if the SOC/CPU selects `HAVE_X86_64_SUPPORT`.

## Protected mode wrappers

On some platforms binary blobs are run to initialize parts of the hardware. When these binary blobs have been compiled for x86_32, then coreboot must switch to protected mode in order to call and run the blobs. Once the invoked blobs finish running, coreboot needs to switch back to long mode.

Since every BLOB is different a SoC must be enabled to support x86_64 mode by providing the correct wrapper around the x86_32 BLOBs.

## TODO

- Support more platforms

- Fix running VGA Option ROMs

- Fix running MRC.bin (Sandy Bridge / Haswell boards)

- Identity map memory above 4GiB in ramstage

- Fine grained page tables for SMM:

  - Must not have execute and write permissions for the same page.

  - Must only allow TSEG pages to be marked as executable.

  - Must reside in SMRAM.

  - Must be placed together with SMM rmodule.

- Support 64bit PCI BARs above 4GiB

- Jump to compatible payloads in long mode

## Porting other boards

- Fix compilation errors

- Test how well CAR works with x86_64 and paging

- Improve mode switches

- Test libgfxinit / VGA Option ROMs / FSP

## Known bugs on real hardware

According to Intel x86_64 mode hasn’t been validated in CAR environments. However, coreboot developers working on x86_64 support have tried this on various Intel platforms, and so far haven’t found any issues with CAR when running in x86_64 mode.

## Known bugs on KVM enabled QEMU

The `x86_64` reference code runs fine in QEMU’s soft-cpu, but has serious issues when using KVM mode on some machines. This is due to various mechanisms trying to accelerate the code execution.

Known issues in QEMU:

- After entering long mode, the FPU doesn’t work anymore, including accessing MMX registers. It works fine before entering long mode. It works fine when switching back to protected mode. Other registers, like SSE registers, are working fine.

- Reading from virtual memory, when the page tables are stored in ROM, causes the MMU to abort the “page table walking” mechanism when the lower address bits of the virtual address to be translated have a specific pattern. Instead of loading the correct physical page, the one containing the page tables in ROM will be loaded and used, which breaks code and data as the page table doesn’t contain the expected data. This in turn leads to undefined behaviour whenever the ‘wrong’ address is being read.

- Disabling paging in compatibility mode crashes the CPU.

- Returning from long mode to compatibility mode crashes the CPU.

- Entering long mode crashes on AMD host platforms.