AVATT: Difference between revisions

From coreboot
Jump to navigation Jump to search
(updated it to be in sync with the latest AVATT code)
Line 1: Line 1:
This page is about the "[http://code.google.com/soc/2008/coresystems/appinfo.html?csaid=ED0729A9DB64B39B All Virtual All The Time]" GSoC 2008 project.
This page is about the "[http://code.google.com/soc/2008/coresystems/appinfo.html?csaid=ED0729A9DB64B39B All Virtual All The Time]" GSoC 2008 project.
This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a custom Linux kernel modified to include [http://kvm.qumranet.com KVM] support, [http://www.uclibc.org uClibc], a custom [http://www.busybox.net busybox] image, and whatever is needed to get KVM up and running in the ROM image. See this [http://panzer.utcluj.ro/~alien/coreboot/AVATT/screenshot.png screenshot] if you want to see it in action.
This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a custom Linux kernel modified to include [http://kvm.qumranet.com KVM] or [http://openvz.org OpenVZ] support, a small userland on top of [http://www.uclibc.org uClibc], consisting mostly from a custom [http://www.busybox.net busybox] image, and whatever is needed to get KVM or OpenVZ up and running. See this [http://panzer.utcluj.ro/~alien/coreboot/AVATT/screenshot.png screenshot] if you want to see it in action.
 
== Prebuilt ROM image ==
== Prebuilt ROM image ==


Line 8: Line 8:
These are some copy/paste instructions that will fetch it and run it in qemu(yes, you should already have it installed):
These are some copy/paste instructions that will fetch it and run it in qemu(yes, you should already have it installed):


  wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/bios.bin -O /tmp/bios.bin
  wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/KVM/bios.bin -O /tmp/bios.bin    # This one is quite old, and may not work anymore
or
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/OpenVZ/bios.bin -O /tmp/bios.bin # This one is based on OpenVZ, but it currently lacks the vzctl utility
 
  wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O /tmp/vgabios-cirrus.bin
  wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O /tmp/vgabios-cirrus.bin
  qemu-system-x86_64 -L /tmp -hda /dev/null -m 768 -serial stdio
  qemu-system-x86_64 -L /tmp -hda /dev/null -m 768 -serial stdio


If your bandwidth allows, you can also get a [http://panzer.utcluj.ro/~alien/coreboot/AVATT/diskimage.img.bz2 disk image], extract it and pass it as the -hda parameter of qemu instead the /dev/null file so that you can mount it and try to run VMs of the CD ISO images present inside it. This won't work for now, see the "Current Status" section to find out why. At [http://panzer.utcluj.ro/~alien/coreboot/AVATT this website] you can find other various data concerning this project, freely available for download.
If your bandwidth allows, you can also get a [http://panzer.utcluj.ro/~alien/coreboot/AVATT/diskimage.img.bz2 disk image] usable with the KVM version. You should extract it and pass it as the -hda parameter of qemu instead the /dev/null file so that you can mount it and try to run VMs of the CD ISO images present inside it. This won't work for now, see the "Current Status" section to find out why. At [http://panzer.utcluj.ro/~alien/coreboot/AVATT this website] you can find other various data concerning this project, freely available for download.


== Building from scratch ==
== Building from scratch ==
Line 19: Line 22:
First of all, you need a x86 Linux distribution with gcc > 4.2, since currently the build fails under x86-64 and gcc-3.x doesn't support some compiler flags we are using. Eventually this will get fixed in the future.
First of all, you need a x86 Linux distribution with gcc > 4.2, since currently the build fails under x86-64 and gcc-3.x doesn't support some compiler flags we are using. Eventually this will get fixed in the future.


In order to build this stuff you need a modified version of [[buildrom]] that can be cloned from my git repo using this command:
In order to build this stuff you need a build system consisting of forked versions of [[buildrom]] and [http://buildroot.uclibc.org/ buildroot] that can be cloned from my git repo using this command:
  git clone http://repo.or.cz/r/kvm-coreboot.git
  git clone http://repo.or.cz/r/avatt.git


You also need to have installed some of the [http://www.coreboot.org/Buildrom#Prerequisites prerequisites of buildrom]. The slackware folks ;] may prefer to skip this and then fix by hand any encountered build failures by installing only the missing programs.
You also need to have installed some of the [http://www.coreboot.org/Buildrom#Prerequisites prerequisites of buildrom], and maybe even others that may be missing from your box. I'll eventually make a list of needed tools for common distros...


=== How to build ===
=== How to build ===
Just type ''make'' from within the buildrom directory and everything should get built.
Choose the version you want to use by using eithret of the "make openvz" or "make kvm" commands. '''Currently KVM is broken, and OpenVZ works but lacks the vzctl utility(whic is currently in progress).'''
Just type ''make'' and everything should get built just fine.


(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:
(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:
  make menuconfig (for buildrom)
  make buildrom-config
make kernel-config
  make buildroot-config  
make uclibc-config
  make busybox-config  


=== How to run the ROM image you just built ===
=== How to run the ROM image you just built ===


You should read the [[QEMU Build Tutorial]] and the [[QEMU]] pages to see how to run a BIOS image with qemu, but since this howto aims to be as short as possible, here you have just the minimum information you need to get it running ASAP
You should read the [[QEMU Build Tutorial]] and the [[QEMU]] pages to see how to run a BIOS image with qemu.
Currently the OpenVZ target can be tested by issuing "make test". The KVM branch must be tested by manually running qemu in 64bit mode.


It's quite handy to have a symlink to "deploy/emulation-qemu-x86.rom" in your $HOME with the name bios.bin, and in the same directory you should have the Cirrus VGA BIOS ROM image.
== Current status ==
Here you have cut/paste commands for this scenario:
=== The contents of the OpenVZ ROM image===
ln -s deploy/emulation-qemu-x86.rom ~/bios.bin
* Linux kernel version 2.6.24 with the OpenVZ patch applied and a custom minimalist config
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O ~/vgabios-cirrus.bin
* uClibc 0.9.30.1
* busybox 1.13.2


Then you must run qemu in 64bit mode so that we can get support for the SVM instructions, just like you run a prebuilt ROM image:
===What works===
qemu-system-x86_64 -L ~ -hda /dev/null -m 768 -serial stdio
====OpenVZ image====


Notes:
* The kernel compiles fine on x86, not yet tested on x86-64 (feedback is welcome), although it should work just fine due to the fact that we now are using buildroot as a full toolchain, and no CFLAGS/LDFLAGS ugly hacks anymore
* The -m 768 parameter is mandatory, since otherwise the kernel will panic because I hardcoded the RAM size to 768M in coreboot. It was 128M before, and that proved to be too small. Sorry for those who have few MBs of RAM.
* There's no userland utility yet, but it should be straightforward to get it compiled
* The -serial parameter is good for those who have keyboard issues with the default SDL output of qemu, like arrow keys not working, etc. This way you can also use the terminal you used to launch qemu as a second console and here all the keys should work fine.
* It was tested and should compile on the following Linux distributions:
**Ubuntu 8.10 & 9.04 alpha
(any feedback is welcome)


== Current status ==
===What doesn't work===
=== The contents of the ROM image===
====The KVM image is outdated====
* Linux kernel version 2.6.22.2 with tiny patches and a custom config
'''As you probably already know, khe KVM image is outdated and currently unsupported'''
* uClibc daily SVN snapshots with aio support added and a custom config - only the .so files are included in the ROM image
* zlib 1.2.3 - only the .so file is included
* ncurses 5.6 - the .so file and some other small necessary files are included
* kvm-74 with some small patches - the extra BIOSes, the nbd tool and the docs are missing
* busybox 1.1.3 with a custom config - NEEDS VERSION BUMP TO SOMETHING NEWER
 
====What works====
* everything compiles fine on x86 but not yet on x86-64
* qemu-img works ok, and so does busybox
* the kvm binary works when ran in qemu fallback mode if either the svn instructions or the /dev/kvm device file are missing. OpenBSD's CD ISO image is starting but crashes later on because of unknown reasons, as you can see in these [http://panzer.utcluj.ro/~alien/coreboot/AVATT/screenshots/running/ screenshots]
* It was tested and should compile on the following Linux distributions:
**Ubuntu 8.04 & 8.10 alpha
**Debian Lenny
**Gentoo


====What doesn't work====
Last time I checked it had some of these issues:
* kvm crashes due to the poor Thread Local Store(TLS) implementation of uClibc, when ran in KVM-accelerated mode
* kvm crashes due to the poor Thread Local Store(TLS) implementation of uClibc, when ran in KVM-accelerated mode
The build failed on these distributions:
The build failed on these distributions:
* Gentoo hardened - failed with this error:  
* Gentoo hardened - failed with this error:  
Line 80: Line 72:


=== TODO ===
=== TODO ===
* make the kvm userspace tool not to crash anymore. A possible solution would be to fix the TLS issues from the version of uClibc we currently use (daily snapshots from their SVN tree). This could even mean porting the uClibc-nptl branch to x86 if both of the x86 linuxthreads branches prove to be too hard to fix.
* Complete the OpenVZ image by adding the vzctl and (maybe) vzquota - work in progress - current main focus
* user-friendly tool that can create and run virtual machines.
* Port it on real hardware and make some cool demos - after we get full OpenVZ support
* automatically starting the virtual machines at boot.
* Make the kvm userspace tool not to crash anymore, and keep it active. We need a fix for the TLS issues of uClibc. The uClibc developers are working on it but there's no time frame in which it should be ready.
* get the network to work in qemu since it fails with both coreboot v2 and v3.
* user-friendly tool that can create and run virtual machines in both OpenVZ and KVM (after it gets usable).
* integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support, to really be useful.
* automatically starting the virtual machines at boot - easy to do with OpenVZ after vzctl is done (they have some initscripts), but it should be done manually for KVM.
* fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for some extra wasted ROM space caused by the 64bit code. We can't cross-compile because we're not using a full toolchain, like buildroot does.
* get the network to work in qemu since it fails with both coreboot v2 and v3 - this is most likely a coreboot issue, any help is welcome
* keep the versions as up-to-date as possible but also compatible with each other
* integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support, to really be useful - we're quite far away from this goal
* fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for some extra wasted ROM space caused by the 64bit code. We can't cross-compile because we're not using a full toolchain, like buildroot does - non-issue anymore, after we switched to buildroot
* keep the versions as up-to-date as possible but also compatible with each other - non-issue anymore, after we switched to buildroot, since they can easily update the software


== Contact ==
== Contact ==
If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi.magherusan NO@SPAM net.utcluj.ro and I'll do it myself.
If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi NO@SPAM cs.utcluj.ro and I'll do it myself.


You can also find me on IRC most of the time, and I'll eventually answer your questions when I come back.  
You can also find me on IRC most of the time, and I'll eventually answer your questions when I come back.  

Revision as of 00:53, 23 March 2009

This page is about the "All Virtual All The Time" GSoC 2008 project. This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a custom Linux kernel modified to include KVM or OpenVZ support, a small userland on top of uClibc, consisting mostly from a custom busybox image, and whatever is needed to get KVM or OpenVZ up and running. See this screenshot if you want to see it in action.

Prebuilt ROM image

I have prepared a pre-built image that you can use in qemu to see what's this all about without having to spend considerable time to compile it.

These are some copy/paste instructions that will fetch it and run it in qemu(yes, you should already have it installed):

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/KVM/bios.bin -O /tmp/bios.bin    # This one is quite old, and may not work anymore

or

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/OpenVZ/bios.bin -O /tmp/bios.bin # This one is based on OpenVZ, but it currently lacks the vzctl utility
wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O /tmp/vgabios-cirrus.bin
qemu-system-x86_64 -L /tmp -hda /dev/null -m 768 -serial stdio

If your bandwidth allows, you can also get a disk image usable with the KVM version. You should extract it and pass it as the -hda parameter of qemu instead the /dev/null file so that you can mount it and try to run VMs of the CD ISO images present inside it. This won't work for now, see the "Current Status" section to find out why. At this website you can find other various data concerning this project, freely available for download.

Building from scratch

Needed tools

First of all, you need a x86 Linux distribution with gcc > 4.2, since currently the build fails under x86-64 and gcc-3.x doesn't support some compiler flags we are using. Eventually this will get fixed in the future.

In order to build this stuff you need a build system consisting of forked versions of buildrom and buildroot that can be cloned from my git repo using this command:

git clone http://repo.or.cz/r/avatt.git

You also need to have installed some of the prerequisites of buildrom, and maybe even others that may be missing from your box. I'll eventually make a list of needed tools for common distros...

How to build

Choose the version you want to use by using eithret of the "make openvz" or "make kvm" commands. Currently KVM is broken, and OpenVZ works but lacks the vzctl utility(whic is currently in progress). Just type make and everything should get built just fine.

(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:

make buildrom-config
make buildroot-config 

How to run the ROM image you just built

You should read the QEMU Build Tutorial and the QEMU pages to see how to run a BIOS image with qemu. Currently the OpenVZ target can be tested by issuing "make test". The KVM branch must be tested by manually running qemu in 64bit mode.

Current status

The contents of the OpenVZ ROM image

  • Linux kernel version 2.6.24 with the OpenVZ patch applied and a custom minimalist config
  • uClibc 0.9.30.1
  • busybox 1.13.2

What works

OpenVZ image

  • The kernel compiles fine on x86, not yet tested on x86-64 (feedback is welcome), although it should work just fine due to the fact that we now are using buildroot as a full toolchain, and no CFLAGS/LDFLAGS ugly hacks anymore
  • There's no userland utility yet, but it should be straightforward to get it compiled
  • It was tested and should compile on the following Linux distributions:
    • Ubuntu 8.10 & 9.04 alpha

(any feedback is welcome)

What doesn't work

The KVM image is outdated

As you probably already know, khe KVM image is outdated and currently unsupported

Last time I checked it had some of these issues:

* kvm crashes due to the poor Thread Local Store(TLS) implementation of uClibc, when ran in KVM-accelerated mode

The build failed on these distributions:

  • Gentoo hardened - failed with this error:
Internal error convert_magic 00006000 != a5a5a5a5

The convert_magic error indicates that the assembler/linker didn't initialize struct image_params at the right place.

  • Debian Etch - fails to strip the kvm binary due to faulty binutils:
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value

TODO

  • Complete the OpenVZ image by adding the vzctl and (maybe) vzquota - work in progress - current main focus
  • Port it on real hardware and make some cool demos - after we get full OpenVZ support
  • Make the kvm userspace tool not to crash anymore, and keep it active. We need a fix for the TLS issues of uClibc. The uClibc developers are working on it but there's no time frame in which it should be ready.
  • user-friendly tool that can create and run virtual machines in both OpenVZ and KVM (after it gets usable).
  • automatically starting the virtual machines at boot - easy to do with OpenVZ after vzctl is done (they have some initscripts), but it should be done manually for KVM.
  • get the network to work in qemu since it fails with both coreboot v2 and v3 - this is most likely a coreboot issue, any help is welcome
  • integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support, to really be useful - we're quite far away from this goal
  • fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for some extra wasted ROM space caused by the 64bit code. We can't cross-compile because we're not using a full toolchain, like buildroot does - non-issue anymore, after we switched to buildroot
  • keep the versions as up-to-date as possible but also compatible with each other - non-issue anymore, after we switched to buildroot, since they can easily update the software

Contact

If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi NO@SPAM cs.utcluj.ro and I'll do it myself.

You can also find me on IRC most of the time, and I'll eventually answer your questions when I come back. My nickname is "alien".


Thanks for passing by,

Cristi

Public domain I, the copyright holder of this work, hereby release it into the public domain. This applies worldwide.

In case this is not legally possible:
I grant anyone the right to use this work for any purpose, without any conditions, unless such conditions are required by law.