AVATT

This page is about the "All Virtual All The Time" GSoC 2008 project. This project aims to create a modified Linux-As-Bootloader coreboot payload that contains a custom Linux kernel modified to include KVM or OpenVZ support, a small userland on top of uClibc, consisting mostly from a custom busybox image, and whatever is needed to get KVM or OpenVZ up and running. See this screenshot if you want to see it in action.

Prebuilt ROM image

I have prepared a pre-built image that you can use in qemu to see what's this all about without having to spend considerable time to compile it.

These are some copy/paste instructions that will fetch it and run it in qemu(yes, you should already have it installed):

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/KVM/bios.bin -O /tmp/bios.bin    # This one is quite old, and may not work anymore

or

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/OpenVZ/bios.bin -O /tmp/bios.bin # This one is based on OpenVZ, but it currently lacks the vzctl utility

wget http://panzer.utcluj.ro/~alien/coreboot/AVATT/BIOS/vgabios-cirrus.bin -O /tmp/vgabios-cirrus.bin
qemu-system-x86_64 -L /tmp -hda /dev/null -m 768 -serial stdio

If your bandwidth allows, you can also get a disk image usable with the KVM version. You should extract it and pass it as the -hda parameter of qemu instead the /dev/null file so that you can mount it and try to run VMs of the CD ISO images present inside it. This won't work for now, see the "Current Status" section to find out why. At this website you can find other various data concerning this project, freely available for download.

Building from scratch

Needed tools

First of all, you need a modern Linux distribution capable of running buildroot and buildrom (anything with gcc > 4.2 should do).

In order to build this stuff you need a build system consisting of forked versions of buildrom and buildroot that can be cloned from my git repo using this command:

git clone http://repo.or.cz/r/avatt.git

You also need to have installed most of the prerequisites of buildrom, and maybe even others that may be missing from your box. I'll eventually update it or make a list of needed tools for common distros...if people report other missing tools.

How to build

Choose the version you want to use by using either of the "make openvz" or "make kvm" commands. Currently KVM is broken, and OpenVZ works but lacks the vzctl utility(whic is currently in progress). Just type make and everything should get built just fine.

(Optional step) If you wish to make some adjustments you may do it by running one of these commands, but normally you shouldn't need to do this:

make buildrom-config
make buildroot-config

How to run the ROM image you just built

You should read the QEMU Build Tutorial and the QEMU pages to see how to run a BIOS image with qemu. Currently the OpenVZ target can be tested by issuing "make test". The KVM branch must be tested by manually running qemu in 64bit mode.

Current status

The contents of the OpenVZ ROM image

• Linux kernel version 2.6.24 with the OpenVZ patch applied and a custom minimalist config
• uClibc 0.9.30.1
• busybox 1.13.2

What works

OpenVZ image

• The kernel compiles fine on x86, not yet tested on x86-64 (feedback is welcome), although it should work just fine due to the fact that we now are using buildroot as a full toolchain, and no CFLAGS/LDFLAGS ugly hacks anymore
• There's no userland utility yet, but it should be straightforward to get it compiled
• It was tested and should compile on the following Linux distributions:
  • Ubuntu 8.10 & 9.04 alpha

(any feedback is welcome)

What doesn't work

The KVM image is outdated

As you probably already know, khe KVM image is outdated and currently unsupported

Last time I checked it had some of these issues:

* kvm crashes due to the poor Thread Local Store(TLS) implementation of uClibc, when ran in KVM-accelerated mode

The build failed on these distributions:

• Gentoo hardened - failed with this error:

Internal error convert_magic 00006000 != a5a5a5a5

The convert_magic error indicates that the assembler/linker didn't initialize struct image_params at the right place.

• Debian Etch - fails to strip the kvm binary due to faulty binutils:

BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value
BFD: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: The first section in the PT_DYNAMIC segment is not the .dynamic section
strip: /home/alien/kvm-coreboot/work/kvm/kvm-74/qemu/x86_64-softmmu/stExbwpM: Bad value

TODO

• Complete the OpenVZ image by adding the vzctl and (maybe) vzquota - work in progress - current main focus
• Port it on real hardware and make some cool demos - after we get full OpenVZ support
• Make the kvm userspace tool not to crash anymore, and keep it active. We need a fix for the TLS issues of uClibc. The uClibc developers are working on it but there's no time frame in which it should be ready.
• User-friendly tool that can create and run virtual machines in both OpenVZ and KVM (after it gets usable).
• Automatically starting the virtual machines at boot - easy to do with OpenVZ after vzctl is done (they have some initscripts), but it should be done manually for KVM.
• Get the network to work in qemu since it fails with both coreboot v2 and v3 - this is most likely a coreboot issue, any help is welcome
• (maybe) integrate the virt-manager daemon inside the ROM image, if it and its dependencies fit the remaining free space. This needs network support, to really be useful - we're quite far away from this goal
• Fix compilation on x86_64 boxes by compiling everything in 64bit mode. We need a 64bit hardware anyway since the SVM instructions are available only on recent 64 bit boxes so this shouldn't matter too much, except for some extra wasted ROM space caused by the 64bit code. We can't cross-compile because we're not using a full toolchain, like buildroot does - non-issue anymore, after we switched to buildroot, but it still needs some testing and feedback
• Keep the versions as up-to-date as possible but also compatible with each other - non-issue anymore, after we switched to buildroot, since it can be used to update the software included in the payload

Contact

If you find anything incomplete or wrong about this page please don't hesitate to fix it. If you can't, please drop me an email at cristi NO@SPAM cs.utcluj.ro and I'll do it myself.

You can also find me on IRC most of the time, and I'll eventually answer your questions when I come back. My nickname is "alien".

Thanks for passing by,

Cristi

I, the copyright holder of this work, hereby release it into the public domain. This applies worldwide. In case this is not legally possible: I grant anyone the right to use this work for any purpose, without any conditions, unless such conditions are required by law.