[title slide] Back in 2008 we faced a pressing problem with virtualization: How do we look at what's going on inside a virtual machine? [diagram of components of a VM] Let's step back: what is a virtual machine? In nuts and bolts terms, it's a big file or partition containing a disk image, and when it's running, it's a complicated emulation of CPUs, memory, and virtual devices like network cards. It's interesting and necessary to be able to look inside all of those things. ("How many packets are coming out of the virtual network card?" "How is the virtual CPU coping with the load?"). But for the purpose of this talk I'm just going to talk about looking inside that disk image. [disk image] The disk image has a rich internal structure: a Master Boot Record; a boot partition; LVM, which has its own internal structures. What might we want to do with the disk image? Clone the machine, changing a few config files like the hostname. Edit grub.conf in a VM which isn't booting. Audit a VM to find out what licensed software is installed. Is the VM running out of disk space? Offline resizing or backups. Make a new virtual machine from scratch ... [kpartx slide] In 2008 you could look inside the disk image. First of all you'd need to be root. Then you could run a command line tool called 'kpartx' which splits the disk image partitions into device mapper devices (this is why you need to be root). These are actually global devices on your host, visible to everyone. If you're lucky, LVM on the host might find the volume groups located in the disk image, but you might have to adjust the global host LVM configuration to get that to work. If you're unlucky, those could conflict with volume groups already in your host. So if you are root, you should usually be able to mount a guest disk in the host. If your program crashes, of course, it will leave unattached device mapper devices, loopback devices and mount points on the host system. It's not clear from a security point of view if mounting untrusted guest devices on the host as root is a good idea. That said, kpartx is a useful tool if: you are already root on the host, you just want to mount a partition, it's ad hoc (no scripting), you can clean up if you make a mistake, and if you can trust the guests. So we considered how we could improve this process and provide more features. You shouldn't need to be root: If you have a word-processor document, you don't need to be root to edit that document. If you have a JPEG file, you don't need to run GIMP as root to crop it. So why are disk image files any different? You should be able to modify disk images from CGI scripts, or from shell scripts. You shouldn't have to clean up after it. There should be no gotchas or corner cases where it doesn't work. [libguestfs, list of languages, list of names of tools] What is libguestfs? An API for creating, accessing, manipulating and modifying filesystems and disk images. Access from many different programming languages, or the command line. A set of useful tools. And applications built on top. Today is going to be mainly a demonstration of what can be done with libguestfs and the tools we've built around this. [demonstration slide: guestfish on a VM] "Guestfish" is the "guest filesystem interactive shell", and you can just run it on any disk image you happen to find. You don't need to be root, unless you need root to access that particular image. In this case, the image is just a local file so I don't need root. You can see this image is a Linux virtual machine of some sort. > cat /etc/fstab We can use the "cat" command to look for some identification: > cat /etc/motd > cat /etc/redhat-release > cat /etc/debian_version We can also edit files. For example, we can edit the /etc/issue file to change the console login message: > vi /etc/issue Guestfish is the shell-scripting interface to the libguestfs API. It exposes the entire API, and as you can see that's quite large: > help Since the API is quite daunting, we do offer an overview of the whole API in the man page. So I won't go through that here, but I'll just mention a few points. Listing the mount points: > mounts Listing the LVM configuration: > lvs > vgs > pvs There are also commands for creating and resizing and deleting LVM things. All the usual file copy, linking, edit, move, make directory, list files etc. commands are available. [demonstration slide: equivalent in languages] And all of this is also available through the language bindings, so you can, as an example, write a non-root CGI script in Python that has full access to the API. This Perl script shows the kind of ad hoc querying (and modifications) that are possible using libguestfs and Augeas. [demonstration slide: guestfish on a Fedora installation CD] We're not limited to hard disk images. Floppy images, CD ISOs, and memory cards can all be handled. run list-partitions mkmountpoint /cd mount /dev/vda1 /cd mkmountpoint /squash mount-loop /cd/LiveOS/squashfs.img /squash ... --- Guestfish and the language bindings are quite low level. On top of this base we've built a number of more useful high level tools. [demonstration slide: virt-df] [demonstration slide: virt-inspector] You can see there are possibilities for auditing, inventory and so on. [demonstration slide: virt-win-reg] [demonstration slide: guestmount] [demonstration slide: guestmount, Windows and hivexsh] [end] Red Hat: last year R&D 20.2% of revenue (18% over last 5 years) $130 million per year