1 TODO list for libguestfs
2 ======================================================================
4 This list contains random ideas and musings on features we could add
5 to libguestfs in future.
12 The API needs more test coverage, particularly lesser-used system
15 The big unresolved issue is UID/GID mapping between guest filesystem
16 IDs and the host. It's not easy to automate this because you need
17 extra details about the guest itself in order to get to its
18 UID->username map (eg. /etc/passwd from the guest).
23 Complete the Haskell bindings (see discussion on haskell-cafe).
28 Add bindtests to PHP bindings.
33 Complete the bind tests - must test the return values and error cases.
35 virt-inspector - make libvirt XML
36 ---------------------------------
38 It should be possible to generate libvirt XML from virt-inspector
39 data, at least partially. This would be just another output type so:
41 virt-inspector --libvirt guest.img
43 Note that recent versions of libvirt/virt-install allow guests to be
44 imported, so this is not so useful any more.
46 "Standalone/local mode"
47 -----------------------
49 Instead of running guestfsd (the daemon) inside qemu, there should be
50 an option to just run guestfsd directly.
52 The architecture in this mode would look like:
67 (1) This only makes sense if we are running as root.
69 (2) There is no console / kernel messages in this configuration, but
70 we might consider capturing stderr from the daemon.
72 (3) guestfs_config and guestfs_add_drive become no-ops.
74 Obviously in this configuration, commands are run directly on the
75 local machine's disks. You could just run the commands themselves
76 directly, but libguestfs provides a convenient API and language
77 bindings. Also deals with tricky stuff like parsing the output of the
78 LVM commands. Also we get to leverage other code such as
81 This is mainly useful from live CDs, ie. virt-p2v.
83 Should we bother having the daemon at all and just link the guestfsd
84 code directly into libguestfs?
86 Ideas for extra commands
87 ------------------------
89 General glibc / core programs:
115 Other initrd-* commands
116 -----------------------
123 Simple editing of configuration files
124 -------------------------------------
126 Some easy non-Augeas methods to edit configuration files.
129 replace /etc/file key value
131 which would look in /etc/file for any instances of
137 and replace them with
143 That would solve about 50% of reconfiguration needs, and for the
144 rest you'd use Augeas, 'download'+'upload' or 'edit'.
146 RWMJ: I had a go at implementing this, but it's quite error-prone to
147 do this sort of editing inside the C-based daemon code. It's far
148 better to do it with Augeas, or else to use an external language like
154 Currently we can't do Perl "one-liners". ie. The current syntax for
155 any short Perl one-liner would be:
157 perl -MSys::Guestfs -e '$g = Sys::Guestfs->new(); $g->add_drive ("foo"); $g->launch; $g->mount ("/dev/sda1", "/"); ....'
159 You can see we're well beyond a single line just getting to the point
160 of adding drives and mounting.
164 $h = create ($filename, \"/dev/sda1\" => \"/\");
166 $h = create ([$file1, $file2], \"/dev/sda1\" => \"/\");
168 To mount read-only, add C<ro =E<gt> 1> like this:
170 $h = create ($filename, \"/dev/sda1\" => \"/\", ro => 1);
172 which is equivalent to the following sequence of calls:
174 $h = Sys::Guestfs->new ();
175 $h->add_drive_ro ($filename);
177 $h->mount_ro (\"/dev/sda1\", \"/\");
179 Command-line form would be:
181 perl -MSys::Guestfs=:all -e '$_=create("guest.img", "/dev/sda1" => "/"); $_->cat ("/etc/fstab");'
183 That's not brief enough for one-liners, so we could have an extra
184 autogenerated module which creates a Sys::Guestfs handle singleton
185 (the handle is an implicit global variable as in guestfish), eg:
187 perl -MSys::Guestfs::One -e 'inspect("guest.img"); cat ("/etc/fstab");'
189 How would editing files work?
195 http://search.cpan.org/~rgiersig/IO-Tty-1.08/Pty.pm
196 http://www.perlmonks.org/index.pl?node_id=582185
198 Note that pty requires cooperation inside the C code too (there are
199 two sides to a pty, and one has to be handled after the fork).
201 [I tried to implement this in the new C virt-rescue, but it doesn't
202 work. qemu is implementing its own ptys, and they are broken. Need
205 Windows-based daemon/appliance
206 ------------------------------
208 See discussion on list:
209 https://www.redhat.com/archives/libguestfs/2009-November/msg00165.html
214 Add -drive file=...,lock=exclusive and -drive file=...,lock=shared
216 Change libguestfs and libvirt to do the right thing, so that multiple
217 instances of qemu cannot stomp on each other.
222 For multi-level disk images such as live CDs:
223 http://rwmj.wordpress.com/2009/07/15/unpack-the-russian-doll-of-a-f11-live-cd/
225 It's possible with libguestfs to recursively look for anything that
226 might be a filesystem, mount-{,loop} it and look in those, revealing
227 anything in a disk image.
229 However this won't work easily for VM disk images in the disk image.
230 One would have to download those to the host and launch another
233 [Not sure this is such a good idea. See also live CD inspection idea below.]
235 Map filesystems to disk blocks
236 ------------------------------
238 Map files/filesystems/(any other object) to the actual disk
245 See also contribs/visualize-alignment/
247 Integration with host intrusion systems
248 ---------------------------------------
250 Perfect way to monitor VMs from outside the VM. Look for file
251 hashes, log events, login/logout etc.
253 http://www.ossec.net/
254 http://la-samhna.de/samhain/
255 http://sourceforge.net/projects/aide/
256 http://osiris.shmoo.com/
257 http://sourceforge.net/projects/tripwire/
262 https://www.redhat.com/archives/libguestfs/2010-June/msg00053.html
263 https://www.redhat.com/archives/libguestfs/2010-June/msg00079.html
265 Freeze/thaw filesystems
266 -----------------------
268 Access to these ioctls:
269 http://git.kernel.org/linus/fcccf502540e3d7
271 Tips for new users in guestfish
272 -------------------------------
275 Tip: You need to 'add disk.img' or 'alloc disk.img nn' to make a new image.
276 Type 'notips' to disable tips permanently.
278 Tip: You need to type 'run' before you can see into the disk image.
280 Tip: Use 'list-filesystems' to see what filesystems are available.
281 ><fs> list-filesystems
283 Tip: Use 'mount fs /' to mount a filesystem.
284 ><fs> mount /dev/vda1 /
285 Tip: Use 'll /' to view the filesystem or ...
288 Could we make guestfish interactive if commands are used without params?
289 ------------------------------------------------------------------------
299 How can we solve these common user problems?
301 [space for common problems here]
303 Better support for encrypted devices
304 ------------------------------------
306 Currently LUKS support only works if the device contains volume
307 groups. If it contains, eg., partitions, you cannot access them.
308 We would like to add:
310 - Direct access to the /dev/mapper device (eg. if it contains
311 anything apart from VGs).
316 Display the structure of an image file as a PS.
318 Greater use of blkid / libblkid
319 -------------------------------
321 guestfs_zero should use wipefs. See wipefs(8).
323 There are various useful functions in libblkid for listing partitions,
324 devices etc which we are essentially duplicating in the daemon. It
325 would make more sense to just use libblkid for this.
327 There are some places where we call out to the 'blkid' program. This
328 might be replaced by direct use of the library (if this is easier).
333 Eric Sandeen pointed out the blktrace tool which is a better way of
334 capturing traces than using patched qemu (see
335 contrib/visualize-alignment). We would still use the same
336 visualization tools in conjunction with blktrace traces.
341 At the moment guestfish uses an ad hoc parser which has many
342 shortcomings. We should change to using a lex/yacc-based scanner and
343 parser (there are better parsers out there, but yacc is sufficient and
344 very widely available).
346 The scanner must deal with the case of parsing a whole command string,
347 eg. for a command that the user types in:
349 ><fs> add-drive-opts "/tmp/foo" readonly:true
351 and also with parsing single words from the command line:
353 guestfish add-drive-opts /tmp/foo readonly:true
355 Note the quotes are for scanning and don't indicate types.
357 We should also allow variables and expressions as part of this new
361 set product inspect-get-product-name %{roots[0]}
363 % is better than $ because of shell escaping and confusion with shell
366 Can we combine this with ability to set and read environment
367 variables? Currently guestfish uses many environment variables like
368 $EDITOR without any corresponding ability to set them.
370 set EDITOR /usr/bin/emacs
371 echo $EDITOR # or %{EDITOR}
372 edit /etc/resolv.conf
374 live CD inspection for Windows 7
375 --------------------------------
377 Windows 7 install CDs are quite different and pretty impenetrable.
378 There are no obvious files to parse.
383 ntfsprogs actually has a lot more useful tools than we currently
384 use. Interesting ones are:
386 ntfslabel: display or change filesystem label (we should unify all
387 set*label APIs into a single set_vfs_label which can deal with any
390 ntfsclone: clone, image, restore, rescue NTFS
392 ntfsinfo: print various information about NTFS volume and files
394 ntfs streams: extract alternate streams from NTFS files
396 ntfsck: checker for NTFS filesystems
406 More mkfs_opts options
407 ----------------------
409 Useful options to offer:
413 Use /proc/self/mountinfo
414 ------------------------
416 This file contains lots of interesting information about
417 what is mounted and where. eg:
419 16 21 0:3 / /proc rw,relatime - proc /proc rw
420 17 21 0:16 / /sys rw,relatime - sysfs /sys rw,seclabel
421 18 23 0:5 / /dev rw,relatime - devtmpfs udev rw,seclabel,size=1906740k,nr_inodes=476685,mode=755
422 26 21 253:3 / /home rw,relatime - ext4 /dev/mapper/vg-lv_home rw,seclabel,barrier=1,data=ordered
424 This could be used instead of current hairy code to parse the output
425 of the 'mount' command. We could add new APIs to return kernel mount
426 options, type of filesystem at a mountpoint etc.
428 guestfish drive letters
429 -----------------------
431 There should be an option to mount all Windows drives as separate
432 paths, like C: => /c/, D: => /d/ etc.
434 More inspection features
435 ------------------------
439 - last time the software was updated
440 - last user who logged in
443 Integrate virt-inspector with CMDBs
444 -----------------------------------
446 Either integrate virt-inspector with Configuration Management
447 Databases (CMDBs) or at least check that virt-inspector produces the
448 right range of data so that integration would be possible. The
449 standards for CMDBs come from the DMTF, see eg:
451 http://dmtf.org/news/pr/2009/7/dmtf-releases-cmdbf-standard-federating-configuration-management-data
453 Efficient way to visit all files
454 --------------------------------
456 https://rwmj.wordpress.com/2010/12/15/tip-audit-virtual-machine-for-setuid-files/#content
458 A naive method would look like:
460 g#visit ~return_stats:true "/" (
465 However this has two disadvantages:
467 - requires hand-written custom bindings in each language
468 - unclear about locking, thread-safety and re-entrancy of handle g
470 A better way would be to have some sort of explicit "download all
471 filenames and stat structures", which could then be iterated over:
473 let files = g#find_opts ~return_stats:true "/" in
479 The problem with this is that 'files' is going to be larger than a
482 This leads to thinking about changes to the protocol / generator to
483 make this simpler. The proposal would be to add RBigStringList,
484 RBigStructList [or RBig (Ranytype ...)]. These would work like
485 FileOut, in that they would use file streaming to stream XDR
486 structures (probably written to a file on the library side).
487 Generated code would hide most of the implementation.
489 We also need to think about security issues: is it possible for the
490 daemon to keep sending back data forever, and if so what happens on
493 [Users can now use virt-ls to solve some of these problems, but it is
494 not a general solution at the API level]
496 Interactive disk creator
497 ------------------------
499 An interactive disk creator program.
501 Attach method for disconnected operation
502 ----------------------------------------
504 http://libguestfs.org/guestfs.3.html#guestfs_set_attach_method
506 "Librarian" has an idea that he should be able to attach to a regular
507 appliance, but disconnect from it and reconnect to it later. This
508 would be some sort of modified attach method (see link above).
510 The complexity here is that we would no longer have access to
511 stdin/stdout (or we'd have to direct that somewhere else).
513 GObject Introspection
514 ---------------------
516 We periodically get asked to implement gobject-introspection (it's a
519 http://live.gnome.org/GObjectIntrospection
521 This would require a separate Gtk C API since the main guestfs handle
522 would have to be encapsulated in a GObject. However the main
523 difficulty is that the annotations supported to define types are not
524 very rich. Notably missing are support for optional arguments
525 (defined but not implemented), support for structs (unless mapped to
528 Also note that the libguestfs API is not "object oriented".
530 libosinfo mappings for virt-inspector
531 -------------------------------------
533 Return libosinfo mappings from inspection API.
538 - touch /.unconfigured ?
539 - other Spacewalk / RHN IDs (?)
541 - Puppet registration
544 (see: https://github.com/clalancette/oz/blob/e74ce83283d468fd987583d6837b441608e5f8f0/oz/Windows.py )
545 - blue skies: change the background image
546 - (librarian suggests ...)
547 . install a firstboot script virt-sysprep --script=/tmp/foo.sh
548 . run an external shell script
549 . run external guestfish script virt-sysprep --fish=/tmp/foo.fish
550 . rm /var/cache/apt/archives/*
551 - /var/run/* and pam_faillock's data files
552 - homedirs/.ssh directory, especially /root/.ssh (Steve Grubb)
553 - if drives are encrypted, then dm-crypt key should be changed
554 and drives all re-encrypted
557 Rpm uses nss. Nss sets up its crypto database in
558 /etc/pki. Depending on how long the machine ran before cloning, you
559 may have picked up some certificates or things. This is an area
560 that you would want to look into.
561 - secure erase of inodes etc using scrub (Steve Grubb)
562 - other directories that could require cleaning include:
566 /var/lib/AccountService/users/*
570 (thanks Marko Myllynen, James Antill)
571 - remove or modify UUIDs in /etc/fstab (eg. on Ubuntu)
572 (thanks Joshua Daniel Franklin)
574 Launch remote sessions over ssh
575 -------------------------------
577 We had an idea you could add a launch method that uses ssh, ie. all
578 febootstrap and qemu commands happen the same as now, but prefixed by
579 ssh so it happens on a remote machine.
581 Note that proper remote support and integration with libvirt is
582 different from this, and people are working on that. ssh would just
585 virt-make-fs and virt-win-reg need to not be in Perl
586 ----------------------------------------------------
588 Probably they should be in C or OCaml.
590 Integrate snap-type functionality in inspection tools
591 -----------------------------------------------------
593 Mo Morsi's "snap" program lets you describe a guest as the list of
594 packages (eg. RPMs) installed + changes made to those RPMs + files
597 http://projects.morsi.org/wiki/Snap
599 This results in a compact description of the guest. He even managed
600 to do a kind of migration of guests by simply recreating the guest
601 from the description on the target machine.
603 It would be ideal to integrate this and/or use inspection to do this.