-Ideas for the Python bindings:
-https://www.redhat.com/archives/fedora-virt/2009-April/msg00114.html
+TODO list for libguestfs
+======================================================================
-----------------------------------------------------------------------
+This list contains random ideas and musings on features we could add
+to libguestfs in future.
-We badly need to actually implement the FTP server mentioned in the
-documentation.
+ - RWMJ
-Or: Implement a FUSE-based filesystem. See the FUSE mountlo
-project which does something similar, albeit only to single
-filesystems:
+FUSE API
+--------
-http://sourceforge.net/project/showfiles.php?group_id=121684&package_id=150116
+The API needs more test coverage, particularly lesser-used system
+calls.
-----------------------------------------------------------------------
+The big unresolved issue is UID/GID mapping between guest filesystem
+IDs and the host. It's not easy to automate this because you need
+extra details about the guest itself in order to get to its
+UID->username map (eg. /etc/passwd from the guest).
-BufferIn and BufferOut should turn into <char *, int> and simple
-strings in other languages that can handle 8 bit clean strings.
-Limit on transfers would still be 2MB for these types.
- - then implement write-file properly
- - and implement read-file
-
-----------------------------------------------------------------------
-
-Implement febootstrap command.
-
-----------------------------------------------------------------------
+Haskell bindings
+----------------
Complete the Haskell bindings (see discussion on haskell-cafe).
-----------------------------------------------------------------------
-
-Complete the bindings tests - must test the return values and
-error cases.
+PHP bindings
+------------
-----------------------------------------------------------------------
+Add bindtests to PHP bindings.
-For virt-inspector:
+Complete bind tests
+-------------------
- - Make a libvirt XML config
+Complete the bind tests - must test the return values and error cases.
- - Test over available OSes
+virt-inspector - make libvirt XML
+---------------------------------
- - Add 'reged' / NT registry support.
+It should be possible to generate libvirt XML from virt-inspector
+data, at least partially. This would be just another output type so:
-----------------------------------------------------------------------
+ virt-inspector --libvirt guest.img
-Use virtio_blk by default. It's faster and more natural.
-Unfortunately it seems like this will rename all devices - see next
-item.
-
-Note: virtio_blk *IS* supported by all our minimum platforms,
-ie. CentOS 5.3, Fedora 11, Debian.
-
-----------------------------------------------------------------------
+Note that recent versions of libvirt/virt-install allow guests to be
+imported, so this is not so useful any more.
"Standalone/local mode"
+-----------------------
Instead of running guestfsd (the daemon) inside qemu, there should be
an option to just run guestfsd directly.
Should we bother having the daemon at all and just link the guestfsd
code directly into libguestfs?
-----------------------------------------------------------------------
-
-PPC problems:
-
- ppc (32 bit) works with qemu from git, however there is no serial console
-
- ppc64 requires extra parameters:
- -M mac99 -cpu ppc64
- however it still fails:
- invalid/unsupported opcode: 01 - 01 - 1a (06301e83) 00000000018c2738 1
- invalid bits: 00400000 for opcode: 0b - 19 - 15 (2d746572) 0000000000009230
-
- no serial console in ppc or ppc64 because no one can tell us what
- console=ttyXX option to use
-
-----------------------------------------------------------------------
-
-Supermin appliance should be moved into febootstrap.
-
-----------------------------------------------------------------------
-
-Extra commands / functionality:
+Ideas for extra commands
+------------------------
General glibc / core programs:
chgrp
- grep (do it locally using pipe?)
- dd (?)
- ln / ln -s
- readlink
- utime / utimes / futimes / futimens / l..
more mk*temp calls
- readdir / readdir-and-stat
- some sort of alloc/fallocate/posix_fallocate call to create empty space
- realpath
- trunc[ate??]
- getfattr (also useful because gives us access to NTFS datastreams)
- setfattr
ext2 properties:
chattr
pivot_root
fts(3) / ftw(3)
-----------------------------------------------------------------------
+Other initrd-* commands
+-----------------------
-Allow swap space from the guest to be used. Is it a good idea?
+Such as:
-----------------------------------------------------------------------
+initrd-extract
+initrd-replace
-Need a way to query a binary or library file for its architecture.
-Using objdump or readelf?
-What about non-ELF files (eg. Windows, BSD).
+Simple editing of configuration files
+-------------------------------------
-To do this properly requires some serious logic, eg. to cover Linux
-and Windows we'd need objdump and i686-pc-mingw32-objdump, and more to
-cover a.out, COFF and 64 bit Windows. Therefore this cannot be done
-inside the daemon, and should be done by a separate, external program
-similar to virt-inspector.
+Some easy non-Augeas methods to edit configuration files.
+I'm thinking:
-Probably we should go all the way and have virt-inspector able to
-determine kernel and userspace architectures of guests.
+ replace /etc/file key value
-----------------------------------------------------------------------
+which would look in /etc/file for any instances of
-Other initrd-* commands, such as:
+ key=...
+ key ...
+ key:...
-initrd-extract
-initrd-replace
+and replace them with
+
+ key=value
+ key value
+ key:value
+
+That would solve about 50% of reconfiguration needs, and for the
+rest you'd use Augeas, 'download'+'upload' or 'edit'.
+
+RWMJ: I had a go at implementing this, but it's quite error-prone to
+do this sort of editing inside the C-based daemon code. It's far
+better to do it with Augeas, or else to use an external language like
+Perl.
+
+Quick Perl scripts
+------------------
+
+Currently we can't do Perl "one-liners". ie. The current syntax for
+any short Perl one-liner would be:
+
+ perl -MSys::Guestfs -e '$g = Sys::Guestfs->new(); $g->add_drive ("foo"); $g->launch; $g->mount ("/dev/sda1", "/"); ....'
+
+You can see we're well beyond a single line just getting to the point
+of adding drives and mounting.
+
+First suggestion:
+
+ $h = create ($filename, \"/dev/sda1\" => \"/\");
+
+ $h = create ([$file1, $file2], \"/dev/sda1\" => \"/\");
+
+To mount read-only, add C<ro =E<gt> 1> like this:
+
+ $h = create ($filename, \"/dev/sda1\" => \"/\", ro => 1);
+
+which is equivalent to the following sequence of calls:
+
+ $h = Sys::Guestfs->new ();
+ $h->add_drive_ro ($filename);
+ $h->launch ();
+ $h->mount_ro (\"/dev/sda1\", \"/\");
+
+Command-line form would be:
+
+ perl -MSys::Guestfs=:all -e '$_=create("guest.img", "/dev/sda1" => "/"); $_->cat ("/etc/fstab");'
+
+That's not brief enough for one-liners, so we could have an extra
+autogenerated module which creates a Sys::Guestfs handle singleton
+(the handle is an implicit global variable as in guestfish), eg:
+
+ perl -MSys::Guestfs::One -e 'inspect("guest.img"); cat ("/etc/fstab");'
+
+How would editing files work?
+
+virt-rescue pty
+---------------
+
+See:
+http://search.cpan.org/~rgiersig/IO-Tty-1.08/Pty.pm
+http://www.perlmonks.org/index.pl?node_id=582185
+
+Note that pty requires cooperation inside the C code too (there are
+two sides to a pty, and one has to be handled after the fork).
+
+[I tried to implement this in the new C virt-rescue, but it doesn't
+work. qemu is implementing its own ptys, and they are broken. Need
+to fix qemu.]
+
+Windows-based daemon/appliance
+------------------------------
+
+See discussion on list:
+https://www.redhat.com/archives/libguestfs/2009-November/msg00165.html
+
+qemu locking
+------------
+
+Add -drive file=...,lock=exclusive and -drive file=...,lock=shared
+
+Change libguestfs and libvirt to do the right thing, so that multiple
+instances of qemu cannot stomp on each other.
+
+virt-disk-explore
+-----------------
+
+For multi-level disk images such as live CDs:
+http://rwmj.wordpress.com/2009/07/15/unpack-the-russian-doll-of-a-f11-live-cd/
+
+It's possible with libguestfs to recursively look for anything that
+might be a filesystem, mount-{,loop} it and look in those, revealing
+anything in a disk image.
+
+However this won't work easily for VM disk images in the disk image.
+One would have to download those to the host and launch another
+libguestfs instance.
+
+[Not sure this is such a good idea. See also live CD inspection idea below.]
+
+Map filesystems to disk blocks
+------------------------------
+
+Map files/filesystems/(any other object) to the actual disk
+blocks they occupy.
+
+And vice versa.
+
+Is it even possible?
+
+See also contribs/visualize-alignment/
+
+Integration with host intrusion systems
+---------------------------------------
+
+Perfect way to monitor VMs from outside the VM. Look for file
+hashes, log events, login/logout etc.
+
+http://www.ossec.net/
+http://la-samhna.de/samhain/
+http://sourceforge.net/projects/aide/
+http://osiris.shmoo.com/
+http://sourceforge.net/projects/tripwire/
+
+Fix 'file'
+----------
+
+https://www.redhat.com/archives/libguestfs/2010-June/msg00053.html
+https://www.redhat.com/archives/libguestfs/2010-June/msg00079.html
+
+Freeze/thaw filesystems
+-----------------------
+
+Access to these ioctls:
+http://git.kernel.org/linus/fcccf502540e3d7
+
+Tips for new users in guestfish
+-------------------------------
+
+$ guestfish
+Tip: You need to 'add disk.img' or 'alloc disk.img nn' to make a new image.
+Type 'notips' to disable tips permanently.
+><fs> add mydisk
+Tip: You need to type 'run' before you can see into the disk image.
+><fs> run
+Tip: Use 'list-filesystems' to see what filesystems are available.
+><fs> list-filesystems
+/dev/vda1
+Tip: Use 'mount fs /' to mount a filesystem.
+><fs> mount /dev/vda1 /
+Tip: Use 'll /' to view the filesystem or ...
+><fs> ll /
+
+Could we make guestfish interactive if commands are used without params?
+------------------------------------------------------------------------
+
+><fs> sparse
+[[Prints man page]]
+Image name? disk.img
+Size of image? 10M
+
+Common problems
+---------------
+
+How can we solve these common user problems?
+
+[space for common problems here]
+
+Better support for encrypted devices
+------------------------------------
+
+Currently LUKS support only works if the device contains volume
+groups. If it contains, eg., partitions, you cannot access them.
+We would like to add:
+
+ - Direct access to the /dev/mapper device (eg. if it contains
+ anything apart from VGs).
+
+Display image as PS
+-------------------
+
+Display the structure of an image file as a PS.
+
+Greater use of blkid / libblkid
+-------------------------------
+
+guestfs_zero should use wipefs. See wipefs(8).
+
+There are various useful functions in libblkid for listing partitions,
+devices etc which we are essentially duplicating in the daemon. It
+would make more sense to just use libblkid for this.
+
+There are some places where we call out to the 'blkid' program. This
+might be replaced by direct use of the library (if this is easier).
+
+Visualization
+-------------
+
+Eric Sandeen pointed out the blktrace tool which is a better way of
+capturing traces than using patched qemu (see
+contrib/visualize-alignment). We would still use the same
+visualization tools in conjunction with blktrace traces.
+
+guestfish parsing
+-----------------
+
+At the moment guestfish uses an ad hoc parser which has many
+shortcomings. We should change to using a lex/yacc-based scanner and
+parser (there are better parsers out there, but yacc is sufficient and
+very widely available).
+
+The scanner must deal with the case of parsing a whole command string,
+eg. for a command that the user types in:
+
+ ><fs> add-drive-opts "/tmp/foo" readonly:true
+
+and also with parsing single words from the command line:
+
+ guestfish add-drive-opts /tmp/foo readonly:true
+
+Note the quotes are for scanning and don't indicate types.
+
+We should also allow variables and expressions as part of this new
+parsing code, eg:
+
+ set roots inspect-os
+ set product inspect-get-product-name %{roots[0]}
+
+% is better than $ because of shell escaping and confusion with shell
+variables.
+
+Can we combine this with ability to set and read environment
+variables? Currently guestfish uses many environment variables like
+$EDITOR without any corresponding ability to set them.
+
+ set EDITOR /usr/bin/emacs
+ echo $EDITOR # or %{EDITOR}
+ edit /etc/resolv.conf
+
+live CD inspection for Windows 7
+--------------------------------
+
+Windows 7 install CDs are quite different and pretty impenetrable.
+There are no obvious files to parse.
+
+More ntfs tools
+---------------
+
+ntfsprogs actually has a lot more useful tools than we currently
+use. Interesting ones are:
+
+ntfslabel: display or change filesystem label (we should unify all
+ set*label APIs into a single set_vfs_label which can deal with any
+ filesystem)
+
+ntfsclone: clone, image, restore, rescue NTFS
+
+ntfsinfo: print various information about NTFS volume and files
+
+ntfs streams: extract alternate streams from NTFS files
+
+ntfsck: checker for NTFS filesystems
+
+Undelete files
+--------------
+
+Two useful tools:
+
+ - ext2undelete
+ - ntfsundelete
+
+More mkfs_opts options
+----------------------
+
+Useful options to offer:
+ - Set label.
+ - Set UUID.
+
+Use /proc/self/mountinfo
+------------------------
+
+This file contains lots of interesting information about
+what is mounted and where. eg:
+
+ 16 21 0:3 / /proc rw,relatime - proc /proc rw
+ 17 21 0:16 / /sys rw,relatime - sysfs /sys rw,seclabel
+ 18 23 0:5 / /dev rw,relatime - devtmpfs udev rw,seclabel,size=1906740k,nr_inodes=476685,mode=755
+ 26 21 253:3 / /home rw,relatime - ext4 /dev/mapper/vg-lv_home rw,seclabel,barrier=1,data=ordered
+
+This could be used instead of current hairy code to parse the output
+of the 'mount' command. We could add new APIs to return kernel mount
+options, type of filesystem at a mountpoint etc.
+
+guestfish drive letters
+-----------------------
+
+There should be an option to mount all Windows drives as separate
+paths, like C: => /c/, D: => /d/ etc.
+
+More inspection features
+------------------------
+
+- last shutdown time
+- DHCP address
+- last time the software was updated
+- last user who logged in
+- lastlog, last, who
+
+Get the guest icon
+------------------
+
+- For Linux guests, use /etc/favicon.png if available, else get it in
+ a distro-specific manner.
+- For Windows guests, parse it out of c:\windows\explorer.exe
+
+Integrate event log parsing more closely
+----------------------------------------
+
+https://rwmj.wordpress.com/2011/04/17/decoding-the-windows-event-log-using-guestfish/
+
+We should at least make sure the tools are packaged up for Fedora. We
+could document formally how to do event log parsing. Also the above
+only works for Windows Vista and later (since the log format changed),
+so we could extend this to work for earlier versions.
+
+Integrate virt-inspector with CMDBs
+-----------------------------------
+
+Either integrate virt-inspector with Configuration Management
+Databases (CMDBs) or at least check that virt-inspector produces the
+right range of data so that integration would be possible. The
+standards for CMDBs come from the DMTF, see eg:
+
+http://dmtf.org/news/pr/2009/7/dmtf-releases-cmdbf-standard-federating-configuration-management-data
git.annexia.org / libguestfs.git / blobdiff