From 35dbedb1b18157b2329e0e55d0b5355f26431814 Mon Sep 17 00:00:00 2001 From: "Richard W.M. Jones" Date: Sat, 13 Nov 2010 11:32:47 +0000 Subject: [PATCH] docs: Rearrange guestfs(3) sections. This rearranges the sections into a more logical order: - synopsis and introduction - API-related overview sections - (security will go here, see next commit) - API in detail - architecture and other internals - usual end sections --- src/guestfs.pod | 508 ++++++++++++++++++++++++++++---------------------------- 1 file changed, 254 insertions(+), 254 deletions(-) diff --git a/src/guestfs.pod b/src/guestfs.pod index 9717543..9fcd6ec 100644 --- a/src/guestfs.pod +++ b/src/guestfs.pod @@ -841,6 +841,171 @@ libguestfs might end up being written out to the swap partition. If this is a concern, scrub the swap partition or don't use libguestfs on encrypted devices. +=head2 MULTIPLE HANDLES AND MULTIPLE THREADS + +All high-level libguestfs actions are synchronous. If you want +to use libguestfs asynchronously then you must create a thread. + +Only use the handle from a single thread. Either use the handle +exclusively from one thread, or provide your own mutex so that two +threads cannot issue calls on the same handle at the same time. + +See the graphical program guestfs-browser for one possible +architecture for multithreaded programs using libvirt and libguestfs. + +=head2 PATH + +Libguestfs needs a kernel and initrd.img, which it finds by looking +along an internal path. + +By default it looks for these in the directory C<$libdir/guestfs> +(eg. C or C). + +Use L or set the environment variable +L to change the directories that libguestfs will +search in. The value is a colon-separated list of paths. The current +directory is I searched unless the path contains an empty element +or C<.>. For example C would +search the current directory and then C. + +=head2 QEMU WRAPPERS + +If you want to compile your own qemu, run qemu from a non-standard +location, or pass extra arguments to qemu, then you can write a +shell-script wrapper around qemu. + +There is one important rule to remember: you I> as +the last command in the shell script (so that qemu replaces the shell +and becomes the direct child of the libguestfs-using program). If you +don't do this, then the qemu process won't be cleaned up correctly. + +Here is an example of a wrapper, where I have built my own copy of +qemu from source: + + #!/bin/sh - + qemudir=/home/rjones/d/qemu + exec $qemudir/x86_64-softmmu/qemu-system-x86_64 -L $qemudir/pc-bios "$@" + +Save this script as C (or wherever), C, +and then use it by setting the LIBGUESTFS_QEMU environment variable. +For example: + + LIBGUESTFS_QEMU=/tmp/qemu.wrapper guestfish + +Note that libguestfs also calls qemu with the -help and -version +options in order to determine features. + +=head2 ABI GUARANTEE + +We guarantee the libguestfs ABI (binary interface), for public, +high-level actions as outlined in this section. Although we will +deprecate some actions, for example if they get replaced by newer +calls, we will keep the old actions forever. This allows you the +developer to program in confidence against the libguestfs API. + +=head2 BLOCK DEVICE NAMING + +In the kernel there is now quite a profusion of schemata for naming +block devices (in this context, by I I mean a physical +or virtual hard drive). The original Linux IDE driver used names +starting with C. SCSI devices have historically used a +different naming scheme, C. When the Linux kernel I +driver became a popular replacement for the old IDE driver +(particularly for SATA devices) those devices also used the +C scheme. Additionally we now have virtual machines with +paravirtualized drivers. This has created several different naming +systems, such as C for virtio disks and C for Xen +PV disks. + +As discussed above, libguestfs uses a qemu appliance running an +embedded Linux kernel to access block devices. We can run a variety +of appliances based on a variety of Linux kernels. + +This causes a problem for libguestfs because many API calls use device +or partition names. Working scripts and the recipe (example) scripts +that we make available over the internet could fail if the naming +scheme changes. + +Therefore libguestfs defines C as the I. Internally C names are translated, if necessary, +to other names as required. For example, under RHEL 5 which uses the +C scheme, any device parameter C is translated to +C transparently. + +Note that this I applies to parameters. The +L, L and similar calls +return the true names of the devices and partitions as known to the +appliance. + +=head3 ALGORITHM FOR BLOCK DEVICE NAME TRANSLATION + +Usually this translation is transparent. However in some (very rare) +cases you may need to know the exact algorithm. Such cases include +where you use L to add a mixture of virtio and IDE +devices to the qemu-based appliance, so have a mixture of C +and C devices. + +The algorithm is applied only to I which are known to be +either device or partition names. Return values from functions such +as L are never changed. + +=over 4 + +=item * + +Is the string a parameter which is a device or partition name? + +=item * + +Does the string begin with C? + +=item * + +Does the named device exist? If so, we use that device. +However if I then we continue with this algorithm. + +=item * + +Replace initial C string with C. + +For example, change C to C. + +If that named device exists, use it. If not, continue. + +=item * + +Replace initial C string with C. + +If that named device exists, use it. If not, return an error. + +=back + +=head3 PORTABILITY CONCERNS WITH BLOCK DEVICE NAMING + +Although the standard naming scheme and automatic translation is +useful for simple programs and guestfish scripts, for larger programs +it is best not to rely on this mechanism. + +Where possible for maximum future portability programs using +libguestfs should use these future-proof techniques: + +=over 4 + +=item * + +Use L or L to list +actual device names, and then use those names directly. + +Since those device names exist by definition, they will never be +translated. + +=item * + +Use higher level ways to identify filesystems, such as LVM names, +UUIDs and filesystem labels. + +=back + =head1 CONNECTION MANAGEMENT =head2 guestfs_h * @@ -1027,30 +1192,7 @@ situations. This returns the current out of memory handler. -=head1 PATH - -Libguestfs needs a kernel and initrd.img, which it finds by looking -along an internal path. - -By default it looks for these in the directory C<$libdir/guestfs> -(eg. C or C). - -Use L or set the environment variable -L to change the directories that libguestfs will -search in. The value is a colon-separated list of paths. The current -directory is I searched unless the path contains an empty element -or C<.>. For example C would -search the current directory and then C. - -=head1 HIGH-LEVEL API ACTIONS - -=head2 ABI GUARANTEE - -We guarantee the libguestfs ABI (binary interface), for public, -high-level actions as outlined in this section. Although we will -deprecate some actions, for example if they get replaced by newer -calls, we will keep the old actions forever. This allows you the -developer to program in confidence against the libguestfs API. +=head1 API CALLS @ACTIONS@ @@ -1245,112 +1387,6 @@ language-specific documentation for more details on that. For guestfish, see L. -=begin html - - - - -=end html - -=head1 ARCHITECTURE - -Internally, libguestfs is implemented by running an appliance (a -special type of small virtual machine) using L. Qemu runs as -a child process of the main program. - - ___________________ - / \ - | main program | - | | - | | child process / appliance - | | __________________________ - | | / qemu \ - +-------------------+ RPC | +-----------------+ | - | libguestfs <--------------------> guestfsd | | - | | | +-----------------+ | - \___________________/ | | Linux kernel | | - | +--^--------------+ | - \_________|________________/ - | - _______v______ - / \ - | Device or | - | disk image | - \______________/ - -The library, linked to the main program, creates the child process and -hence the appliance in the L function. - -Inside the appliance is a Linux kernel and a complete stack of -userspace tools (such as LVM and ext2 programs) and a small -controlling daemon called L. The library talks to -L using remote procedure calls (RPC). There is a mostly -one-to-one correspondence between libguestfs API calls and RPC calls -to the daemon. Lastly the disk image(s) are attached to the qemu -process which translates device access by the appliance's Linux kernel -into accesses to the image. - -A common misunderstanding is that the appliance "is" the virtual -machine. Although the disk image you are attached to might also be -used by some virtual machine, libguestfs doesn't know or care about -this. (But you will care if both libguestfs's qemu process and your -virtual machine are trying to update the disk image at the same time, -since these usually results in massive disk corruption). - -=head1 STATE MACHINE - -libguestfs uses a state machine to model the child process: - - | - guestfs_create - | - | - ____V_____ - / \ - | CONFIG | - \__________/ - ^ ^ ^ \ - / | \ \ guestfs_launch - / | _\__V______ - / | / \ - / | | LAUNCHING | - / | \___________/ - / | / - / | guestfs_launch - / | / - ______ / __|____V - / \ ------> / \ - | BUSY | | READY | - \______/ <------ \________/ - -The normal transitions are (1) CONFIG (when the handle is created, but -there is no child process), (2) LAUNCHING (when the child process is -booting up), (3) alternating between READY and BUSY as commands are -issued to, and carried out by, the child process. - -The guest may be killed by L, or may die -asynchronously at any time (eg. due to some internal error), and that -causes the state to transition back to CONFIG. - -Configuration commands for qemu such as L can only -be issued when in the CONFIG state. - -The API offers one call that goes from CONFIG through LAUNCHING to -READY. L blocks until the child process is READY to -accept commands (or until some failure or timeout). -L internally moves the state from CONFIG to LAUNCHING -while it is running. - -API actions such as L can only be issued when in the -READY state. These API calls block waiting for the command to be -carried out (ie. the state to transition to BUSY and then back to -READY). There are no non-blocking versions, and no way to issue more -than one command per handle at the same time. - -Finally, the child process sends asynchronous messages back to the -main program, such as kernel log messages. You can register a -callback to receive these messages. - =head2 SETTING CALLBACKS TO HANDLE EVENTS The child process generates events in some situations. Current events @@ -1509,108 +1545,111 @@ and note that only one callback can be registered for a handle). The private data area is implemented using a hash table, and should be reasonably efficient for moderate numbers of keys. -=head1 BLOCK DEVICE NAMING - -In the kernel there is now quite a profusion of schemata for naming -block devices (in this context, by I I mean a physical -or virtual hard drive). The original Linux IDE driver used names -starting with C. SCSI devices have historically used a -different naming scheme, C. When the Linux kernel I -driver became a popular replacement for the old IDE driver -(particularly for SATA devices) those devices also used the -C scheme. Additionally we now have virtual machines with -paravirtualized drivers. This has created several different naming -systems, such as C for virtio disks and C for Xen -PV disks. - -As discussed above, libguestfs uses a qemu appliance running an -embedded Linux kernel to access block devices. We can run a variety -of appliances based on a variety of Linux kernels. - -This causes a problem for libguestfs because many API calls use device -or partition names. Working scripts and the recipe (example) scripts -that we make available over the internet could fail if the naming -scheme changes. - -Therefore libguestfs defines C as the I. Internally C names are translated, if necessary, -to other names as required. For example, under RHEL 5 which uses the -C scheme, any device parameter C is translated to -C transparently. - -Note that this I applies to parameters. The -L, L and similar calls -return the true names of the devices and partitions as known to the -appliance. - -=head2 ALGORITHM FOR BLOCK DEVICE NAME TRANSLATION - -Usually this translation is transparent. However in some (very rare) -cases you may need to know the exact algorithm. Such cases include -where you use L to add a mixture of virtio and IDE -devices to the qemu-based appliance, so have a mixture of C -and C devices. - -The algorithm is applied only to I which are known to be -either device or partition names. Return values from functions such -as L are never changed. - -=over 4 - -=item * - -Is the string a parameter which is a device or partition name? - -=item * - -Does the string begin with C? - -=item * - -Does the named device exist? If so, we use that device. -However if I then we continue with this algorithm. - -=item * +=begin html -Replace initial C string with C. + + -For example, change C to C. +=end html -If that named device exists, use it. If not, continue. +=head1 ARCHITECTURE -=item * +Internally, libguestfs is implemented by running an appliance (a +special type of small virtual machine) using L. Qemu runs as +a child process of the main program. -Replace initial C string with C. + ___________________ + / \ + | main program | + | | + | | child process / appliance + | | __________________________ + | | / qemu \ + +-------------------+ RPC | +-----------------+ | + | libguestfs <--------------------> guestfsd | | + | | | +-----------------+ | + \___________________/ | | Linux kernel | | + | +--^--------------+ | + \_________|________________/ + | + _______v______ + / \ + | Device or | + | disk image | + \______________/ -If that named device exists, use it. If not, return an error. +The library, linked to the main program, creates the child process and +hence the appliance in the L function. -=back +Inside the appliance is a Linux kernel and a complete stack of +userspace tools (such as LVM and ext2 programs) and a small +controlling daemon called L. The library talks to +L using remote procedure calls (RPC). There is a mostly +one-to-one correspondence between libguestfs API calls and RPC calls +to the daemon. Lastly the disk image(s) are attached to the qemu +process which translates device access by the appliance's Linux kernel +into accesses to the image. -=head2 PORTABILITY CONCERNS +A common misunderstanding is that the appliance "is" the virtual +machine. Although the disk image you are attached to might also be +used by some virtual machine, libguestfs doesn't know or care about +this. (But you will care if both libguestfs's qemu process and your +virtual machine are trying to update the disk image at the same time, +since these usually results in massive disk corruption). -Although the standard naming scheme and automatic translation is -useful for simple programs and guestfish scripts, for larger programs -it is best not to rely on this mechanism. +=head1 STATE MACHINE -Where possible for maximum future portability programs using -libguestfs should use these future-proof techniques: +libguestfs uses a state machine to model the child process: -=over 4 + | + guestfs_create + | + | + ____V_____ + / \ + | CONFIG | + \__________/ + ^ ^ ^ \ + / | \ \ guestfs_launch + / | _\__V______ + / | / \ + / | | LAUNCHING | + / | \___________/ + / | / + / | guestfs_launch + / | / + ______ / __|____V + / \ ------> / \ + | BUSY | | READY | + \______/ <------ \________/ -=item * +The normal transitions are (1) CONFIG (when the handle is created, but +there is no child process), (2) LAUNCHING (when the child process is +booting up), (3) alternating between READY and BUSY as commands are +issued to, and carried out by, the child process. -Use L or L to list -actual device names, and then use those names directly. +The guest may be killed by L, or may die +asynchronously at any time (eg. due to some internal error), and that +causes the state to transition back to CONFIG. -Since those device names exist by definition, they will never be -translated. +Configuration commands for qemu such as L can only +be issued when in the CONFIG state. -=item * +The API offers one call that goes from CONFIG through LAUNCHING to +READY. L blocks until the child process is READY to +accept commands (or until some failure or timeout). +L internally moves the state from CONFIG to LAUNCHING +while it is running. -Use higher level ways to identify filesystems, such as LVM names, -UUIDs and filesystem labels. +API actions such as L can only be issued when in the +READY state. These API calls block waiting for the command to be +carried out (ie. the state to transition to BUSY and then back to +READY). There are no non-blocking versions, and no way to issue more +than one command per handle at the same time. -=back +Finally, the child process sends asynchronous messages back to the +main program, such as kernel log messages. You can register a +callback to receive these messages. =head1 INTERNALS @@ -1766,45 +1805,6 @@ The daemon self-limits the frequency of progress messages it sends (see C). Not all calls generate progress messages. -=head1 MULTIPLE HANDLES AND MULTIPLE THREADS - -All high-level libguestfs actions are synchronous. If you want -to use libguestfs asynchronously then you must create a thread. - -Only use the handle from a single thread. Either use the handle -exclusively from one thread, or provide your own mutex so that two -threads cannot issue calls on the same handle at the same time. - -See the graphical program guestfs-browser for one possible -architecture for multithreaded programs using libvirt and libguestfs. - -=head1 QEMU WRAPPERS - -If you want to compile your own qemu, run qemu from a non-standard -location, or pass extra arguments to qemu, then you can write a -shell-script wrapper around qemu. - -There is one important rule to remember: you I> as -the last command in the shell script (so that qemu replaces the shell -and becomes the direct child of the libguestfs-using program). If you -don't do this, then the qemu process won't be cleaned up correctly. - -Here is an example of a wrapper, where I have built my own copy of -qemu from source: - - #!/bin/sh - - qemudir=/home/rjones/d/qemu - exec $qemudir/x86_64-softmmu/qemu-system-x86_64 -L $qemudir/pc-bios "$@" - -Save this script as C (or wherever), C, -and then use it by setting the LIBGUESTFS_QEMU environment variable. -For example: - - LIBGUESTFS_QEMU=/tmp/qemu.wrapper guestfish - -Note that libguestfs also calls qemu with the -help and -version -options in order to determine features. - =head1 LIBGUESTFS VERSION NUMBERS Since April 2010, libguestfs has started to make separate development -- 1.8.3.1