2 # Copyright (C) 2015 Richard W.M. Jones <rjones@redhat.com>
3 # Copyright (C) 2015 Red Hat Inc.
5 # This program is free software; you can redistribute it and/or modify
6 # it under the terms of the GNU General Public License as published by
7 # the Free Software Foundation; either version 2 of the License, or
8 # (at your option) any later version.
10 # This program is distributed in the hope that it will be useful,
11 # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 # GNU General Public License for more details.
15 # You should have received a copy of the GNU General Public License
16 # along with this program; if not, write to the Free Software
17 # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
25 use File::Temp qw(tempdir);
26 use POSIX qw(_exit setgid setuid strftime);
33 import-to-ovirt.pl - Import virtual machine disk image to RHEV or oVirt
37 sudo ./import-to-ovirt.pl disk.img server:/esd
39 sudo ./import-to-ovirt.pl disk.img /esd_mountpoint
41 =head1 IMPORTANT NOTES
43 In the latest oVirt/RHEV/RHV there is a GUI option to import disks.
44 You B<do not need to use this script> if you are using a sufficiently
47 This tool should B<only> be used if the guest can already run on KVM.
49 B<If you need to convert the guest from some foreign hypervisor, like VMware, Xen or Hyper-V, you should use L<virt-v2v(1)> instead.>
53 Import a KVM guest to the Export Storage Domain of your RHEV or oVirt
54 system. The NFS mount of the Export Storage Domain is C<server:/esd>.
56 sudo ./import-to-ovirt.pl disk.img server:/esd
58 Import a KVM guest to an already-mounted Export Storage Domain:
60 sudo ./import-to-ovirt.pl disk.img /esd_mountpoint
62 If the single guest has multiple disks, use:
64 sudo ./import-to-ovirt.pl disk1.img [disk2.img [...]] server:/esd
66 If you are importing multiple guests then you must import each one
67 separately. Do not use multiple disk parameters if each disk comes
68 from a different guest.
72 This is a command line script for importing a KVM virtual machine to
73 RHEV or oVirt. The script assumes that the guest can already run on
74 KVM, ie. that it was previously running on KVM and has the required
75 drivers. If the guest comes from a foreign hypervisor like VMware,
76 Xen or Hyper-V, use L<virt-v2v(1)> instead.
78 This script only imports the guest into the oVirt "Export Storage
79 Domain" (ESD). After the import is complete, you must then go to the
80 oVirt user interface, go to the C<Storage> tab, select the right ESD,
81 and use C<VM Import> to take the guest from the ESD to the data
82 domain. This process is outside the scope of this script, but could
83 be automated using the oVirt API.
89 ./import-to-ovirt.pl [list of disks] server:/esd
91 If you are unclear about the C<server:/esd> parameter, go to the oVirt
92 Storage tab, select the C<Domain Type> C<Export> and look in the
93 C<General> tab under C<Path>.
95 If the ESD is already mounted on your machine (or if you are using a
96 non-NFS ESD), then you can supply a direct path to the mountpoint
99 ./import-to-ovirt.pl [list of disks] /esd_mountpoint
101 The list of disks should all belong to a single guest (most guests
102 will only have a single disk). If you want to import multiple guests,
103 you must run the script multiple times.
105 Importing from OVA etc is not supported. Try C<ovirt-image-uploader>
106 (if the OVA was exported from oVirt), or L<virt-v2v(1)> (if the OVA
107 was exported from VMware).
111 You probably need to run this script as root, because it has to create
112 files on the ESD as a special C<vdsm> user (UID:GID C<36:36>).
114 It may also be possible to run the script as the vdsm user. But if
115 you run it as some non-root, non-vdsm user, then oVirt won't be able
116 to read the data from the ESD and will give an error.
118 NFS "root squash" should be turned off on the NFS server, since it
119 stops us from creating files as the vdsm user. Also NFSv4 may not
120 work unless you have set up idmap correctly (good luck!)
122 =head2 Network card and disk model
124 (See also L</TO DO> below)
126 Currently this script doesn't add a network card to the guest. You
127 will need to add one yourself in the C<VM Import> tab when importing
130 Similarly, the script always adds the disks as virtio-blk devices. If
131 the guest is expecting IDE, SCSI or virtio-scsi, you will need to
132 change the disk type when importing the guest.
144 Display brief help and exit.
152 Display the manual page and exit.
156 my $memory_mb = 1024;
160 Set the memory size I<in megabytes>. The default is 1024.
168 Set the guest name. If not present, a name is made up based on
169 the filename of the first disk.
177 Set the number of virtual CPUs. The default is 1.
181 my $vmtype = "Desktop";
183 =item B<--vmtype> Desktop
185 =item B<--vmtype> Server
187 Set the VmType field in the OVF. It must be C<Desktop> or
188 C<Server>. The default is C<Desktop>.
198 GetOptions ("help|?" => \$help,
200 "memory=i" => \$memory_mb,
202 "vcpus=i" => \$vcpus,
203 "vmtype=s" => \$vmtype,
205 or die "$0: unknown command line option\n";
207 pod2usage (1) if $help;
208 pod2usage (-exitval => 0, -verbose => 2) if $man;
210 # Get the parameters.
212 die "Use '$0 --man' to display the manual.\n"
215 my @disks = @ARGV[0 .. $#ARGV-1];
216 my $output = $ARGV[$#ARGV];
218 if (!defined $name) {
221 $name =~ s{\.[^.]+}{};
224 if ($vmtype =~ /^Desktop$/i) {
226 } elsif ($vmtype =~ /^Server$/i) {
229 die "$0: --vmtype parameter must be 'Desktop' or 'Server'\n"
232 # Does qemu-img generally work OK?
233 system ("qemu-img create -f qcow2 .test.qcow2 10M >/dev/null") == 0
234 or die "qemu-img command not installed or not working\n";
236 # Does this version of qemu-img support compat=0.10? RHEL 6
237 # did NOT support it.
238 my $qemu_img_supports_compat = 0;
239 system ("qemu-img create -f qcow2 -o compat=0.10 .test.qcow2 10M >/dev/null") == 0
240 and $qemu_img_supports_compat = 1;
241 unlink ".test.qcow2";
243 # Open the guest in libguestfs so we can inspect it.
244 my $g = Sys::Guestfs->new ();
245 $g->set_program ("virt-import-to-ovirt");
246 $g->add_drive_opts ($_, readonly => 1) foreach (@disks);
248 my @roots = $g->inspect_os ();
250 die "$0: no operating system was found on the disk\n"
253 die "$0: either this is a multi-OS disk, or you passed multiple unrelated guest disks on the command line\n"
255 my $root = $roots[0];
257 # Save the inspection data.
258 my $type = $g->inspect_get_type ($root);
259 my $distro = $g->inspect_get_distro ($root);
260 my $arch = $g->inspect_get_arch ($root);
261 my $major_version = $g->inspect_get_major_version ($root);
262 my $minor_version = $g->inspect_get_major_version ($root);
263 my $product_name = $g->inspect_get_product_name ($root);
264 my $product_variant = $g->inspect_get_product_variant ($root);
266 # Get the virtual size of each disk.
269 push @virtual_sizes, $g->disk_virtual_size ($_);
274 # Map inspection data to RHEV ostype.
276 if ($type eq "linux" && $distro eq "rhel" && $major_version <= 6) {
277 if ($arch eq "x86_64") {
278 $ostype = "RHEL${major_version}x64"
280 $ostype = "RHEL$major_version"
283 elsif ($type eq "linux" && $distro eq "rhel") {
284 if ($arch eq "x86_64") {
285 $ostype = "rhel_${major_version}x64"
287 $ostype = "rhel_$major_version"
290 elsif ($type eq "linux") {
291 $ostype = "OtherLinux"
293 elsif ($type eq "windows" && $major_version == 5 && $minor_version == 1) {
294 $ostype = "WindowsXP"
296 elsif ($type eq "windows" && $major_version == 5 && $minor_version == 2) {
297 if ($product_name =~ /XP/) {
298 $ostype = "WindowsXP"
299 } elsif ($arch eq "x86_64") {
300 $ostype = "Windows2003x64"
302 $ostype = "Windows2003"
305 elsif ($type eq "windows" && $major_version == 6 && $minor_version == 0) {
306 if ($arch eq "x86_64") {
307 $ostype = "Windows2008x64"
309 $ostype = "Windows2008"
312 elsif ($type eq "windows" && $major_version == 6 && $minor_version == 1) {
313 if ($product_variant eq "Client") {
314 if ($arch eq "x86_64") {
315 $ostype = "Windows7x64"
320 $ostype = "Windows2008R2x64"
323 elsif ($type eq "windows" && $major_version == 6 && $minor_version == 2) {
324 if ($product_variant eq "Client") {
325 if ($arch eq "x86_64") {
326 $ostype = "windows_8x64"
328 $ostype = "windows_8"
331 $ostype = "windows_2012x64"
334 elsif ($type eq "windows" && $major_version == 6 && $minor_version == 3) {
335 $ostype = "windows_2012R2x64"
338 $ostype = "Unassigned"
341 # Mount the ESD if needed (or just check it exists).
344 $mountpoint = $output;
345 } elsif ($output =~ m{^.*:.*$}) {
347 $umount = $mountpoint = tempdir (CLEANUP => 1);
348 system ("mount", "-t", "nfs", $output, $mountpoint) == 0
349 or die "$0: mount $output failed: $?\n";
350 END { system ("umount", $umount) if defined $umount }
352 die "$0: ESD $output is not a directory or an NFS mountpoint\n"
355 # Check the ESD looks like an ESD.
356 my @entries = <$mountpoint/*>;
358 grep { m{/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$}i }
361 die "$0: does $output really point to an oVirt Export Storage Domain?\n"
364 die "$0: multiple GUIDs found in oVirt Export Storage Domain\n"
366 my $esd_uuid_dir = $entries[0];
367 my $esd_uuid = $esd_uuid_dir;
368 $esd_uuid =~ s{.*/}{};
370 if (! -d $esd_uuid_dir ||
371 ! -d "$esd_uuid_dir/images" ||
372 ! -d "$esd_uuid_dir/master" ||
373 ! -d "$esd_uuid_dir/master/vms") {
374 die "$0: $output doesn't look like an Export Storage Domain\n"
378 print "Importing $product_name to $output...\n";
380 # A helper function that forks and runs some code / a command as
381 # an alternate UID:GID.
387 die "fork: $!" unless defined $pid;
401 waitpid ($pid, 0) or die "waitpid: $!";
403 die "$0: run_as_vdsm: child process failed (status $?)\n";
410 local $_ = `uuidgen -r`;
412 die unless length $_ >= 30; # Sanity check.
416 # Generate some random UUIDs.
417 my $vm_uuid = uuidgen ();
420 push @image_uuids, uuidgen ();
424 push @vol_uuids, uuidgen ();
427 # Make sure the output is deleted on unsuccessful exit. We set
428 # $delete_output_on_exit to false at the end of the script.
429 my $delete_output_on_exit = 1;
431 if ($delete_output_on_exit) {
432 # Can't use run_as_vdsm in an END{} block.
433 foreach (@image_uuids) {
434 system ("rm", "-rf", "$esd_uuid_dir/images/$_");
436 system ("rm", "-rf", "$esd_uuid_dir/master/vms/$vm_uuid");
440 # Copy and convert the disk images.
443 my $iso_time = strftime ("%Y/%m/%d %H:%M:%S", gmtime ());
444 my $imported_by = "Imported by import-to-ovirt.pl";
447 for ($i = 0; $i < @disks; ++$i) {
448 my $input_file = $disks[$i];
449 my $image_uuid = $image_uuids[$i];
451 my $path = "$esd_uuid_dir/images/$image_uuid";
452 mkdir ($path, 0755) or die "mkdir: $path: $!";
454 my $output_file = "$esd_uuid_dir/images/$image_uuid/".$vol_uuids[$i];
456 open (my $fh, ">", $output_file) or die "open: $output_file: $!";
457 # Well done NFS root_squash, you make the world less secure.
458 chmod (0666, $output_file) or die "chmod: $output_file: $!";
460 print "Copying $input_file ...\n";
461 my @compat_option = ();
462 if ($qemu_img_supports_compat) {
463 @compat_option = ("-o", "compat=0.10") # for RHEL 6-based ovirt nodes
465 system ("qemu-img", "convert", "-p",
470 or die "qemu-img: $input_file: failed (status $?)";
471 push @real_sizes, -s $output_file;
473 my $size_in_sectors = $virtual_sizes[$i] / 512;
475 # Create .meta files per disk.
483 PUUID=00000000-0000-0000-0000-000000000000
486 SIZE=$size_in_sectors
489 DESCRIPTION=$imported_by
491 my $meta_file = $output_file . ".meta";
493 open (my $fh, ">", $meta_file) or die "open: $meta_file: $!";
499 print "Creating OVF metadata ...\n";
501 my $rasd_ns = "http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData";
502 my $vssd_ns = "http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData";
503 my $xsi_ns = "http://www.w3.org/2001/XMLSchema-instance";
504 my $ovf_ns = "http://schemas.dmtf.org/ovf/envelope/1/";
511 my @forced_ns_decls = keys %prefix_map;
513 my $w = XML::Writer->new (
516 PREFIX_MAP => \%prefix_map,
517 FORCED_NS_DECLS => \@forced_ns_decls,
522 $w->startTag ([$ovf_ns, "Envelope"],
523 [$ovf_ns, "version"] => "0.9");
524 $w->comment ($imported_by);
526 $w->startTag ("References");
528 for ($i = 0; $i < @disks; ++$i)
530 my $href = $image_uuids[$i] . "/" . $vol_uuids[$i];
531 $w->startTag ("File",
532 [$ovf_ns, "href"] => $href,
533 [$ovf_ns, "id"] => $vol_uuids[$i],
534 [$ovf_ns, "size"] => $virtual_sizes[$i],
535 [$ovf_ns, "description"] => $imported_by);
541 $w->startTag ("Section",
542 [$xsi_ns, "type"] => "ovf:NetworkSection_Type");
543 $w->startTag ("Info");
544 $w->characters ("List of networks");
548 $w->startTag ("Section",
549 [$xsi_ns, "type"] => "ovf:DiskSection_Type");
550 $w->startTag ("Info");
551 $w->characters ("List of Virtual Disks");
554 for ($i = 0; $i < @disks; ++$i)
556 my $virtual_size_in_gb = $virtual_sizes[$i];
557 $virtual_size_in_gb /= 1024;
558 $virtual_size_in_gb /= 1024;
559 $virtual_size_in_gb /= 1024;
560 my $real_size_in_gb = $real_sizes[$i];
561 $real_size_in_gb /= 1024;
562 $real_size_in_gb /= 1024;
563 $real_size_in_gb /= 1024;
564 my $href = $image_uuids[$i] . "/" . $vol_uuids[$i];
568 $boot_drive = "True";
570 $boot_drive = "False";
573 $w->startTag ("Disk",
574 [$ovf_ns, "diskId" ] => $vol_uuids[$i],
575 [$ovf_ns, "actual_size"] =>
576 sprintf ("%.0f", $real_size_in_gb),
578 sprintf ("%.0f", $virtual_size_in_gb),
579 [$ovf_ns, "fileRef"] => $href,
580 [$ovf_ns, "parentRef"] => "",
581 [$ovf_ns, "vm_snapshot_id"] => uuidgen (),
582 [$ovf_ns, "volume-format"] => "COW",
583 [$ovf_ns, "volume-type"] => "Sparse",
584 [$ovf_ns, "format"] => "http://en.wikipedia.org/wiki/Byte",
585 [$ovf_ns, "disk-interface"] => "VirtIO",
586 [$ovf_ns, "disk-type"] => "System",
587 [$ovf_ns, "boot"] => $boot_drive);
593 $w->startTag ("Content",
594 [$ovf_ns, "id"] => "out",
595 [$xsi_ns, "type"] => "ovf:VirtualSystem_Type");
596 $w->startTag ("Name");
597 $w->characters ($name);
599 $w->startTag ("TemplateId");
600 $w->characters ("00000000-0000-0000-0000-000000000000");
602 $w->startTag ("TemplateName");
603 $w->characters ("Blank");
605 $w->startTag ("Description");
606 $w->characters ($imported_by);
608 $w->startTag ("Domain");
610 $w->startTag ("CreationDate");
611 $w->characters ($iso_time);
613 $w->startTag ("IsInitilized"); # sic
614 $w->characters ("True");
616 $w->startTag ("IsAutoSuspend");
617 $w->characters ("False");
619 $w->startTag ("TimeZone");
621 $w->startTag ("IsStateless");
622 $w->characters ("False");
624 $w->startTag ("Origin");
625 $w->characters ("0");
627 $w->startTag ("VmType");
628 $w->characters ($vmtype);
630 $w->startTag ("DefaultDisplayType");
631 $w->characters ("1"); # qxl
634 $w->startTag ("Section",
635 [$ovf_ns, "id"] => $vm_uuid,
636 [$ovf_ns, "required"] => "false",
637 [$xsi_ns, "type"] => "ovf:OperatingSystemSection_Type");
638 $w->startTag ("Info");
639 $w->characters ($product_name);
641 $w->startTag ("Description");
642 $w->characters ($ostype);
646 $w->startTag ("Section",
647 [$xsi_ns, "type"] => "ovf:VirtualHardwareSection_Type");
648 $w->startTag ("Info");
649 $w->characters (sprintf ("%d CPU, %d Memory", $vcpus, $memory_mb));
652 $w->startTag ("Item");
653 $w->startTag ([$rasd_ns, "Caption"]);
654 $w->characters (sprintf ("%d virtual cpu", $vcpus));
656 $w->startTag ([$rasd_ns, "Description"]);
657 $w->characters ("Number of virtual CPU");
659 $w->startTag ([$rasd_ns, "InstanceId"]);
660 $w->characters ("1");
662 $w->startTag ([$rasd_ns, "ResourceType"]);
663 $w->characters ("3");
665 $w->startTag ([$rasd_ns, "num_of_sockets"]);
666 $w->characters ($vcpus);
668 $w->startTag ([$rasd_ns, "cpu_per_socket"]);
673 $w->startTag ("Item");
674 $w->startTag ([$rasd_ns, "Caption"]);
675 $w->characters (sprintf ("%d MB of memory", $memory_mb));
677 $w->startTag ([$rasd_ns, "Description"]);
678 $w->characters ("Memory Size");
680 $w->startTag ([$rasd_ns, "InstanceId"]);
681 $w->characters ("2");
683 $w->startTag ([$rasd_ns, "ResourceType"]);
684 $w->characters ("4");
686 $w->startTag ([$rasd_ns, "AllocationUnits"]);
687 $w->characters ("MegaBytes");
689 $w->startTag ([$rasd_ns, "VirtualQuantity"]);
690 $w->characters ($memory_mb);
694 $w->startTag ("Item");
695 $w->startTag ([$rasd_ns, "Caption"]);
696 $w->characters ("USB Controller");
698 $w->startTag ([$rasd_ns, "InstanceId"]);
699 $w->characters ("3");
701 $w->startTag ([$rasd_ns, "ResourceType"]);
702 $w->characters ("23");
704 $w->startTag ([$rasd_ns, "UsbPolicy"]);
705 $w->characters ("Disabled");
709 $w->startTag ("Item");
710 $w->startTag ([$rasd_ns, "Caption"]);
711 $w->characters ("Graphical Controller");
713 $w->startTag ([$rasd_ns, "InstanceId"]);
714 $w->characters (uuidgen ());
716 $w->startTag ([$rasd_ns, "ResourceType"]);
717 $w->characters ("20");
719 $w->startTag ("Type");
720 $w->characters ("video");
722 $w->startTag ([$rasd_ns, "VirtualQuantity"]);
723 $w->characters ("1");
725 $w->startTag ([$rasd_ns, "Device"]);
726 $w->characters ("qxl");
730 for ($i = 0; $i < @disks; ++$i)
732 my $href = $image_uuids[$i] . "/" . $vol_uuids[$i];
734 $w->startTag ("Item");
736 $w->startTag ([$rasd_ns, "Caption"]);
737 $w->characters ("Drive " . ($i+1));
739 $w->startTag ([$rasd_ns, "InstanceId"]);
740 $w->characters ($vol_uuids[$i]);
742 $w->startTag ([$rasd_ns, "ResourceType"]);
743 $w->characters ("17");
745 $w->startTag ("Type");
746 $w->characters ("disk");
748 $w->startTag ([$rasd_ns, "HostResource"]);
749 $w->characters ($href);
751 $w->startTag ([$rasd_ns, "Parent"]);
752 $w->characters ("00000000-0000-0000-0000-000000000000");
754 $w->startTag ([$rasd_ns, "Template"]);
755 $w->characters ("00000000-0000-0000-0000-000000000000");
757 $w->startTag ([$rasd_ns, "ApplicationList"]);
759 $w->startTag ([$rasd_ns, "StorageId"]);
760 $w->characters ($esd_uuid);
762 $w->startTag ([$rasd_ns, "StoragePoolId"]);
763 $w->characters ("00000000-0000-0000-0000-000000000000");
765 $w->startTag ([$rasd_ns, "CreationDate"]);
766 $w->characters ($iso_time);
768 $w->startTag ([$rasd_ns, "LastModified"]);
769 $w->characters ($iso_time);
771 $w->startTag ([$rasd_ns, "last_modified_date"]);
772 $w->characters ($iso_time);
778 $w->endTag ("Section"); # ovf:VirtualHardwareSection_Type
780 $w->endTag ("Content");
782 $w->endTag ([$ovf_ns, "Envelope"]);
785 my $ovf = $w->to_string;
787 #print "OVF:\n$ovf\n";
789 my $ovf_dir = "$esd_uuid_dir/master/vms/$vm_uuid";
791 mkdir ($ovf_dir, 0755) or die "mkdir: $ovf_dir: $!";
793 my $ovf_file = "$ovf_dir/$vm_uuid.ovf";
795 open (my $fh, ">", $ovf_file) or die "open: $ovf_file: $!";
800 $delete_output_on_exit = 0;
802 print "OVF written to $ovf_file\n";
804 print "Import finished without errors. Now go to the Storage tab ->\n";
805 print "Export Storage Domain -> VM Import, and import the guest.\n";
816 Add a network card to the OVF. The problem is detecting what
817 network devices the guest can support.
821 Detect what disk models (eg. IDE, virtio-blk, virtio-scsi) the
822 guest can support and add the correct type of disk.
826 =head1 DEBUGGING IMPORT FAILURES
828 When you export to the ESD, and then import that guest through the
829 oVirt / RHEV-M UI, you may encounter an import failure. Diagnosing
830 these failures is infuriatingly difficult as the UI generally hides
831 the true reason for the failure.
833 There are two log files of interest. The first is stored on the oVirt
834 engine / RHEV-M server itself, and is called
835 F</var/log/ovirt-engine/engine.log>
837 The second file, which is the most useful, is found on the SPM host
838 (SPM stands for "Storage Pool Manager"). This is a oVirt node that is
839 elected to do all metadata modifications in the data center, such as
840 image or snapshot creation. You can find out which host is the
841 current SPM from the "Hosts" tab "Spm Status" column. Once you have
842 located the SPM, log into it and grab the file
843 F</var/log/vdsm/vdsm.log> which will contain detailed error messages
844 from low-level commands.
848 L<https://bugzilla.redhat.com/show_bug.cgi?id=998279>,
849 L<https://bugzilla.redhat.com/show_bug.cgi?id=1049604>,
851 L<engine-image-uploader(8)>.
855 Richard W.M. Jones <rjones@redhat.com>
859 Copyright (C) 2015 Richard W.M. Jones <rjones@redhat.com>
861 Copyright (C) 2015 Red Hat Inc.
865 This program is free software; you can redistribute it and/or modify it
866 under the terms of the GNU General Public License as published by the
867 Free Software Foundation; either version 2 of the License, or (at your
868 option) any later version.
870 This program is distributed in the hope that it will be useful, but
871 WITHOUT ANY WARRANTY; without even the implied warranty of
872 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
873 General Public License for more details.
875 You should have received a copy of the GNU General Public License along
876 with this program; if not, write to the Free Software Foundation, Inc.,
877 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.