3 # Copyright (C) 2010 Red Hat Inc.
5 # This program is free software; you can redistribute it and/or modify
6 # it under the terms of the GNU General Public License as published by
7 # the Free Software Foundation; either version 2 of the License, or
8 # (at your option) any later version.
10 # This program is distributed in the hope that it will be useful,
11 # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 # GNU General Public License for more details.
15 # You should have received a copy of the GNU General Public License
16 # along with this program; if not, write to the Free Software
17 # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
23 use Sys::Guestfs::Lib qw(feature_available);
27 use File::Temp qw(tempdir);
28 use POSIX qw(mkfifo floor);
30 use String::ShellQuote qw(shell_quote);
31 use Locale::TextDomain 'libguestfs';
37 virt-make-fs - Make a filesystem from a tar archive or files
41 virt-make-fs [--options] input.tar output.img
43 virt-make-fs [--options] input.tar.gz output.img
45 virt-make-fs [--options] directory output.img
49 Virt-make-fs is a command line tool for creating a filesystem from a
50 tar archive or some files in a directory. It is similar to tools like
51 L<mkisofs(1)>, L<genisoimage(1)> and L<mksquashfs(1)>. Unlike those
52 tools, it can create common filesystem types like ext2/3 or NTFS,
53 which can be useful if you want to attach these filesystems to
54 existing virtual machines (eg. to import large amounts of read-only
59 virt-make-fs input output
61 where C<input> is either a directory containing files that you want to
62 add, or a tar archive (either uncompressed tar or gzip-compressed
63 tar); and C<output> is a disk image. The input type is detected
64 automatically. The output disk image defaults to a raw ext2 image
65 unless you specify extra flags (see L</OPTIONS> below).
69 Unlike formats such as tar and squashfs, a filesystem does not "just
70 fit" the files that it contains, but might have extra space.
71 Depending on how you are going to use the output, you might think this
72 extra space is wasted and want to minimize it, or you might want to
73 leave space so that more files can be added later. Virt-make-fs
74 defaults to minimizing the extra space, but you can use the C<--size>
75 flag to leave space in the filesystem if you want it.
77 An alternative way to leave extra space but not make the output image
78 any bigger is to use an alternative disk image format (instead of the
79 default "raw" format). Using C<--format=qcow2> will use the native
80 QEmu/KVM qcow2 image format (check your hypervisor supports this
81 before using it). This allows you to choose a large C<--size> but the
82 extra space won't actually be allocated in the image until you try to
83 store something in it.
85 Don't forget that you can also use local commands including
86 L<resize2fs(8)> and L<virt-resize(1)> to resize existing filesystems,
87 or rerun virt-make-resize to build another image from scratch.
91 virt-make-fs --format=qcow2 --size=+200M input output.img
93 =head2 FILESYSTEM TYPE
95 The default filesystem type is C<ext2>. Just about any filesystem
96 type that libguestfs supports can be used (but I<not> read-only
97 formats like ISO9660). Here are some of the more common choices:
103 Note that ext3 filesystems contain a journal, typically 1-32 MB in size.
104 If you are not going to use the filesystem in a way that requires the
105 journal, then this is just wasted overhead.
107 =item I<ntfs> or I<vfat>
109 Useful if exporting data to a Windows guest.
111 I<Note for vfat>: The tar archive or local directory must only contain
112 files which are owned by root (ie. UID:GID = 0:0). The reason is that
113 the tar program running within libguestfs is unable to change the
114 ownership of non-root files, since vfat itself does not support this.
118 Lower overhead than C<ext2>, but certain limitations on filename
119 length and total filesystem size.
125 virt-make-fs --type=minix input minixfs.img
127 =head2 TO PARTITION OR NOT TO PARTITION
129 Optionally virt-make-fs can add a partition table to the output disk.
131 Adding a partition can make the disk image more compatible with
132 certain virtualized operating systems which don't expect to see a
133 filesystem directly located on a block device (Linux doesn't care and
134 will happily handle both types).
136 On the other hand, if you have a partition table then the output image
137 is no longer a straight filesystem. For example you cannot run
138 L<fsck(8)> directly on a partitioned disk image. (However libguestfs
139 tools such as L<guestfish(1)> and L<virt-resize(1)> can still be
144 Add an MBR partition:
146 virt-make-fs --partition -- input disk.img
148 If the output disk image could be terabyte-sized or larger, it's
149 better to use an EFI/GPT-compatible partition table:
151 virt-make-fs --partition=gpt --size=+4T --format=qcow2 input disk.img
171 Display version number and exit.
179 Enable debugging information.
185 =item B<--size=E<lt>NE<gt>>
187 =item B<--size=+E<lt>NE<gt>>
189 =item B<-s E<lt>NE<gt>>
191 =item B<-s +E<lt>NE<gt>>
193 Use the C<--size> (or C<-s>) option to choose the size of the output
196 If this option is I<not> given, then the output image will be just
197 large enough to contain all the files, with not much wasted space.
199 To choose a fixed size output disk, specify an absolute number
200 followed by b/K/M/G/T/P/E to mean bytes, Kilobytes, Megabytes,
201 Gigabytes, Terabytes, Petabytes or Exabytes. This must be large
202 enough to contain all the input files, else you will get an error.
204 To leave extra space, specify C<+> (plus sign) and a number followed
205 by b/K/M/G/T/P/E to mean bytes, Kilobytes, Megabytes, Gigabytes,
206 Terabytes, Petabytes or Exabytes. For example: C<--size=+200M> means
207 enough space for the input files, and (approximately) an extra 200 MB
210 Note that virt-make-fs estimates free space, and therefore will not
211 produce filesystems containing precisely the free space requested.
212 (It is much more expensive and time-consuming to produce a filesystem
213 which has precisely the desired free space).
219 =item B<--format=E<lt>fmtE<gt>>
221 =item B<-F E<lt>fmtE<gt>>
223 Choose the output disk image format.
225 The default is C<raw> (raw disk image).
227 For other choices, see the L<qemu-img(1)> manpage. The only other
228 choice that would really make sense here is C<qcow2>.
234 =item B<--type=E<lt>fsE<gt>>
236 =item B<-t E<lt>fsE<gt>>
238 Choose the output filesystem type.
240 The default is C<ext2>.
242 Any filesystem which is supported read-write by libguestfs can be used
251 =item B<--partition=E<lt>parttypeE<gt>>
253 If specified, this flag adds an MBR partition table to the output disk
256 You can change the partition table type, eg. C<--partition=gpt> for
259 Note that if you just use a lonesome C<--partition>, the Perl option
260 parser might consider the next parameter to be the partition type.
263 virt-make-fs --partition input.tar ...
265 would cause virt-make-fs to think you wanted to use a partition type
266 of C<input.tar> which is completely wrong. To avoid this, use C<-->
267 (a double dash) between options and the input file argument:
269 virt-make-fs --partition -- input.tar ...
275 GetOptions ("help|?" => \$help,
276 "version" => \$version,
278 "s|size=s" => \$size,
279 "F|format=s" => \$format,
280 "t|type=s" => \$type,
281 "partition:s" => \$partition,
283 pod2usage (1) if $help;
285 my $g = Sys::Guestfs->new ();
286 my %h = $g->version ();
287 print "$h{major}.$h{minor}.$h{release}$h{extra}\n";
291 die __"virt-make-fs input output\n" if @ARGV != 2;
293 my $input = $ARGV[0];
294 my $output = $ARGV[1];
296 # Input. What is it? Estimate how much space it will need.
298 # Estimation is a Hard Problem. Some factors which make it hard:
300 # - Superblocks, block free bitmaps, FAT and other fixed overhead
301 # - Indirect blocks (ext2, ext3), and extents
303 # - Internal fragmentation of files
305 # What we could also do is try shrinking the filesystem after creating
306 # and populating it, but that is complex given partitions.
308 my $estimate; # Estimated size required (in bytes).
309 my $ifmt; # Input format.
314 my @cmd = ("du", "--apparent-size", "-b", "-s", $input);
315 open PIPE, "-|", @cmd or die "du $input: $!";
321 die __"unexpected output from 'du' command";
324 local $ENV{LANG} = "C";
325 my @cmd = ("file", "-bsLz", $input);
326 open PIPE, "-|", @cmd or die "file $input: $!";
332 if ($ifmt !~ /tar archive/) {
333 die __x("{f}: unknown input format: {fmt}\n",
334 f => $input, fmt => $ifmt);
337 if ($ifmt =~ /compress.d/) {
338 if ($ifmt =~ /compress'd/) {
339 @cmd = ("uncompress", "-c", $input);
340 } elsif ($ifmt =~ /gzip compressed/) {
341 @cmd = ("gzip", "-cd", $input);
342 } elsif ($ifmt =~ /bzip2 compressed/) {
343 @cmd = ("bzip2", "-cd", $input);
344 } elsif ($ifmt =~ /xz compressed/) {
345 @cmd = ("xz", "-cd", $input);
347 die __x("{f}: unknown input format: {fmt}\n",
348 f => $input, fmt => $ifmt);
351 open PIPE, "-|", @cmd or die "uncompress $input: $!";
353 $estimate += length while <PIPE>;
354 close PIPE or die "close: $!";
356 # Plain tar file, just get the size directly. Tar files have
357 # a 512 byte block size (compared with typically 1K or 4K for
358 # filesystems) so this isn't very accurate.
359 $estimate = -s $input;
364 printf STDERR "input format = %s\n", $ifmt;
365 printf STDERR "estimate = %s bytes (%s 1K blocks, %s 4K blocks)\n",
366 $estimate, $estimate / 1024, $estimate / 4096;
369 $estimate += 256 * 1024; # For superblocks &c.
371 if ($type =~ /^ext[3-9]/) {
372 $estimate += 1024 * 1024; # For ext3/4, add some more for the journal.
375 if ($type =~ /^ntfs/) {
376 $estimate += 4 * 1024 * 1024; # NTFS journal.
379 $estimate *= 1.10; # Add 10%, see above.
381 # Calculate the output size.
383 if (!defined $size) {
386 if ($size =~ /^\+([.\d]+)([bKMGTPE])$/) {
387 $size = $estimate + sizebytes ($1, $2);
388 } elsif ($size =~ /^([.\d]+)([bKMGTPE])$/) {
389 $size = sizebytes ($1, $2);
391 die __x("virt-make-fs: cannot parse size parameter: {sz}\n",
396 # Create the output disk.
397 # Take the unusual step of invoking qemu-img here.
399 my @cmd = ("qemu-img", "create", "-f", $format, $output, $size);
400 system (@cmd) == 0 or
401 die __"qemu-img create: failed to create disk image, see earlier error messages\n";
404 print STDERR "starting libguestfs ...\n" if $debug;
407 my $g = Sys::Guestfs->new ();
408 $g->add_drive ($output);
411 if ($type eq "ntfs" && !feature_available ($g, "ntfs3g", "ntfsprogs")) {
412 die __"virt-make-fs: NTFS support was disabled when libguestfs was compiled\n"
415 # Partition the disk.
416 my $dev = "/dev/sda";
417 if (defined $partition) {
418 $partition = "mbr" if $partition eq "";
419 $g->part_disk ($dev, $partition);
423 print STDERR "creating $type filesystem on $dev ...\n" if $debug;
425 # Create the filesystem.
426 $g->mkfs ($type, $dev);
427 $g->mount_options ("", $dev, "/");
432 if ($ifmt eq "directory") {
433 my $pfile = create_pipe ();
434 my $cmd = sprintf ("tar -C %s -cf - . > $pfile &",
435 shell_quote ($input));
436 print STDERR "command: $cmd\n" if $debug;
437 system ($cmd) == 0 or die __"tar: failed, see earlier messages\n";
440 if ($ifmt =~ /compress.d/) {
441 my $pfile = create_pipe ();
443 if ($ifmt =~ /compress'd/) {
444 $cmd = sprintf ("uncompress -c %s > $pfile",
445 shell_quote ($input));
446 } elsif ($ifmt =~ /gzip compressed/) {
447 $cmd = sprintf ("gzip -cd %s", shell_quote ($input));
448 } elsif ($ifmt =~ /bzip2 compressed/) {
449 $cmd = sprintf ("bzip2 -cd %s", shell_quote ($input));
450 } elsif ($ifmt =~ /xz compressed/) {
451 $cmd = sprintf ("xz -cd %s", shell_quote ($input));
453 die __x("{f}: unknown input format: {fmt}\n",
454 f => $input, fmt => $ifmt);
456 $cmd .= " > $pfile &";
457 print STDERR "command: $cmd\n" if $debug;
458 system ($cmd) == 0 or
459 die __"uncompress command failed, see earlier messages\n";
462 print STDERR "reading directly from $input\n" if $debug;
468 # For debugging, print statvfs before and after doing
470 my %stat = $g->statvfs ("/");
471 print STDERR "Before uploading ...\n";
472 print STDERR Dumper(\%stat);
475 print STDERR "Uploading from $ifile to / ...\n" if $debug;
476 $g->tar_in ($ifile, "/");
479 my %stat = $g->statvfs ("/");
480 print STDERR "After uploading ...\n";
481 print STDERR Dumper(\%stat);
484 print STDERR "finishing off\n" if $debug;
490 # Error: delete the output before exiting.
493 if ($err =~ /tar_in/) {
494 print STDERR __"virt-make-fs: error copying contents into filesystem\nAn error here usually means that the program did not estimate the\nfilesystem size correctly. Please read the BUGS section of the manpage.\n";
507 $_ *= 1024 if $unit =~ /[KMGTPE]/;
508 $_ *= 1024 if $unit =~ /[MGTPE]/;
509 $_ *= 1024 if $unit =~ /[GTPE]/;
510 $_ *= 1024 if $unit =~ /[TPE]/;
511 $_ *= 1024 if $unit =~ /[PE]/;
512 $_ *= 1024 if $unit =~ /[E]/;
520 my $dir = tempdir (CLEANUP => 1);
521 my $pipe = "$dir/pipe";
522 mkfifo ($pipe, 0600) or
523 die "mkfifo: $pipe: $!";
539 L<http://libguestfs.org/>.
543 When reporting bugs, please enable debugging and capture the
546 export LIBGUESTFS_DEBUG=1
547 virt-make-fs --debug [...] > /tmp/virt-make-fs.log 2>&1
549 Attach /tmp/virt-make-fs.log to a new bug report at
550 L<https://bugzilla.redhat.com/>
554 Richard W.M. Jones L<http://people.redhat.com/~rjones/>
558 Copyright (C) 2010 Red Hat Inc.
560 This program is free software; you can redistribute it and/or modify
561 it under the terms of the GNU General Public License as published by
562 the Free Software Foundation; either version 2 of the License, or
563 (at your option) any later version.
565 This program is distributed in the hope that it will be useful,
566 but WITHOUT ANY WARRANTY; without even the implied warranty of
567 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
568 GNU General Public License for more details.
570 You should have received a copy of the GNU General Public License
571 along with this program; if not, write to the Free Software
572 Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.