3 # Copyright (C) 2010 Red Hat Inc.
5 # This program is free software; you can redistribute it and/or modify
6 # it under the terms of the GNU General Public License as published by
7 # the Free Software Foundation; either version 2 of the License, or
8 # (at your option) any later version.
10 # This program is distributed in the hope that it will be useful,
11 # but WITHOUT ANY WARRANTY; without even the implied warranty of
12 # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
13 # GNU General Public License for more details.
15 # You should have received a copy of the GNU General Public License
16 # along with this program; if not, write to the Free Software
17 # Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
26 use File::Temp qw(tempdir);
27 use POSIX qw(mkfifo floor);
29 use String::ShellQuote qw(shell_quote);
30 use Locale::TextDomain 'libguestfs';
36 virt-make-fs - Make a filesystem from a tar archive or files
40 virt-make-fs [--options] input.tar output.img
42 virt-make-fs [--options] input.tar.gz output.img
44 virt-make-fs [--options] directory output.img
48 Virt-make-fs is a command line tool for creating a filesystem from a
49 tar archive or some files in a directory. It is similar to tools like
50 L<mkisofs(1)>, L<genisoimage(1)> and L<mksquashfs(1)>. Unlike those
51 tools, it can create common filesystem types like ext2/3 or NTFS,
52 which can be useful if you want to attach these filesystems to
53 existing virtual machines (eg. to import large amounts of read-only
58 virt-make-fs input output
60 where C<input> is either a directory containing files that you want to
61 add, or a tar archive (either uncompressed tar or gzip-compressed
62 tar); and C<output> is a disk image. The input type is detected
63 automatically. The output disk image defaults to a raw ext2 image
64 unless you specify extra flags (see L</OPTIONS> below).
68 Unlike formats such as tar and squashfs, a filesystem does not "just
69 fit" the files that it contains, but might have extra space.
70 Depending on how you are going to use the output, you might think this
71 extra space is wasted and want to minimize it, or you might want to
72 leave space so that more files can be added later. Virt-make-fs
73 defaults to minimizing the extra space, but you can use the C<--size>
74 flag to leave space in the filesystem if you want it.
76 An alternative way to leave extra space but not make the output image
77 any bigger is to use an alternative disk image format (instead of the
78 default "raw" format). Using C<--format=qcow2> will use the native
79 QEmu/KVM qcow2 image format (check your hypervisor supports this
80 before using it). This allows you to choose a large C<--size> but the
81 extra space won't actually be allocated in the image until you try to
82 store something in it.
84 Don't forget that you can also use local commands including
85 L<resize2fs(8)> and L<virt-resize(1)> to resize existing filesystems,
86 or rerun virt-make-resize to build another image from scratch.
90 virt-make-fs --format=qcow2 --size=+200M input output.img
92 =head2 FILESYSTEM TYPE
94 The default filesystem type is C<ext2>. Just about any filesystem
95 type that libguestfs supports can be used (but I<not> read-only
96 formats like ISO9660). Here are some of the more common choices:
102 Note that ext3 filesystems contain a journal, typically 1-32 MB in size.
103 If you are not going to use the filesystem in a way that requires the
104 journal, then this is just wasted overhead.
106 =item I<ntfs> or I<vfat>
108 Useful if exporting data to a Windows guest.
110 I<Note for vfat>: The tar archive or local directory must only contain
111 files which are owned by root (ie. UID:GID = 0:0). The reason is that
112 the tar program running within libguestfs is unable to change the
113 ownership of non-root files, since vfat itself does not support this.
117 Lower overhead than C<ext2>, but certain limitations on filename
118 length and total filesystem size.
124 virt-make-fs --type=minix input minixfs.img
126 =head2 TO PARTITION OR NOT TO PARTITION
128 Optionally virt-make-fs can add a partition table to the output disk.
130 Adding a partition can make the disk image more compatible with
131 certain virtualized operating systems which don't expect to see a
132 filesystem directly located on a block device (Linux doesn't care and
133 will happily handle both types).
135 On the other hand, if you have a partition table then the output image
136 is no longer a straight filesystem. For example you cannot run
137 L<fsck(8)> directly on a partitioned disk image. (However libguestfs
138 tools such as L<guestfish(1)> and L<virt-resize(1)> can still be
143 Add an MBR partition:
145 virt-make-fs --partition -- input disk.img
147 If the output disk image could be terabyte-sized or larger, it's
148 better to use an EFI/GPT-compatible partition table:
150 virt-make-fs --partition=gpt --size=+4T --format=qcow2 input disk.img
170 Display version number and exit.
178 Enable debugging information.
184 =item B<--size=E<lt>NE<gt>>
186 =item B<--size=+E<lt>NE<gt>>
188 =item B<-s E<lt>NE<gt>>
190 =item B<-s +E<lt>NE<gt>>
192 Use the C<--size> (or C<-s>) option to choose the size of the output
195 If this option is I<not> given, then the output image will be just
196 large enough to contain all the files, with not much wasted space.
198 To choose a fixed size output disk, specify an absolute number
199 followed by b/K/M/G/T/P/E to mean bytes, Kilobytes, Megabytes,
200 Gigabytes, Terabytes, Petabytes or Exabytes. This must be large
201 enough to contain all the input files, else you will get an error.
203 To leave extra space, specify C<+> (plus sign) and a number followed
204 by b/K/M/G/T/P/E to mean bytes, Kilobytes, Megabytes, Gigabytes,
205 Terabytes, Petabytes or Exabytes. For example: C<--size=+200M> means
206 enough space for the input files, and (approximately) an extra 200 MB
209 Note that virt-make-fs estimates free space, and therefore will not
210 produce filesystems containing precisely the free space requested.
211 (It is much more expensive and time-consuming to produce a filesystem
212 which has precisely the desired free space).
218 =item B<--format=E<lt>fmtE<gt>>
220 =item B<-F E<lt>fmtE<gt>>
222 Choose the output disk image format.
224 The default is C<raw> (raw disk image).
226 For other choices, see the L<qemu-img(1)> manpage. The only other
227 choice that would really make sense here is C<qcow2>.
233 =item B<--type=E<lt>fsE<gt>>
235 =item B<-t E<lt>fsE<gt>>
237 Choose the output filesystem type.
239 The default is C<ext2>.
241 Any filesystem which is supported read-write by libguestfs can be used
250 =item B<--partition=E<lt>parttypeE<gt>>
252 If specified, this flag adds an MBR partition table to the output disk
255 You can change the partition table type, eg. C<--partition=gpt> for
258 Note that if you just use a lonesome C<--partition>, the Perl option
259 parser might consider the next parameter to be the partition type.
262 virt-make-fs --partition input.tar ...
264 would cause virt-make-fs to think you wanted to use a partition type
265 of C<input.tar> which is completely wrong. To avoid this, use C<-->
266 (a double dash) between options and the input file argument:
268 virt-make-fs --partition -- input.tar ...
274 GetOptions ("help|?" => \$help,
275 "version" => \$version,
277 "s|size=s" => \$size,
278 "F|format=s" => \$format,
279 "t|type=s" => \$type,
280 "partition:s" => \$partition,
282 pod2usage (1) if $help;
284 my $g = Sys::Guestfs->new ();
285 my %h = $g->version ();
286 print "$h{major}.$h{minor}.$h{release}$h{extra}\n";
290 die __"virt-make-fs input output\n" if @ARGV != 2;
292 my $input = $ARGV[0];
293 my $output = $ARGV[1];
295 # Input. What is it? Estimate how much space it will need.
297 # Estimation is a Hard Problem. Some factors which make it hard:
299 # - Superblocks, block free bitmaps, FAT and other fixed overhead
300 # - Indirect blocks (ext2, ext3), and extents
302 # - Internal fragmentation of files
304 # What we could also do is try shrinking the filesystem after creating
305 # and populating it, but that is complex given partitions.
307 my $estimate; # Estimated size required (in bytes).
308 my $ifmt; # Input format.
313 my @cmd = ("du", "--apparent-size", "-b", "-s", $input);
314 open PIPE, "-|", @cmd or die "du $input: $!";
320 die __"unexpected output from 'du' command";
323 local $ENV{LANG} = "C";
324 my @cmd = ("file", "-bsLz", $input);
325 open PIPE, "-|", @cmd or die "file $input: $!";
331 if ($ifmt !~ /tar archive/) {
332 die __x("{f}: unknown input format: {fmt}\n",
333 f => $input, fmt => $ifmt);
336 if ($ifmt =~ /compress.d/) {
337 if ($ifmt =~ /compress'd/) {
338 @cmd = ("uncompress", "-c", $input);
339 } elsif ($ifmt =~ /gzip compressed/) {
340 @cmd = ("gzip", "-cd", $input);
341 } elsif ($ifmt =~ /bzip2 compressed/) {
342 @cmd = ("bzip2", "-cd", $input);
343 } elsif ($ifmt =~ /xz compressed/) {
344 @cmd = ("xz", "-cd", $input);
346 die __x("{f}: unknown input format: {fmt}\n",
347 f => $input, fmt => $ifmt);
350 open PIPE, "-|", @cmd or die "uncompress $input: $!";
352 $estimate += length while <PIPE>;
353 close PIPE or die "close: $!";
355 # Plain tar file, just get the size directly. Tar files have
356 # a 512 byte block size (compared with typically 1K or 4K for
357 # filesystems) so this isn't very accurate.
358 $estimate = -s $input;
363 printf STDERR "input format = %s\n", $ifmt;
364 printf STDERR "estimate = %s bytes (%s 1K blocks, %s 4K blocks)\n",
365 $estimate, $estimate / 1024, $estimate / 4096;
368 $estimate += 256 * 1024; # For superblocks &c.
370 if ($type =~ /^ext[3-9]/) {
371 $estimate += 1024 * 1024; # For ext3/4, add some more for the journal.
374 if ($type =~ /^ntfs/) {
375 $estimate += 4 * 1024 * 1024; # NTFS journal.
378 $estimate *= 1.10; # Add 10%, see above.
380 # Calculate the output size.
382 if (!defined $size) {
385 if ($size =~ /^\+([.\d]+)([bKMGTPE])$/) {
386 $size = $estimate + sizebytes ($1, $2);
387 } elsif ($size =~ /^([.\d]+)([bKMGTPE])$/) {
388 $size = sizebytes ($1, $2);
390 die __x("virt-make-fs: cannot parse size parameter: {sz}\n",
395 # Create the output disk.
396 # Take the unusual step of invoking qemu-img here.
398 my @cmd = ("qemu-img", "create", "-f", $format, $output, $size);
399 system (@cmd) == 0 or
400 die __"qemu-img create: failed to create disk image, see earlier error messages\n";
403 print STDERR "starting libguestfs ...\n" if $debug;
406 my $g = Sys::Guestfs->new ();
407 $g->add_drive ($output);
410 if ($type eq "ntfs") {
411 $g->available ([ "ntfs3g" ]);
414 # Partition the disk.
415 my $dev = "/dev/sda";
416 if (defined $partition) {
417 $partition = "mbr" if $partition eq "";
418 $g->part_disk ($dev, $partition);
422 print STDERR "creating $type filesystem on $dev ...\n" if $debug;
424 # Create the filesystem.
425 $g->mkfs ($type, $dev);
426 $g->mount_options ("", $dev, "/");
431 if ($ifmt eq "directory") {
432 my $pfile = create_pipe ();
433 my $cmd = sprintf ("tar -C %s -cf - . > $pfile &",
434 shell_quote ($input));
435 print STDERR "command: $cmd\n" if $debug;
436 system ($cmd) == 0 or die __"tar: failed, see earlier messages\n";
439 if ($ifmt =~ /compress.d/) {
440 my $pfile = create_pipe ();
442 if ($ifmt =~ /compress'd/) {
443 $cmd = sprintf ("uncompress -c %s > $pfile",
444 shell_quote ($input));
445 } elsif ($ifmt =~ /gzip compressed/) {
446 $cmd = sprintf ("gzip -cd %s", shell_quote ($input));
447 } elsif ($ifmt =~ /bzip2 compressed/) {
448 $cmd = sprintf ("bzip2 -cd %s", shell_quote ($input));
449 } elsif ($ifmt =~ /xz compressed/) {
450 $cmd = sprintf ("xz -cd %s", shell_quote ($input));
452 die __x("{f}: unknown input format: {fmt}\n",
453 f => $input, fmt => $ifmt);
455 $cmd .= " > $pfile &";
456 print STDERR "command: $cmd\n" if $debug;
457 system ($cmd) == 0 or
458 die __"uncompress command failed, see earlier messages\n";
461 print STDERR "reading directly from $input\n" if $debug;
467 # For debugging, print statvfs before and after doing
469 my %stat = $g->statvfs ("/");
470 print STDERR "Before uploading ...\n";
471 print STDERR Dumper(\%stat);
474 print STDERR "Uploading from $ifile to / ...\n" if $debug;
475 $g->tar_in ($ifile, "/");
478 my %stat = $g->statvfs ("/");
479 print STDERR "After uploading ...\n";
480 print STDERR Dumper(\%stat);
483 print STDERR "finishing off\n" if $debug;
489 # Error: delete the output before exiting.
492 if ($err =~ /tar_in/) {
493 print STDERR __"virt-make-fs: error copying contents into filesystem\nAn error here usually means that the program did not estimate the\nfilesystem size correctly. Please read the BUGS section of the manpage.\n";
506 $_ *= 1024 if $unit =~ /[KMGTPE]/;
507 $_ *= 1024 if $unit =~ /[MGTPE]/;
508 $_ *= 1024 if $unit =~ /[GTPE]/;
509 $_ *= 1024 if $unit =~ /[TPE]/;
510 $_ *= 1024 if $unit =~ /[PE]/;
511 $_ *= 1024 if $unit =~ /[E]/;
519 my $dir = tempdir (CLEANUP => 1);
520 my $pipe = "$dir/pipe";
521 mkfifo ($pipe, 0600) or
522 die "mkfifo: $pipe: $!";
538 L<http://libguestfs.org/>.
542 When reporting bugs, please enable debugging and capture the
545 export LIBGUESTFS_DEBUG=1
546 virt-make-fs --debug [...] > /tmp/virt-make-fs.log 2>&1
548 Attach /tmp/virt-make-fs.log to a new bug report at
549 L<https://bugzilla.redhat.com/>
553 Richard W.M. Jones L<http://et.redhat.com/~rjones/>
557 Copyright (C) 2010 Red Hat Inc.
559 This program is free software; you can redistribute it and/or modify
560 it under the terms of the GNU General Public License as published by
561 the Free Software Foundation; either version 2 of the License, or
562 (at your option) any later version.
564 This program is distributed in the hope that it will be useful,
565 but WITHOUT ANY WARRANTY; without even the implied warranty of
566 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
567 GNU General Public License for more details.
569 You should have received a copy of the GNU General Public License
570 along with this program; if not, write to the Free Software
571 Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.