+"sparsification". Some tools you can use to sparsify a disk are:
+
+ fstrim
+ virt-sparsify --in-place
+
+Sparsification part 2
+----------------------------------------------------------------------
+
+ ssh
+ file -------> snapshot <------ virt-sparsify
+ ------> qemu-img convert
+ ^
+ |
+ zero clusters are saved in here
+
+I'm going to take the same scenario as before, but use
+sparsification before doing the copy.
+
+(Run these commands and show the output and ls of the snapshot)
+
+
+
+Benchmark A
+----------------------------------------------------------------------
+
+Now you might think this is all a bit obscure, but how does it apply
+to copying disk images. In this first benchmark, I've compared
+copying a disk in several different ways to see which is fastest. All
+of the copying happens between two idle machines, over a slow network.
+
+The full methodology is in the background notes that accompany this
+talk, which I'll link at the end.
+
+ scp scp remote:fedora-33.img local.img
+
+ ssh
+ sparsify file -> qcow2 snapshot <- virt-sparsify
+ -> qemu-img convert
+
+ without (as above but without sparsifying)
+ sparsify
+
+ ssh
+ nbdcopy file -> nbdkit cow filter <- virt-sparsify
+ -> nbdcopy
+
+Which do you think will be faster?
+
+
+Benchmark A results
+----------------------------------------------------------------------
+
+(Same slides with timings added)
+
+Lock contention in the cow filter is thought to be the
+reason for the poor performance of nbdkit + nbdcopy.
+
+
+Opening OVA files
+----------------------------------------------------------------------
+
+DIAGRAM:
+
+ guest.ova -----> tar-filter <- virt-sparsify
+ -> qemu-img convert
+
+ guest.ova------------+
+ | guest.ovf |
+ | disk1.raw|vmdk |
+ +--------------------+
+
+ tar file = header - file - header - file - ...
+
+This technique isn't just useful for remote files. Another trick we
+use in virt-v2v is using an nbdkit filter to unpack VMware's OVA files
+without any copies. OVA files are really uncompressed tar files. The
+disk inside can be in a variety of formats, often raw or VMDK.
+
+We can ask the 'tar' command to give us the offset and size of the
+disk image within the file and simply read it out of the file
+directly.
+
+
+Benchmark B
+----------------------------------------------------------------------
+
+ cp test.ova test2.ova
+
+ tar xf test.ova fedora-33.img
+
+
+ nbdkit -> tar filter <- sparsify
+ -> qemu-img convert
+
+ nbdkit -f --exit-with-parent --filter=tar file test.ova tar-entry=fedora-33.img
+ qemu-img create -f qcow2 -b nbd:localhost:10809 snapshot.qcow2
+ virt-sparsify --inplace snapshot.qcow2
+ qemu-img convert -f qcow2 snapshot.qcow2 -O raw local.img
+
+Which is faster?
+
+Benchmark B results
+----------------------------------------------------------------------
+
+(Same as above, with results)
+
+The results are interesting, but if you remember what we said about
+the disk format and sparsification then it shouldn't be surprising.
+
+The copy and tar commands have to churn through the entire
+disk image - zeroes and deleted files.
+
+With nbdkit, sparsification and qemu-img convert we only copy a
+fraction of the data.
+
+Note the two methods do NOT produce bit-for-bit equivalent outputs.
+Q: Is this a problem?
+A: No different from if the owner of the VM had run "fstrim".
+
+
+Modifications
+----------------------------------------------------------------------
+
+Virt-v2v doesn't only make efficient copies, it also modifies the disk
+image in flight. Some kinds of modifications that are made:
+
+ - installing virtio drivers
+
+ - removing VMware tools
+
+ - modifying the bootloader
+
+ - rebuilding initramfs
+
+ - changing device names in /etc files
+
+ - changing the Windows registry
+
+ - (and much more)
+
+These are significant modifications, and they happen entirely during
+the transfer, without touching the source and without making large
+temporary copies.
+
+I'm not going to talk about this in great detail because it's a very
+complex topic. Instead I will show you a simple demonstration of a
+similar technique.
+
+ qemu-img create -f qcow2 -b ssh://kool/mnt/scratch/pipes/fedora-33.img snapshot.qcow2
+ virt-sparsify --inplace snapshot.qcow2
+ guestfish -a snapshot.qcow2 -i write /etc/motd 'HEY, IT WORKED!'
+ ls -lh snapshot.qcow2
+ qemu-img convert -f qcow2 snapshot.qcow2 -O raw local.img -p
+ virt-cat -a local.img /etc/motd
+
+(Show this as a demo. Show original untouched)
+
+
+Complete virt-v2v paths
+----------------------------------------------------------------------
+
+DIAGRAM:
+
+ proprietary
+ transport
+ VMware -----> nbdkit ----> nbdkit ----> qcow2
+ ESXi vddk rate snapshot
+ plugin filter
+
+ qcow2 <---- sparsify
+ snapshot <---- install drivers
+ -----> qemu-img convert
+
+ nbd HTTPS
+ qemu-img convert ----> nbdkit -----> imageio
+ python
+ plugin
+
+Discuss:
+
+ - separate input and output sides
+
+ - NBD used extensively
+
+ - very efficient and no large temporary copies