From: Richard W.M. Jones <rjones@redhat.com>
Date: Tue, 9 Feb 2021 13:52:31 +0000 (+0000)
Subject: Update notes for 2021 Disk Image Pipelines.
X-Git-Url: http://git.annexia.org/?a=commitdiff_plain;h=29c7d0de6a6f6e012587ff4f74ac4fc948170a02;p=libguestfs-talks.git

Update notes for 2021 Disk Image Pipelines.
---

diff --git a/2021-pipelines/benchmarking.txt b/2021-pipelines/benchmarking.txt
new file mode 100644
index 0000000..21af30f
--- /dev/null
+++ b/2021-pipelines/benchmarking.txt
@@ -0,0 +1,68 @@
+Scenario (A)
+----------------------------------------------------------------------
+
+(1) Create fedora-33 "dirty" disk image (simulating a long-running VM)
+on a remote SSH server.  See "create.sh" script.
+
+(2) The remote server is located over a 100 Mbps ethernet link.  Both
+the client and server have plenty of local CPU and are otherwise idle
+during the test.
+
+(3) scp test:
+
+scp remote:fedora-33.img local.img
+
+(4) qcow2 snapshot + sparsify + qemu-img convert.  The time taken to
+run all of the following operations in series.  See "testA4.sh".
+
+(5) As above but without sparsify.  See "testA5.sh".
+
+(6) As above but using nbdkit + nbdcopy.  See "testA6.sh".
+
+
+Results of scenario (A)
+----------------------------------------------------------------------
+
+(3) (scp) 48m40
+    ls -lsh local.img :
+
+(4) (sparsify + qemu-img convert) 2m21
+
+(5) (qemu-img convert)  59m39
+    ls -lsh local.img :
+    4.1G -rw-r--r--. 1 rjones rjones 32G Feb 10 11:11 local.img
+
+(6) (nbdkit + nbdcopy) 3m49 [probably lock contention in cow filter]
+    $ ls -lsh local.img
+    1.2G -rw-r--r--. 1 rjones rjones 32G Feb 10 11:16 local.img
+
+
+
+Scenario (B)
+----------------------------------------------------------------------
+
+(1) Create local OVA file from disk image in scenario (A) using:
+
+tar -cf test.ova -C /path/to/remote fedora-33.img
+# -S option is not usually used with OVA files
+
+(2) Time how long it takes to copy the file:
+
+time cp test.ova test2.ova
+
+(3) Time how long it takes to unpack this using tar -xf:
+
+time tar -xf test.ova fedora-33.img
+
+(4) Time how long it takes to copy out the sparsified image.
+See "testB4.sh".
+
+
+Results of scenario (B)
+----------------------------------------------------------------------
+
+(2) (cp) 40s
+
+(3) (tar xf) 43s
+
+(4) (sparsify + qemu-img convert) 7.6s
diff --git a/2021-pipelines/notes.txt b/2021-pipelines/notes.txt
index a4038e8..6b2cb3b 100644
--- a/2021-pipelines/notes.txt
+++ b/2021-pipelines/notes.txt
@@ -18,6 +18,9 @@ The bad:
 
 
 
+
+
+
 Simple copying
 ----------------------------------------------------------------------
 
@@ -109,8 +112,8 @@ long time accumulate lots of deleted files and other stuff that isn't
 needed by the operating system but also isn't zeroes.
 
 
-We can cope with both of these things.  The technique
-is called "sparsification".
+We can cope with both of these things.  The technique is called
+"sparsification", and it's similar to the "fstrim" command.
 
   qemu-img create -f qcow2 -b ssh://kool/mnt/scratch/fedora-33.img snapshot.qcow2
   virt-sparsify --inplace snapshot.qcow2