2021 talk about Disk Image Pipelines.

Add very basic notes, very incomplete.

diff --git a/2021-pipelines/notes.txt b/2021-pipelines/notes.txt
new file mode 100644 (file)
index 0000000..a4038e8
--- /dev/null
+++ b/2021-pipelines/notes.txt
@@ -0,0 +1,118 @@
+Disk Image Pipelines
+February 15th 2021
+======================================================================
+
+Introduction
+----------------------------------------------------------------------
+
+Today I'm going to talk about a topic which seems very obvious to me.
+But I work in the business of moving disk images between systems, and
+it may not be obvious to other people.
+
+INTRO INTRO INTRO
+INVOLVE PIPELINES DIAGRAM
+
+The bad:
+ - creating unbounded temporary files
+ - slow copies
+
+
+
+Simple copying
+----------------------------------------------------------------------
+
+Let's start with something trivial: Let's boot a virtual machine from
+a local disk image.  Normally I'd say use libvirt, but for this demo
+I'm going to get very simple and run qemu directly.
+
+DIAGRAM:
+
+  local file -> qemu
+
+  qemu-system-x86_64 -machine accel=kvm:tcg -cpu host
+                     -m 2048 -display none \
+                     -drive file=fedora-33.img,format=raw,if=virtio \
+                     -serial stdio
+
+
+DIAGRAM:
+
+         ssh
+  file -------> qemu
+
+A lesser-known fact about qemu is that it contains an SSH client so
+you can boot from a remote file over SSH:
+
+COMMAND:
+
+  qemu-system-x86_64 -machine accel=kvm:tcg -cpu host
+                     -m 2048 -display none \
+                     -drive file=ssh://kool/mnt/scratch/fedora-33.img,format=raw,if=virtio \
+                     -serial stdio
+
+
+Snapshots
+----------------------------------------------------------------------
+
+DIAGRAM:
+
+         ssh
+  file -------> snapshot ------> qemu
+
+That command opens the remote file for writes.  If we want to prevent
+modifications to the remote file, we can place a snapshot into the
+path.  A snapshot in this case is a qcow2 file with the backing file
+set to the SSH URL.  Any modifications we make are saved into the
+snapshot.
+
+COMMAND:
+
+  qemu-img create -f qcow2 -b ssh://kool/mnt/scratch/fedora-33.img snapshot.qcow2
+  qemu-system-x86_64 -machine accel=kvm:tcg -cpu host
+                     -m 2048 -display none \
+                     -drive file=snapshot.qcow2,format=qcow2,if=virtio \
+                     -serial stdio
+
+
+Copying
+----------------------------------------------------------------------
+
+Instead of booting the disk, let's make a full local copy:
+
+COMMAND:
+
+  qemu-img convert -f qcow2 snapshot.qcow2 -O raw disk.img -p
+
+
+
+Sparsification
+----------------------------------------------------------------------
+
+DIAGRAM:
+
+  [ XXXXXXXXXX .... DDDDDDDD XXXXXXXXXXXX .......... ]
+
+Now let's take side-step to talk about what's inside disk images.
+
+Firstly disk images are often HUGE.  A BluRay movie is 50 gigabytes,
+but that's really nothing compared to the larger disk images that we
+move about when we do "lift and shift" of workloads from foreign
+hypervisors to KVM.  Those can be hundreds of gigabytes or terabytes,
+and we move hundreds of them in a single batch.
+
+But the good news is that these disk images are often quite sparse.
+They may contain much less actual data than the virtual size.
+A lot of the disk may be filled with zeroes.
+
+But the bad news is that virtual machines that have been running for a
+long time accumulate lots of deleted files and other stuff that isn't
+needed by the operating system but also isn't zeroes.
+
+
+We can cope with both of these things.  The technique
+is called "sparsification".
+
+  qemu-img create -f qcow2 -b ssh://kool/mnt/scratch/fedora-33.img snapshot.qcow2
+  virt-sparsify --inplace snapshot.qcow2
+  ls -l snapshot.qcow2
+