2019-kvm-forum/papers/NBD-changes.tex

   1 \documentclass[12pt,a4paper]{article}
   2 \usepackage[utf8x]{inputenc}
   3 \usepackage{parskip}
   4 \usepackage{hyperref}
   5 \usepackage{xcolor}
   6 \hypersetup{
   7     colorlinks,
   8     linkcolor={red!50!black},
   9     citecolor={blue!50!black},
  10     urlcolor={blue!80!black}
  11 }
  12 %\usepackage{abstract}
  13 %\usepackage{graphicx}
  14 %\DeclareGraphicsExtensions{.pdf,.png,.jpg}
  15 \usepackage{float}
  16 \floatstyle{boxed}
  17 \restylefloat{figure}
  18 \usepackage{fancyhdr}
  19   \pagestyle{fancy}
  20   %\fancyhead{}
  21   %\fancyfoot{}
  22
  23 \title{Summary of changes in NBD-related projects since mid 2016}
  24
  25 \begin{document}
  26 \maketitle
  27
  28 \section{NBD protocol}
  29 \url{https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md}
  30
  31 \begin{description}
  32 \item[2016]
  33
  34 \verb|WRITE_ZEROES|
  35
  36 \verb|NBD_OPT_INFO| and \verb|NBD_OPT_GO| (finalized)
  37
  38 TLS support (finalized)
  39
  40 \verb|NBD_OPT_BLOCK_SIZE|
  41
  42 \verb|NBD_FLAG_CAN_MULTI_CONN|
  43
  44 \item[2017]
  45
  46 \verb|STRUCTURED_REPLY| merged
  47
  48 \item[2018]
  49
  50 \verb|NBD_CMD_BLOCK_STATUS|, metadata contexts, \verb|base:|
  51 namespace, etc. merged
  52
  53 Preferred block size specified as 512 bytes.  Document maximum
  54 request size limits.
  55
  56 \verb|NBD_CMD_CACHE|
  57
  58 \item[2019]
  59
  60 \verb|NBD_CMD_FLAG_FAST_ZERO|
  61
  62 Recommend use of \verb|TCP_NODELAY|
  63
  64 \end{description}
  65
  66 \subsection{URIs}
  67
  68 \begin{description}
  69 \item[2019]
  70 NBD URI specification was introduced.
  71 \url{https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md}
  72 \end{description}
  73
  74
  75 \f
  76 \section{qemu-nbd}
  77
  78
  79
  80 \f
  81 \section{nbdkit}
  82
  83 \begin{description}
  84
  85 \item[1.4, July 2018]
  86
  87 \url{https://www.redhat.com/archives/libguestfs/2018-July/msg00023.html}
  88
  89 New plugins: Tcl, ext2, random, zero
  90
  91 New filters: blocksize, fua, log, nozero
  92
  93 TLS-PSK
  94
  95 TRIM support
  96
  97 \item[1.6, Aug 2018]
  98
  99 \url{https://www.redhat.com/archives/libguestfs/2018-August/msg00279.html}
 100
 101 New plugins: Lua, pattern, data
 102
 103 New filters: truncate, error
 104
 105 \verb|NBD_OPT_GO| supported
 106
 107 FreeBSD port
 108
 109 \verb|-D| options for debugging
 110
 111 \item[1.8, Nov 2018]
 112
 113 \url{https://www.redhat.com/archives/libguestfs/2018-November/msg00111.html}
 114
 115 New plugins: sh, iso, floppy, partitioning
 116
 117 Haiku port
 118
 119 \item[1.10, Jan 2019]
 120
 121 \url{https://www.redhat.com/archives/libguestfs/2019-January/msg00135.html}
 122
 123 New plugins: full
 124
 125 New filters: cache, delay, xz
 126
 127 Multi-conn support
 128
 129 \verb|NBD_OPT_INFO|
 130
 131 Fuzz testing
 132
 133 \item[1.12, Apr 2019]
 134
 135 \url{https://www.redhat.com/archives/virt-tools-list/2019-April/msg00045.html}
 136
 137 New plugins: ssh, linuxdisk, rust
 138
 139 New filters: rate, readahead, noextents
 140
 141 \verb|NBD_CMD_BLOCK_STATUS|
 142
 143 \item[1.14, Aug 2019]
 144
 145 \url{https://www.redhat.com/archives/libguestfs/2019-August/msg00375.html}
 146
 147 New filters: stats, nocache, noparallel, cacheextents
 148
 149 Many performance improvements
 150
 151 \verb|NBD_CMD_CACHE|
 152
 153 \end{description}
 154
 155 \f
 156 \section{libnbd}
 157
 158 Project started last day of April 2019.
 159
 160 \begin{description}
 161
 162 \item[May 2019]
 163
 164 Most \verb|NBD_CMD_| implemented, and working connection to nbdkit,
 165 within first 10 days.  Interoperating with qemu-nbd within 2 weeks.
 166 TLS implemented and interoperating with qemu-nbd within 16 days.
 167
 168 Generated C API, Python 3 bindings and \verb|nbdsh|, OCaml bindings,
 169 hierarchical state machine.
 170
 171 Export names
 172
 173 Subprocess server (\verb|nbd_connect_command|)
 174
 175 Structured replies
 176
 177 \verb|NBD_CMD_BLOCK_STATUS| and \verb|base:allocation|
 178
 179 Multiple commands ready to issue and in flight
 180
 181 Callbacks
 182
 183 ``Internal'' multi-conn support added then removed
 184
 185 Valgrinding
 186
 187 Some API calls can be handled without taking the lock
 188
 189 libnbd $0.1.2$ was released on May $30^{th}$ 2019.
 190
 191 \item[June 2019]
 192
 193 NBD URIs supported
 194
 195 \verb|nbdsh -c| parameter
 196
 197 \verb|MSG_MORE| optimization
 198
 199 Better handling of multiple in-flight commands, a lot of
 200 optimization, and finessing of the API.
 201
 202 Python bindings custom exception \verb|nbd.Error|
 203
 204 \item[July 2019]
 205
 206 Port to FreeBSD
 207
 208 Further work on callbacks
 209
 210 Optimize command queue to be $O(1)$ instead of $O(n)$
 211
 212 Symbol versions
 213
 214 libnbd $0.9.6$ was released on Jul $30^{th}$ 2019.
 215
 216 \item[August 2019]
 217
 218 More tests
 219
 220 Debug output includes function names, parameters, return values
 221
 222 Optional function parameters
 223
 224 Man pages for every API
 225
 226 Port to RHEL 7
 227
 228 First stable version $1.0$ was released on August $27^{th}$ 2019.
 229
 230 \item[September 2019]
 231
 232 Support for \verb|FAST_ZERO|
 233
 234 New APIs for testing servers without Structured Replies and explicitly
 235 setting handshake flags.  New APIs for finding out if TLS was
 236 negotiated, and what NBD protocol is being used.
 237
 238 Reproducible build
 239
 240 Two CVEs found
 241
 242 Share \verb|nbd-protocol.h| file with nbdkit
 243
 244 \item[October 2019]
 245
 246 Systemd socket activation
 247
 248 Fuzz testing (discovers one serious CVE)
 249
 250 New API to connect directly to socket
 251
 252 New \verb|nbdfuse| tool
 253
 254 \end{description}
 255
 256 \end{document}