RISC-V ("Risc Five") is an open source Instruction Set Architecture (ISA). Today I will explain what that means and talk about what we have done to port Fedora and upstream projects to RISC-V. 1. Background - What is RISC-V? ---------------------------------------------------------------------- First we ask: What is an Instruction Set Architecture (ISA)? ISAs are the lowest level at which software talks to the machine. An ISA consists of: - machine code that runs on a CPU - a wide-ranging description of many other aspects of the hardware (Intel's ISA document is over 2000 pages long) Today, ISAs are mainly owned by companies. Intel owns the x86 architecture used in the majority of servers and laptops. ARM owns the ARM architecture used in the majority of phones and tablets. IBM owns two architectures (s/390 and POWER). And there are others but those are the major ones today. Can you make your own x86-compatible chip? No. Companies protect ISAs aggressively with patents, copyrights and trade secrets. Intel and AMD cross-license patents and prevent anyone else from developing x86 chips. [SLIDE: ARM] ARM operates slightly differently: their architecture is also heavily defended with patents, but anyone can come along and license that architecture to make their own chips, but they will pay millions up front plus a few dollars for each chip they sell. [SLIDE: RISC-V LOGO AND FOUNDATION MEMBERS] RISC-V is different: RISC-V is an attempt to create a modern ISA which is *not* covered by patents, and has a liberal open source license (BSD). There are no up front costs or per unit license fees. The only real limitation is use of the RISC-V Trade Mark which requires a fairly modest fee, and doesn't prevent you from making chips as long as you don't use the Trade Mark. RISC-V is run by the RISC-V Foundation (https://riscv.org/) of which IBM is a member (but not Red Hat, yet). There are hundreds of companies who are members of the Foundation. [SLIDE: RISC-V CODE] Let's have a look at some RISC-V machine code. * Most instructions are 32 bits (4 bytes) long. * Compressed instructions are 16 bits. * Some instructions can be longer than 32 bits. * Decoding instruction boundaries is easy, even if you can't or don't fully parse them. [SLIDE: RISC-V REGISTERS] * 32 general purpose registers * 32 floating point registers (extension) * Zero register * Always little-endian * Influenced by MIPS architecture * Proven to be patent-free The architecture is "boring"! It doesn't contain any surprise features. It's is meant to be as independent as possible of the micro-architecture. It's simple to decode, simple to implement (at least if you don't care about performance), and simple to emulate. There are no weird features like register windows or branch delay slots. [SLIDE: RISC-V EXTENSIONS] RISC-V is designed to be extensible. The base architectures are: * RV32I = 32 bit, integer arithmetic, basic instructions like compare, jump etc * RV32E = 32 bit embedded variant, 16 registers, not yet standardized * RV64I = 64 bit + integers * RV128I = 128 bit + integers (not yet standardized) Normally you will add a few extensions. The common ones are: * I = base + integer (this is not an extension, it's required) * M = integer multiplication and division * A = atomic operations * F = floating point * D = double-precision floating point * C = compressed instructions * G = IMAFD Fedora and Debian are targetting RV64GC There are other extensions too. The notable ones are: * B = bit manipulation * J = JIT features * V = vectorization [SLIDE: RISC-V SPECIFICATIONS] There are two important documents: * "User spec" (RISC-V ISM Vol 1) version 2.2 / 20190608 236 pages Covers what userspace code needs to know: User instructions, Memory ordering, Calling conventions, etc. * "Priv spec" (RISC-V ISM Vol 2) version 1.10 / 20190608 109 pages Covers what the kernel needs to know: Priv instructions, Page tables, Machine Status Registers, Interrupts, Timers, etc. These documents are freely downloadable and distributed under a Creative Commons license. [SLIDE: RISC-V OPEN SOURCE IMPLEMENTATIONS] At least 7 open source implementations: * Rocket chip: Simple in-order processor * BOOM and BOOMv2: OOO Superscalar processor * PicoRV32 * Two Western Digital processors * Two educational implementations that I won't talk about today Rocket is the most widely used. It is written in Chisel which is a Scala-based metalanguage that generates Verilog. PicoRV32 is a lot of fun too: It's written in Verilog and can be programmed on to a very cheap (under $40) FPGA. Great for tinkering. [SLIDE: OTHER PARTS OF THE ECOSYSTEM] As well as the CPU core, there are also open source implementations of: * L1 and L2 Cache hierarchy and coherence * ChipLink/TileLink: Communicate between sockets and with peripherals * Interrupt controllers * Ethernet PHYs * Serial ports * DDR DRAM controller So you can (sort of) create your own computer. There are still many missing bits: * High-performance DDR * High-performance ethernet * SATA * BMC management * Standard boot environment (although this is coming together) There are three emulators: * QEMU has supported RV64GC since around 2017. * Spike is the RISC-V Foundation's cycle-accurate simulator * Fabrice Bellard's emulator There are ports of the major tools: * Linux (of course!) since around 2018 * binutils since 2017 * GCC since 2016 * glibc since 2018 2. RISC-V and Fedora, Debian and upstream communities ---------------------------------------------------------------------- I did the first bootstrap of Fedora in September 2016. [SLIDE: BOOTSTRAPPING] Bootstrapping Fedora is complicated: I started off with an x86-64 disk image which could run rpmbuild. I then removed all of the x86-64 binaries and libraries. I replaced them with cross-compiled RISC-V binaries and libraries. About 60 packages had to be cross-compiled and hundreds of binaries were built. After several weeks of work I ended up with a disk image which was bootable under QEMU emulation, which could run rpmbuild. This was called the "stage 3" disk image. The stage 3 disk image allowed me to (very slowly) build RPMs natively. After many more weeks of work I had built enough RPMs by hand so that I could create a disk image from just RPMs. This was called the "stage 4" disk image. Using the stage 4 disk image we were then able to rebuild most of Fedora, successively building newer and newer disk images. Git repo: https://github.com/rwmjones/fedora-riscv-bootstrap Initially we had a custom autobuilder which went through the whole of Fedora blindly attempting to rebuild each package. We spent many months fixing Fedora packages, fixing upstream programs, and manually breaking dependency chains. [SLIDE: SECOND AND THIRD BOOTSTRAPS] Because of changes that kept breaking glibc backwards compatibility, we knew that the first bootstrap wouldn't be the last one. In fact we ended up doing two more bootstraps. The second was a practice one to produce a "script" for bootstrapping Fedora. The third one was done in February 2018 after glibc was released with a permanently stable ABI. The third one should be the last one and is the basis for all current work. [SLIDE: KOJI] Nowadays we have a normal Koji builder which shadows Fedora 29, 30 and Rawhide. [SLIDE: FEDORA RISC-V] You can grab Fedora for RISC-V and run it either on the real hardware or under QEMU on your laptop. [I will show a demo of Fedora RISC-V, showing that it looks very much like Fedora on any other architecture] While we were working on Fedora, we were also working closely with the Debian and upstream communities. Patches and techniques are shared with upstream and with Debian. To get involved with Fedora RISC-V please join us on Freenode: #fedora-riscv 3. Companies making RISC-V hardware ---------------------------------------------------------------------- Initial RISC-V implementations were small-run silicon or FPGAs. It wasn't until 2018 that we started to see companies producing RISC-V processors and boards. It seems that most companies at the moment are looking at the embedded space, and are doing this in order to save on ARM licensing fees. It's still very early days. Note that embedded processors (RV32) will *not* run Fedora or Debian, and some won't even run Linux. [SLIDE: SIFIVE] The RISC-V single board computer you can buy today is made by SiFive, and it's called the *HiFive Unleashed*. I have two of these, and we have 4 or 5 in total across Red Hat. This uses the Rocket chip design, with 4 cores. It's mostly open source. SiFive also have a 32 bit embedded single board computer. [SLIDE: OTHERS] * SHAKTI * Andes N25 and NX25 * Kendryte * Codasip * Syntacore * Nvidia * Western Digital * lowRISC * PULPino * Esperanto * Adapteva 4. RISC-V on the Server ---------------------------------------------------------------------- [This section will be a more free-form discussion about servers and whether RISC-V will ever capture any significant share of the marketplace. Here I just outline topics for discussion.] * What is a server? What is the difference between a server and a laptop? What specialized components go into a server? - NUMA and interrupt routing - headless remote management and BMC - very high speed network interfaces, RDMA - separation of storage from compute, SANs, iSCSI, FC - GPGPU, TPU, custom hardware - hardware offload and accelerators - virtualization, NFV, SR-IOV, ... - fencing * How is server hardware deployed? Single servers versus datacenters. * How is server software provisioned? - mass provisioning of nodes (eg. with Ironic) - orchestration and configuration management * Single kernel image works everywhere. * Standard boot method. * Hardware discovery. * Reliability - distributed computing, fast fail-over, containers, etc. - mainframe-style redundancy