- Spectrum Development - spectrum-os.org

[PATCH] rootfs: support default package list extension
by Ivan Nikolaenko 09 Jan '23

09 Jan '23

This patch will allow users to add their packages to the Spectrum OS rootfs. To do so, the user needs to declare a variable "additionalPackages" in his overlay with the list of required packages. Please, note that static packages need to be taken from pkgsStatic list, since additionalPackages are added from pkgs list. Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- host/rootfs/default.nix | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..37c820b 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -75,7 +75,8 @@ let CONFIG_RMMOD n ''; }) - ] ++ (with pkgsGui; [ foot westonLite ]); + ] ++ (with pkgsGui; [ foot westonLite ]) + ++ (pkgs.additionalPackages or []); nixosAllHardware = nixos ({ modulesPath, ... }: { imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; -- 2.34.1

1 0

[PATCH] Documentation: Architecture Decision Record
by Ville Ilvonen 06 Jan '23

06 Jan '23

* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- Documentation/architecture.adoc | 13 ++++++++++ ...architecture-decision-record-template.adoc | 13 ++++++++++ .../decisions/001-host-update-mechanism.adoc | 19 ++++++++++++++ .../decisions/002-install-options.adoc | 18 +++++++++++++ Documentation/decisions/003-partitioning.adoc | 25 +++++++++++++++++++ .../004-data-at-rest-encryption.adoc | 16 ++++++++++++ .../005-virtual-machine-manager.adoc | 24 ++++++++++++++++++ .../decisions/006-drivers-on-host.adoc | 17 +++++++++++++ .../decisions/007-USB-virtual-machine.adoc | 14 +++++++++++ ...008-Inter-VM-communication-mechanisms.adoc | 18 +++++++++++++ 10 files changed, 177 insertions(+) create mode 100644 Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc create mode 100644 Documentation/decisions/001-host-update-mechanism.adoc create mode 100644 Documentation/decisions/002-install-options.adoc create mode 100644 Documentation/decisions/003-partitioning.adoc create mode 100644 Documentation/decisions/004-data-at-rest-encryption.adoc create mode 100644 Documentation/decisions/005-virtual-machine-manager.adoc create mode 100644 Documentation/decisions/006-drivers-on-host.adoc create mode 100644 Documentation/decisions/007-USB-virtual-machine.adoc create mode 100644 Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc index 157907f..185740c 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/architecture.adoc @@ -20,6 +20,19 @@ devices and provides network services to application VMs). Refer to xref:creating-vms.adoc[Creating VMs] and xref:running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. +== Architecture Decision Record (ADR) + +https://adr.github.io/[Architecturally significant decisions] are +recorded as https://github.com/joelparkerhenderson/architecture-decision-record/blob/ma… ADRs] + +Status of Spectrum OS ADRs: +Accepted - Implemented and likely not to change. +Proposed - Designed and possibly partially implmented. May change. +Other - Not yet in use. + +Comments and contributions to ADRs are welcome. ADRs can be found at +Documentation/decisions + == The Spectrum host system Compartmentalization is implemented using diff --git a/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc new file mode 100644 index 0000000..087ec44 --- /dev/null +++ b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc @@ -0,0 +1,13 @@ +# Title + +## Status +What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.? + +## Context +What is the issue that we're seeing that is motivating this decision or change? + +## Decision +What is the change that we're proposing and/or doing? + +## Consequences +What becomes easier or more difficult to do because of this change? diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc new file mode 100644 index 0000000..03bbae2 --- /dev/null +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -0,0 +1,19 @@ +# Host update mechanism + +## Status +Proposed + +## Context +Spectrum OS has no implementation for software update. The host - consisting of +Linux kernel, KVM, cloud-hypervisor and minimal user space tools - software +updates are required to support feature development and security fixes. + +## Decision +A-B partitioning created by Spectrum installer Installer sets up the system on +partition A of the block device A-B update scheme where user (or installer) +writes the update image to partition B Bootloader provides four boot options: +A, A mutable, B, B mutable + +## Consequences +Default boot selection, incremental updates (e.g. overlays), network update +postponed for later. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc new file mode 100644 index 0000000..f5857c5 --- /dev/null +++ b/Documentation/decisions/002-install-options.adoc @@ -0,0 +1,18 @@ +# Install options + +## Status +Proposed + +## Context +Based on identified different audiences for the Spectrum OS release it is +proposed we support three base configurations to use with Spectrum OS in the +first boot. + +## Decision +* Minimal - Spectrum OS host + system VMs: netvm, guivm, usbvm + home-directory +(optionally encrypted - see 004-disk-encryption.md) +* Common - Minimal + browser app VM + 2-3 selected app VMs +* Power - Common + NixOS VM + +## Consequences +Requires first-boot-vm (like wizard) to support user to get started. diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc new file mode 100644 index 0000000..345619f --- /dev/null +++ b/Documentation/decisions/003-partitioning.adoc @@ -0,0 +1,25 @@ +# Partitioning + +## Status +Proposed + +## Context +Partitions are required to install the Spectrum OS, VMs and store user data. + +## Decision +---- +<blockdevice> # EFI system partition +<blockdevice> # XBOOTLDR +<blockdevice> # A +<blockdevice> # B +# first 32 GB are reserved for Spectrum system +# rest of the disk is reserved for user data +<blockdevice>n-1 # bootstrap user data +<blockdevice>n to the end of disk # user data +---- + +## Consequences +LVM may support resizing - both increasing and decreasing with some limitation +when there's alreay data on volume(s). Does LVM work with all disk types? We +have to implement XBOOTLDR to support EFI system partition created by Windows - +to support dual boot diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc new file mode 100644 index 0000000..3ed9abb --- /dev/null +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -0,0 +1,16 @@ +# Data at rest encryption + +## Status +Proposed + +## Context +To support user data and privacy protection, encryption of data at rest is +required. + +## Decision +User data is encrypted. + +## Consequences +Spectrum OS needs to come with enough SW to get the encryption key via different +methods (password, usb, fido, etc.) Can we use dm-crypt for everything instead +of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-manager.adoc b/Documentation/decisions/005-virtual-machine-manager.adoc new file mode 100644 index 0000000..b4af595 --- /dev/null +++ b/Documentation/decisions/005-virtual-machine-manager.adoc @@ -0,0 +1,24 @@ +# Virtual Machine Manager + +## Status +Accepted + +## Context +rust-vmm-based VMM provides memory and concurrency safe solution. +cloud-hypervisor was chosen because firecrack does not support other +virtio-devices than net or block. crosvm was not chosen because cloud-hypervisor +has more flexible IPC mechanisms, more engaging community as LF-project. +cloud-hypervisor has more core features - such as snapshotting, live migration +and more general hot plugging. crosvm supports more devices we will also need. +It was seen easier to port devices from crosvm to cloud-hypervisor than to port +core features from cloud-hypervisor to crosvm. + +## Decision +Spectrum OS design and implementation decision is to use cloud-hypervisor as the +primary VMM. + +## Consequences +We gotta port some stuff from crosvm to cloud-hypervisor. It's easier for +Spectrum to handle virtualization dynamically with cloud-hypervisor. If the +primary VMM, cloud-hypervisor, is exchanged for trials etc. functionality is +expected to break or not supported. diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc new file mode 100644 index 0000000..052b596 --- /dev/null +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -0,0 +1,17 @@ +# Drivers on host + +## Status +Accepted + +## Context +To harden the trusted computing base and make it more minimal, the target is to +minimize the amount of drivers on the Spectrum host kernel. + +## Decision +We are aiming to have as few drivers as possible on the host. + +## Consequences +No networking on the host. Responsibilities of the host are expected to get +smaller over time. More flexible management of devices. We need to decouple +device classes - like net, usb, bluetooth and gui - from host to their +respective VMs. diff --git a/Documentation/decisions/007-USB-virtual-machine.adoc b/Documentation/decisions/007-USB-virtual-machine.adoc new file mode 100644 index 0000000..2072427 --- /dev/null +++ b/Documentation/decisions/007-USB-virtual-machine.adoc @@ -0,0 +1,14 @@ +# USB Virtual Machine + +## Status +Proposed + +## Context +To support specific USB devices on specific VMs + +## Decision +The decision is to pass-through USB controller to a VM with authorization +controls inside the VMs to forward a specific USB device using USBIP. + +## Consequences +We need to modify the upstream USBIP daemon to support authorization. diff --git a/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc new file mode 100644 index 0000000..1007037 --- /dev/null +++ b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc @@ -0,0 +1,18 @@ +### Inter VM communication mechanisms + +### Status +Proposed + +### Context +Guest VM software needs to communicate with software in other guest VMs + +### Decision +Spectrum provides two mechanism +- TCP/IP with virtio-net +- Wayland with virtio-gpu (nevermind the semantics) for streamed IPC protocol to + send references to shared memory + + +### Consequences +- Examples required on how to write applications which communicate over + virtio-gpu -- 2.36.0

4 9

Overriding Linux kernel in host/rootfs/default.nix
by Ivan Nikolaenko 21 Dec '22

21 Dec '22

Hello everyone. Recently I faced an issue with overriding Linux kernel in rootfs recipe: Rootfs has an attribute kernel in the `let ... in` section, which is used for setting up correct paths to kernel modules in rootfs derivation. But this attribute is also used by the derivation `packagesSysroot` which is used by the derivation `packagesTar`, whiсh, in its turn, used by rootfs derivation. All that derivations introduced in the scope of rootfs derivation (the same `let ... in` section) and are anonymous. What happens when I override kernel attribute of the rootfs derivation: 1. Nix evaluates original host/rootfs/default.nix file and replaces all ${kernel} occurences with the actual value of kernel attribute (which is linux_latest at this moment). 2. Nix sets linux_latest as the input for rootfs and packagesSysroot derivations. 3. Nix evaluates overlay with kernel override and overrides (in fact, introduces new) kernel attribute. 4. Nothing happens after that. Now we have: - `kernel` attribute of rootfs derivation pointing to the imx8 kernel. - `MODULES_ALIAS` and `MODULES_ORDER` variables of rootfs derivation are pointing to linux_latest modules. The root of the issue - line `ln -s ${kernel}/lib/modules ${firmware}/lib/firmware $out/lib` from `packagesSysroot` derivation build command, which creates the /lib link in the rootfs pointing to linux_latest modules. As I found, kernel attribure is not even passed as an attribute to the rootfs derivation file. But it is possible to override MODULES_ALIAS and MODULES_ORDER with correct kernel modules path instead. This also touches initramfs The possible solution is just to override linux_latest, but in this case all VMs will use host kernel version, and we need to avoid this. I would propose something like this for rootfs' default.nix, because this approach solves all kernel issues in an easy and clear way: diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index ad6ea1f..7bf16d2 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, kernel ? linux_latest }: let @@ -73,8 +73,6 @@ let imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; }); - kernel = pkgs.linux_latest; - appvm = import ../../img/app { inherit config; inherit (foot) terminfo; I played a bit with this, and seems that it worked. Thanks.

2 1

[PATCH 1/2] Add flakes support
by Valentin Kharin 21 Dec '22

21 Dec '22

Signed-off-by: Valentin Kharin <valentin.kharin(a)unikie.com> --- flake.lock | 43 +++++++++++++++++++++++++++++++++ flake.nix | 71 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 114 insertions(+) create mode 100644 flake.lock create mode 100644 flake.nix diff --git a/flake.lock b/flake.lock new file mode 100644 index 0000000..aa4ee5e --- /dev/null +++ b/flake.lock @@ -0,0 +1,43 @@ +{ + "nodes": { + "flake-utils": { + "locked": { + "lastModified": 1667395993, + "narHash": "sha256-nuEHfE/LcWyuSWnS8t12N1wc105Qtau+/OdUAjtQ0rA=", + "owner": "numtide", + "repo": "flake-utils", + "rev": "5aed5285a952e0b949eb3ba02c12fa4fcfef535f", + "type": "github" + }, + "original": { + "owner": "numtide", + "repo": "flake-utils", + "type": "github" + } + }, + "nixpkgs": { + "locked": { + "lastModified": 1669635185, + "narHash": "sha256-vYg6GjnsEWNWt/4TmfFN9WtQmSXb4S796J2UOfyTcW0=", + "ref": "refs/heads/rootfs", + "rev": "3176ddef4b4cec85faa2f49d29ce74816d452dc0", + "revCount": 429673, + "type": "git", + "url": "https://spectrum-os.org/git/nixpkgs/" + }, + "original": { + "ref": "refs/heads/rootfs", + "type": "git", + "url": "https://spectrum-os.org/git/nixpkgs/" + } + }, + "root": { + "inputs": { + "flake-utils": "flake-utils", + "nixpkgs": "nixpkgs" + } + } + }, + "root": "root", + "version": 7 +} diff --git a/flake.nix b/flake.nix new file mode 100644 index 0000000..6e77006 --- /dev/null +++ b/flake.nix @@ -0,0 +1,71 @@ +{ + description = "A compartmentalized operating system"; + + # NOTE: Revision specification format is ?ref=refs%2fheads%2f<BRANCH>&rev=<COMMIT_REVISION> + inputs.nixpkgs.url = + "git+https://spectrum-os.org/git/nixpkgs/?ref=refs%2fheads%2frootfs"; + inputs.flake-utils.url = "github:numtide/flake-utils"; + + outputs = { self, nixpkgs, flake-utils }: + flake-utils.lib.eachDefaultSystem (system: + let + pkgs = nixpkgs.legacyPackages.${system}; + config = { inherit pkgs; }; + lib = pkgs.lib; + + mkEntryPoint = { name ? builtins.baseNameOf path, path + , enableShell ? true, enablePackage ? true }: + let + shell = { + # NOTE: https://stackoverflow.com/a/43850372 + devShells.${name} = + import (path + "/shell.nix") { inherit config; }; + }; + package = { packages.${name} = import path { inherit config; }; }; + in (if enableShell then shell else { }) + // (if enablePackage then package else { }); + + # Entry point is a directory with shell.nix and default.nix + # This function maps every entry point to corresponding devShell and package + mapEntryPoints = epoints: + builtins.foldl' lib.recursiveUpdate { } (map mkEntryPoint epoints); + in lib.recursiveUpdate (mapEntryPoints [ + { + path = ./.; + enablePackage = false; + } + { path = ./host/initramfs; } + { path = ./host/rootfs; } + { path = ./host/start-vm; } + { path = ./img/app; } + { path = ./release/live; } + { path = ./vm/sys/net; } + ]) { + # Add some other flake schema related stuff here. + # NOTE: flake-utils.lib.eachDefaultSystem automagically adds ${system}. + devShells.documentation = import ./Documentation { inherit config; }; + packages.documentation = import ./Documentation { inherit config; }; + + nixosModules = let + substituters = [ "https://cache.dataaturservice.se/spectrum/" ]; + trusted-public-keys = [ + "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" + "spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ=" + ]; + in { + # NOTE: See https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-substit… + # and https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-trusted… + # to understand difference between these two modules. + binary-cache = { ... }: { + nix.settings = { inherit trusted-public-keys substituters; }; + }; + # Doesn't enabled by + trusted-binary-cache = { ... }: { + nix.settings = { + inherit trusted-public-keys; + trusted-substituters = substituters; + }; + }; + }; + }); +} -- 2.38.1

2 3

Attempting to use virtio-gpu on Apple Silicon
by Alyssa Ross 09 Dec '22

09 Dec '22

Just a quick write-up of my most recent attempt to use virtio-gpu on my M2 MacBook Pro, so I (or anybody else) has something to refer to when trying again in future. I decided to try again today because Asahi Linux just released their GPU driver[1]. The result of the experiment was the discovery that virglrenderer requires OpenGL (ES) 3.1[2], but Asahi's driver currently only supports OpenGL (ES) 2. Quoting the release announcement: > What’s next? Supporting more applications. While OpenGL (ES) 2 > suffices for some applications, newer ones (especially games) demand > more OpenGL features. OpenGL (ES) 3 brings with it a slew of new > features, like multiple render targets, multisampling, and transform > feedback. Work on these features is well under way, but they will each > take a great deal of additional development effort, and all are needed > before OpenGL (ES) 3.0 is available. So for us, it sounds like the thing to do is to wait until newer OpenGL (ES) is supported, and then try again. Anyway, here's what I did: First, I installed installed the GPU drivers on my Asahi install following the instructions in the release announcement. Then, I checked out the wayland branches for Spectrum and Spectrum's Nixpkgs, and grabbed the CARGO_TARGET_*_LINKER and PKG_CONFIG_WAYLAND_PROTOCOLS_PKGDATADIR fixes from commit fb60a5c9473b0796738aa1b1443b8ee5e5ecd1cb to fix the crosvm build on ARM. Then I checked the AsahiLinux/PKGBUILDs repository to see where their mesa sources lived, and found them using commit af7db4903008b012bca13558261d552ed0828b7d from https://gitlab.freedesktop.org/asahi/mesa.git. This Mesa tree was more recent than the one in our Wayland branch, so I needed to backport some Nixpkgs changes to be able to build it. I cherry-picked the following commits: 952dbf0a4ae mesa: build more Vulkan drivers on aarch64-linux 906947555d2 mesa: 22.2.2 -> 22.2.3 7a222ffee31 mesa: fix build on darwin 8d7db0597f9 mesa: rm unused attributes aa8d873a4eb mesa: 22.2.1 -> 22.2.2 33a5279d620 mesa: add vulkanLayers option and default it to device-select, overlay b4b99f16da9 mesa: re-enable video-codecs 445887f636e mesa: 22.1.7 -> 22.2.1 dd3fc581129 mesa: Install radv override configs into $out 40ee116f25a spirv-llvm-translator_14: init at 14.0.0 ea2ca5e6394 spirv-llvm-translator: version depends on llvm version And then I applied Nixpkgs PR #204125, which at the time of writing is still a draft. At this point, I was able to build Mesa 22.3, and then swap in the asahi sources. To get the right drivers to build, I set galliumDrivers to ["swrast" "virgl" "kmsro" "asahi"] and enableGalliumNine to false. Finally, I set up /run/opengl-driver to be a symlink to mesa's drivers output, since that path doesn't usually exist on non-NixOS. To debug why it wasn't working, on Puck's advice I ran the VM in the img/app development shell like this: nix-shell --arg run ../../vm/app/hello-wayland.nix \ --run 'make clean && LIBGL_DEBUG=verbose MESA_DEBUG=1 EGL_LOG_LEVEL=debug make -j4 run VMM=crosvm' This produced (among others) the following message: libEGL debug: context api is 0x40 while config supports 0xd Puck looked up the value of these constants, and determined that it means it doesn't find the required OpenGL (ES) version. [1]: https://asahilinux.org/2022/12/gpu-drivers-now-in-asahi-linux/ [2]: https://gitlab.freedesktop.org/virgl/virglrenderer/-/issues/202

2 2

Firefox appVM patches and appVM refactoring
by Vadim Likholetov 07 Dec '22

07 Dec '22

I've made an Firefox appVM for wayland using my waypipe patches. To make this appVM I have had to refactor Spectrum OS appvm infrastructure. The main idea of refactoring is enabling appvm to have a user with normal priviledges, not superuser. Running everything from root is not the best idea for secure OS :-) So now the .nix file for appvm has two sections, one that is executed as root and one as user. Here is the sample of this definitions: { config ? import ../../../nix/eval-config.nix {} }: import ../make-vm.nix { inherit config; } { providers.net = [ "netvm" ]; run = config.pkgs.pkgsStatic.callPackage ( { writeScript }: writeScript "run-root-shell" '' #!/bin/execlineb -P /bin/sh '' ) { }; run-as-user = config.pkgs.pkgsStatic.callPackage ( { writeScript, lynx }: writeScript "run-lynx" '' #!/bin/execlineb -P ${lynx}/bin/lynx https://spectrum-os.org '' ) { }; } Cloud-hypervisor has virtual hardware limitations -- it supports only one console device and only one serial device. SpectrumOS is using serial device for kernel logs of appVM and console device as a console. To have access both to root-executed part and to user-executed part of the VM payload, I installed a tmux on console. Now, when you're running vm-console command you get access to the tmux and have the ability to switch between root and user consoles, that can be useful during debugging VM payload. To run Firefox appVM use vm-start-way command: vm-start-way appvm-firefox :) I beleive that as soon as SpectrumOS features will cover basic user needs it's popularity and community will grow and this will make positive impact on SpectrumOS itself. Using appvm-firefox prototype you may build another wayland-enabled appVMs.

5 17

[PATCH 1/2] Enables forwarding Wayland from appVM to host with waypipe
by vadim likholetov 04 Dec '22

04 Dec '22

Signed-off-by: vadim likholetov <vadim.likholetov(a)unikie.com> --- host/initramfs/extfs.nix | 4 +++- host/rootfs/Makefile | 1 + host/rootfs/default.nix | 4 ++-- host/rootfs/usr/bin/vm-start-way | 10 ++++++++++ img/app/default.nix | 3 +++ vm/app/hello-waypipe.nix | 23 +++++++++++++++++++++++ 6 files changed, 42 insertions(+), 3 deletions(-) create mode 100755 host/rootfs/usr/bin/vm-start-way create mode 100644 vm/app/hello-waypipe.nix diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 5c5850f..f49e519 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -11,12 +11,13 @@ let appvm-catgirl = import ../../vm/app/catgirl.nix { inherit config; }; appvm-lynx = import ../../vm/app/lynx.nix { inherit config; }; + appvm-hello-waypipe = import ../../vm/app/hello-waypipe.nix { inherit config; }; in runCommand "ext.ext4" { nativeBuildInputs = [ e2fsprogs ]; } '' - mkdir -p root/svc/data/appvm-{catgirl,lynx} + mkdir -p root/svc/data/appvm-{catgirl,lynx,hello-waypipe} cd root tar -C ${netvm} -c data | tar -C svc -x @@ -24,6 +25,7 @@ runCommand "ext.ext4" { tar -C ${appvm-catgirl} -c . | tar -C svc/data/appvm-catgirl -x tar -C ${appvm-lynx} -c . | tar -C svc/data/appvm-lynx -x + tar -C ${appvm-hello-waypipe} -c . | tar -C svc/data/appvm-hello-waypipe -x mkfs.ext4 -d . $out 16T resize2fs -M $out diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 9559c06..06e3e8e 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -39,6 +39,7 @@ FILES = \ usr/bin/lsvm \ usr/bin/vm-console \ usr/bin/vm-start \ + usr/bin/vm-start-way \ usr/bin/vm-stop DIRS = dev etc/s6-linux-init/env ext run proc sys diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..de4a6b6 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, e2fsprogs, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, waypipe }: let @@ -49,7 +49,7 @@ let packages = [ cloud-hypervisor e2fsprogs execline jq kmod mdevd s6 s6-linux-init s6-rc - socat start-vm + socat start-vm waypipe (cryptsetup.override { programs = { diff --git a/host/rootfs/usr/bin/vm-start-way b/host/rootfs/usr/bin/vm-start-way new file mode 100755 index 0000000..bda9934 --- /dev/null +++ b/host/rootfs/usr/bin/vm-start-way @@ -0,0 +1,10 @@ +#!/bin/sh +# SPDX-License-Identifier: EUPL-1.2+ + +s6-rc -bu change ext-rc +s6-rc -l /run/s6-rc.ext -u change $1 +ch-remote --api-socket /run/service/ext-${1}/env/cloud-hypervisor.sock add-vsock cid=4,socket=/run/u.${1}.socket +cd /run +nohup waypipe --socket /run/waypipe.sock client & +nohup socat unix-listen:/run/u.${1}.socket_5000,reuseaddr,fork unix-connect:/run/waypipe.sock & + diff --git a/img/app/default.nix b/img/app/default.nix index e7d5366..80f23c2 100644 --- a/img/app/default.nix +++ b/img/app/default.nix @@ -59,6 +59,9 @@ let DRM_BOCHS = yes; DRM = yes; AGP = yes; + VSOCKETS = yes; + VIRTIO_VSOCKETS = yes; + VIRTIO_VSOCKETS_COMMON = yes; }; }; in diff --git a/vm/app/hello-waypipe.nix b/vm/app/hello-waypipe.nix new file mode 100644 index 0000000..601b638 --- /dev/null +++ b/vm/app/hello-waypipe.nix @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> + +{ config ? import ../../../nix/eval-config.nix {} }: + +import ../make-vm.nix { inherit config; } { + providers.net = [ "netvm" ]; + run = config.pkgs.callPackage ( + { writeScript, waypipe, havoc, foot, hello-wayland, socat}: + writeScript "run-waypipe-app" '' + #!/bin/sh + mkdir /run/0 + export XDG_RUNTIME_DIR=/run/0 + ${socat}/bin/socat unix-listen:/run/waypipe.sock,reuseaddr,fork vsock-connect:2:5000 & + sleep 1 + ${waypipe}/bin/waypipe --display wayland-local --socket /run/waypipe.sock server -- sleep inf & + export WAYLAND_DISPLAY=wayland-local + ${havoc}/bin/havoc + ${hello-wayland}/bin/hello-wayland + ${foot}/bin/foot + '' + ) { }; +} -- 2.36.2

1 1

Enabling forwarding Wayland from appVM to host with waypipe
by Vadim Likholetov 30 Nov '22

30 Nov '22

This patchset enables forwarding Wayland connections from appVM to the host with the help of waypipe tool. It doesn't reqire major changes in Spectrum and tries to utilise existing packages and solutions. Main idea is to create VSOCK virtual device on the start of appVM and use this virtual socket to connect host and guest. On the host side I implement vm-start-way tool that is based on vm-start, but does necessary preparations for wayland forwarfing. The guest VM is appvm-hello-waypipe and described in hello-waypipe.nix file. To test it boot the Spectrum and do 'vm-start-way appvm-hello-waypipe'. Be patient and in couple of seconds will appear a terminal window from appVM. This patches are for test the concept, so possibly redisign is needed for production use. ps sorry for mess in the mailing list

2 7

[PATCH] Patches to waypipe package to enable static build with the cost of fancy feature
by vadim likholetov 30 Nov '22

30 Nov '22

Signed-off-by: vadim likholetov <vadim.likholetov(a)unikie.com> --- .../applications/networking/remote/waypipe/default.nix | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/pkgs/applications/networking/remote/waypipe/default.nix b/pkgs/applications/networking/remote/waypipe/default.nix index ec94d504b9d..bcc3770ecf7 100644 --- a/pkgs/applications/networking/remote/waypipe/default.nix +++ b/pkgs/applications/networking/remote/waypipe/default.nix @@ -20,7 +20,15 @@ stdenv.mkDerivation rec { nativeBuildInputs = [ meson ninja pkg-config scdoc ]; buildInputs = [ # Optional dependencies: - mesa lz4 zstd ffmpeg libva + # mesa lz4 zstd ffmpeg libva # TODO: one day recompile with acceleration + lz4 zstd + ]; + + mesonFlags = [ + "-Dwith_video=disabled" + "-Dwith_systemtap=false" + "-Dwith_vaapi=disabled" + "-Dwith_dmabuf=disabled" ]; meta = with lib; { -- 2.36.2

3 2

[PATCH] Enables forwarding Wayland from appVM to host with waypipe
by vadim likholetov 30 Nov '22

30 Nov '22

Signed-off-by: vadim likholetov <vadim.likholetov(a)unikie.com> --- host/initramfs/extfs.nix | 4 +++- host/rootfs/Makefile | 1 + host/rootfs/default.nix | 4 ++-- host/rootfs/usr/bin/vm-start-way | 10 ++++++++++ img/app/default.nix | 3 +++ vm/app/hello-waypipe.nix | 23 +++++++++++++++++++++++ 6 files changed, 42 insertions(+), 3 deletions(-) create mode 100755 host/rootfs/usr/bin/vm-start-way create mode 100644 vm/app/hello-waypipe.nix diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 5c5850f..f49e519 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -11,12 +11,13 @@ let appvm-catgirl = import ../../vm/app/catgirl.nix { inherit config; }; appvm-lynx = import ../../vm/app/lynx.nix { inherit config; }; + appvm-hello-waypipe = import ../../vm/app/hello-waypipe.nix { inherit config; }; in runCommand "ext.ext4" { nativeBuildInputs = [ e2fsprogs ]; } '' - mkdir -p root/svc/data/appvm-{catgirl,lynx} + mkdir -p root/svc/data/appvm-{catgirl,lynx,hello-waypipe} cd root tar -C ${netvm} -c data | tar -C svc -x @@ -24,6 +25,7 @@ runCommand "ext.ext4" { tar -C ${appvm-catgirl} -c . | tar -C svc/data/appvm-catgirl -x tar -C ${appvm-lynx} -c . | tar -C svc/data/appvm-lynx -x + tar -C ${appvm-hello-waypipe} -c . | tar -C svc/data/appvm-hello-waypipe -x mkfs.ext4 -d . $out 16T resize2fs -M $out diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 9559c06..06e3e8e 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -39,6 +39,7 @@ FILES = \ usr/bin/lsvm \ usr/bin/vm-console \ usr/bin/vm-start \ + usr/bin/vm-start-way \ usr/bin/vm-stop DIRS = dev etc/s6-linux-init/env ext run proc sys diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..de4a6b6 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, e2fsprogs, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, waypipe }: let @@ -49,7 +49,7 @@ let packages = [ cloud-hypervisor e2fsprogs execline jq kmod mdevd s6 s6-linux-init s6-rc - socat start-vm + socat start-vm waypipe (cryptsetup.override { programs = { diff --git a/host/rootfs/usr/bin/vm-start-way b/host/rootfs/usr/bin/vm-start-way new file mode 100755 index 0000000..bda9934 --- /dev/null +++ b/host/rootfs/usr/bin/vm-start-way @@ -0,0 +1,10 @@ +#!/bin/sh +# SPDX-License-Identifier: EUPL-1.2+ + +s6-rc -bu change ext-rc +s6-rc -l /run/s6-rc.ext -u change $1 +ch-remote --api-socket /run/service/ext-${1}/env/cloud-hypervisor.sock add-vsock cid=4,socket=/run/u.${1}.socket +cd /run +nohup waypipe --socket /run/waypipe.sock client & +nohup socat unix-listen:/run/u.${1}.socket_5000,reuseaddr,fork unix-connect:/run/waypipe.sock & + diff --git a/img/app/default.nix b/img/app/default.nix index e7d5366..80f23c2 100644 --- a/img/app/default.nix +++ b/img/app/default.nix @@ -59,6 +59,9 @@ let DRM_BOCHS = yes; DRM = yes; AGP = yes; + VSOCKETS = yes; + VIRTIO_VSOCKETS = yes; + VIRTIO_VSOCKETS_COMMON = yes; }; }; in diff --git a/vm/app/hello-waypipe.nix b/vm/app/hello-waypipe.nix new file mode 100644 index 0000000..601b638 --- /dev/null +++ b/vm/app/hello-waypipe.nix @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> + +{ config ? import ../../../nix/eval-config.nix {} }: + +import ../make-vm.nix { inherit config; } { + providers.net = [ "netvm" ]; + run = config.pkgs.callPackage ( + { writeScript, waypipe, havoc, foot, hello-wayland, socat}: + writeScript "run-waypipe-app" '' + #!/bin/sh + mkdir /run/0 + export XDG_RUNTIME_DIR=/run/0 + ${socat}/bin/socat unix-listen:/run/waypipe.sock,reuseaddr,fork vsock-connect:2:5000 & + sleep 1 + ${waypipe}/bin/waypipe --display wayland-local --socket /run/waypipe.sock server -- sleep inf & + export WAYLAND_DISPLAY=wayland-local + ${havoc}/bin/havoc + ${hello-wayland}/bin/hello-wayland + ${foot}/bin/foot + '' + ) { }; +} -- 2.36.2

1 0

[PATCH] Patches to waypipe package to enable static build with the cost of fancy feature
by vadim likholetov 29 Nov '22

29 Nov '22

--- .../applications/networking/remote/waypipe/default.nix | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/pkgs/applications/networking/remote/waypipe/default.nix b/pkgs/applications/networking/remote/waypipe/default.nix index ec94d504b9d..bcc3770ecf7 100644 --- a/pkgs/applications/networking/remote/waypipe/default.nix +++ b/pkgs/applications/networking/remote/waypipe/default.nix @@ -20,7 +20,15 @@ stdenv.mkDerivation rec { nativeBuildInputs = [ meson ninja pkg-config scdoc ]; buildInputs = [ # Optional dependencies: - mesa lz4 zstd ffmpeg libva + # mesa lz4 zstd ffmpeg libva # TODO: one day recompile with acceleration + lz4 zstd + ]; + + mesonFlags = [ + "-Dwith_video=disabled" + "-Dwith_systemtap=false" + "-Dwith_vaapi=disabled" + "-Dwith_dmabuf=disabled" ]; meta = with lib; { -- 2.36.2

1 0

[PATCH] This patchset enables forwarding Wayland connections from appVM to the host with the help of waypipe tool. It doesn't reqire major changes in Spectrum and tries to utilise existing packages and solutions. Main idea is to create VSOCK virtual device on the start of appVM and use this virtual socket to connect host and guest. On the host side I implement vm-start-way tool that is based on vm-start, but does nesessary preparations for wayland forwarfing. The guest VM is appvm-hello-waypipe and describe
by vadim likholetov 29 Nov '22

29 Nov '22

--- host/initramfs/extfs.nix | 4 +++- host/rootfs/Makefile | 1 + host/rootfs/default.nix | 4 ++-- host/rootfs/usr/bin/vm-start-way | 10 ++++++++++ img/app/default.nix | 3 +++ vm/app/hello-waypipe.nix | 23 +++++++++++++++++++++++ 6 files changed, 42 insertions(+), 3 deletions(-) create mode 100755 host/rootfs/usr/bin/vm-start-way create mode 100644 vm/app/hello-waypipe.nix diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 5c5850f..f49e519 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -11,12 +11,13 @@ let appvm-catgirl = import ../../vm/app/catgirl.nix { inherit config; }; appvm-lynx = import ../../vm/app/lynx.nix { inherit config; }; + appvm-hello-waypipe = import ../../vm/app/hello-waypipe.nix { inherit config; }; in runCommand "ext.ext4" { nativeBuildInputs = [ e2fsprogs ]; } '' - mkdir -p root/svc/data/appvm-{catgirl,lynx} + mkdir -p root/svc/data/appvm-{catgirl,lynx,hello-waypipe} cd root tar -C ${netvm} -c data | tar -C svc -x @@ -24,6 +25,7 @@ runCommand "ext.ext4" { tar -C ${appvm-catgirl} -c . | tar -C svc/data/appvm-catgirl -x tar -C ${appvm-lynx} -c . | tar -C svc/data/appvm-lynx -x + tar -C ${appvm-hello-waypipe} -c . | tar -C svc/data/appvm-hello-waypipe -x mkfs.ext4 -d . $out 16T resize2fs -M $out diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 9559c06..06e3e8e 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -39,6 +39,7 @@ FILES = \ usr/bin/lsvm \ usr/bin/vm-console \ usr/bin/vm-start \ + usr/bin/vm-start-way \ usr/bin/vm-stop DIRS = dev etc/s6-linux-init/env ext run proc sys diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0a84f55..de4a6b6 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, e2fsprogs, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, waypipe }: let @@ -49,7 +49,7 @@ let packages = [ cloud-hypervisor e2fsprogs execline jq kmod mdevd s6 s6-linux-init s6-rc - socat start-vm + socat start-vm waypipe (cryptsetup.override { programs = { diff --git a/host/rootfs/usr/bin/vm-start-way b/host/rootfs/usr/bin/vm-start-way new file mode 100755 index 0000000..bda9934 --- /dev/null +++ b/host/rootfs/usr/bin/vm-start-way @@ -0,0 +1,10 @@ +#!/bin/sh +# SPDX-License-Identifier: EUPL-1.2+ + +s6-rc -bu change ext-rc +s6-rc -l /run/s6-rc.ext -u change $1 +ch-remote --api-socket /run/service/ext-${1}/env/cloud-hypervisor.sock add-vsock cid=4,socket=/run/u.${1}.socket +cd /run +nohup waypipe --socket /run/waypipe.sock client & +nohup socat unix-listen:/run/u.${1}.socket_5000,reuseaddr,fork unix-connect:/run/waypipe.sock & + diff --git a/img/app/default.nix b/img/app/default.nix index e7d5366..80f23c2 100644 --- a/img/app/default.nix +++ b/img/app/default.nix @@ -59,6 +59,9 @@ let DRM_BOCHS = yes; DRM = yes; AGP = yes; + VSOCKETS = yes; + VIRTIO_VSOCKETS = yes; + VIRTIO_VSOCKETS_COMMON = yes; }; }; in diff --git a/vm/app/hello-waypipe.nix b/vm/app/hello-waypipe.nix new file mode 100644 index 0000000..601b638 --- /dev/null +++ b/vm/app/hello-waypipe.nix @@ -0,0 +1,23 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> + +{ config ? import ../../../nix/eval-config.nix {} }: + +import ../make-vm.nix { inherit config; } { + providers.net = [ "netvm" ]; + run = config.pkgs.callPackage ( + { writeScript, waypipe, havoc, foot, hello-wayland, socat}: + writeScript "run-waypipe-app" '' + #!/bin/sh + mkdir /run/0 + export XDG_RUNTIME_DIR=/run/0 + ${socat}/bin/socat unix-listen:/run/waypipe.sock,reuseaddr,fork vsock-connect:2:5000 & + sleep 1 + ${waypipe}/bin/waypipe --display wayland-local --socket /run/waypipe.sock server -- sleep inf & + export WAYLAND_DISPLAY=wayland-local + ${havoc}/bin/havoc + ${hello-wayland}/bin/hello-wayland + ${foot}/bin/foot + '' + ) { }; +} -- 2.36.2

1 0

[PATCH] Documentation: add ADR for VM Sources
by Alyssa Ross 29 Nov '22

29 Nov '22

Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> --- Documentation/decisions/009-vm-sources.adoc | 42 +++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 Documentation/decisions/009-vm-sources.adoc diff --git a/Documentation/decisions/009-vm-sources.adoc b/Documentation/decisions/009-vm-sources.adoc new file mode 100644 index 0000000..457491d --- /dev/null +++ b/Documentation/decisions/009-vm-sources.adoc @@ -0,0 +1,42 @@ += 009 VM Sources +:page-parent: Architecture Decision Records +:page-grand_parent: About Spectrum + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +== Status + +Proposed + +== Context + +It should be possible for Spectrum VMs to come from the root +filesystem (for built-in VMs), or one or more +xref:../appendices/user-partition.adoc[user partitions]. One part of +making this work will be to have the system search multiple sources of +VM definitions, to produce a single view of what VMs are available. + +== Decision + +The system will scan /usr/img/builtin, as well as the svc/data +partition on each user partition, for VM definitions. A single s6-rc +service directory will contain generated services for VMs from all +sources, so that tools like vm-start don't have to be aware of where +VMs come from. (This will continue to be distinct from the s6-rc +database used for host system services.) When a new source of VMs is +discovered, an updated database will be generated and atomically +replace the old one using +xref:https://www.skarnet.org/software/s6-rc/s6-rc-update.html[s6-rc-update]. + +Changes to how user partitions will be discovered and mounted, to +support having multiple user partitions, will be determined later. + +== Consequences + +Built-in VMs can be moved into the root filesystem, allowing the +default user partition to be empty. This will make it easier for +people making custom Spectrum images to add custom VMs without having +to build Spectrum itself themselves -- they can simply replace the +default, empty, partition with a custom one they can produce however +they like. base-commit: 95fba8ac6c872c2ba5e2ea8be5da9f2eb63374c8 -- 2.35.1

1 0

Waypipe Wayland guest to host forwarding prototype
by Vadim Likholetov 23 Nov '22

23 Nov '22

Hi, I’ve built a prototype of forwarding Wayland client connections with the help of waypipe. It is broken the same way same way as current Alyssa’s implementation. On imx8 green cat is ok, foot don’t start with no clear diagnostics. Is this prototype have interest in community? Should I submit patches of this prototype?

2 1

[PATCH] Documentation: move Appendices to Development, fixed links
by Jenni Nikolaenko 23 Nov '22

23 Nov '22

Both sections "User Partition" and "UUID Reference" are more development topics, so by Alyssa's advice they should go directly under the Development chapter. The section Appendices was removed. In addition, I replaced broken links. Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/about/architecture.adoc | 6 +++--- Documentation/appendices/index.adoc | 13 ------------- .../{appendices => development}/user-partition.adoc | 6 +++--- .../{appendices => development}/uuid-reference.adoc | 11 +++++------ Documentation/index.adoc | 8 ++++---- Documentation/using-spectrum/creating-vms.adoc | 2 +- 6 files changed, 16 insertions(+), 30 deletions(-) delete mode 100644 Documentation/appendices/index.adoc rename Documentation/{appendices => development}/user-partition.adoc (87%) rename Documentation/{appendices => development}/uuid-reference.adoc (85%) diff --git a/Documentation/about/architecture.adoc b/Documentation/about/architecture.adoc index b311e77..38e0a22 100644 --- a/Documentation/about/architecture.adoc +++ b/Documentation/about/architecture.adoc @@ -15,8 +15,8 @@ The default set of virtual machines includes two application VMs, _appvm-catgirl_ (an IRC client) and _appvm-lynx_ (a text-based web browser); and a system VM, _netvm_ (which handles hardware network devices and provides network services to application VMs). -Refer to xref:../getting-started/creating-vms.adoc[Creating VMs] and -xref:../getting-started/running-vms.adoc[Running +Refer to xref:../using-spectrum/creating-vms.adoc[Creating VMs] and +xref:../using-spectrum/running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. == Architecture Decision Records (ADRs) @@ -28,7 +28,7 @@ https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions[AD…. == Spectrum Host System Compartmentalization is implemented using -https://cloud-hypervisor.org/[cloud-hypervisor] virtual machines. +https://cloudhypervisor.org/[cloud-hypervisor] virtual machines. cloud-hypervisor uses Linux's https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[Kernel-based Virtual Machine] (KVM) to provide lightweight, hardware-accelerated VMs. diff --git a/Documentation/appendices/index.adoc b/Documentation/appendices/index.adoc deleted file mode 100644 index ebacda7..0000000 --- a/Documentation/appendices/index.adoc +++ /dev/null @@ -1,13 +0,0 @@ -= Appendices -:description: Additional information. -:page-nav_order: 5 -:page-has_children: true -:page-has_toc: false - -// SPDX-FileCopyrightText: 2022 Unikie -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -This section provides additional information on how to work with: - -* xref:../appendices/user-partition.adoc[user partition] -* xref:../appendices/uuid-reference.adoc[universally unique identifier (UUID)]. diff --git a/Documentation/appendices/user-partition.adoc b/Documentation/development/user-partition.adoc similarity index 87% rename from Documentation/appendices/user-partition.adoc rename to Documentation/development/user-partition.adoc index 4121340..d8fd956 100644 --- a/Documentation/appendices/user-partition.adoc +++ b/Documentation/development/user-partition.adoc @@ -1,6 +1,6 @@ -= Appendix A. User Partition -:page-parent: Appendices -:page-nav_order: 1 += User Partition +:page-parent: Development +:page-nav_order: 5 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/appendices/uuid-reference.adoc b/Documentation/development/uuid-reference.adoc similarity index 85% rename from Documentation/appendices/uuid-reference.adoc rename to Documentation/development/uuid-reference.adoc index 5099ec3..81300ee 100644 --- a/Documentation/appendices/uuid-reference.adoc +++ b/Documentation/development/uuid-reference.adoc @@ -1,7 +1,6 @@ -= Appendix B. UUID Reference -:page-parent: Appendices -:toclevels: 1 -:page-nav_order: 2 += UUID Reference +:page-parent: Development +:page-nav_order: 6 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -25,13 +24,13 @@ Specification]. === `4f68bce3-e8cd-4db1-96e7-fbcaf984b709` "Root Partition (amd64/x86_64)" from the -https://systemd.io/DISCOVERABLE_PARTITIONS/[Discoverable Partitions +https://uapi-group.org/specifications/specs/discoverable_partitions_specification/[Discoverable Partitions Specification]. === `2c7357ed-ebd2-46d9-aec1-23d437ec2bf5` "Root Verity Partition (amd64/x86_64)" from the -https://systemd.io/DISCOVERABLE_PARTITIONS/[Discoverable Partitions +https://uapi-group.org/specifications/specs/discoverable_partitions_specification/[Discoverable Partitions Specification]. === `9293e1ff-cee4-4658-88be-898ec863944f` diff --git a/Documentation/index.adoc b/Documentation/index.adoc index e0a45c7..51a6e7d 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -15,7 +15,7 @@ lower barrier to entry and is easy to use and maintain. To learn about what Spectrum is and how it's implemented, start with the xref:about/architecture.adoc[architecture overview]. -If you want to try Spectrum, see xref:../installation/index.adoc[Build and Run] +If you want to try Spectrum, see xref:installation/index.adoc[Build and Run] to setup a development environment. @@ -25,8 +25,8 @@ Spectrum is made of free and open-source software. It is free for anyone to use, modify, and distribute. Once you are up and running, see - xref:../development/index.adoc[Development] to understand how to work with + xref:development/index.adoc[Development] to understand how to work with Spectrum's code and participate in the collaborative development process. -If you are thinking of contributing to Spectrum docs, see - xref:../development/building-documentation.adoc[Building Documentation]. +If you are thinking of contributing to the Spectrum docs, see + xref:development/building-documentation.adoc[Building Documentation]. diff --git a/Documentation/using-spectrum/creating-vms.adoc b/Documentation/using-spectrum/creating-vms.adoc index 6847249..2c57562 100644 --- a/Documentation/using-spectrum/creating-vms.adoc +++ b/Documentation/using-spectrum/creating-vms.adoc @@ -8,7 +8,7 @@ == Configuration Each VM is configured using a directory under svc/data/ on the -xref:user-partition.adoc[user partition]. The name of the directory +xref:../development/user-partition.adoc[user partition]. The name of the directory determines the name of the VM. The directory can contain the following files: -- 2.34.1

3 2

[PATCH v2 1/2] Documentation/scripts: fix shellcheck issues
by Alyssa Ross 18 Nov '22

18 Nov '22

There's an open issue about teaching shellcheck to automatically recognize nix-shell shebangs: https://github.com/koalaman/shellcheck/issues/1210 Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> --- v2: no changes Documentation/scripts/build.sh | 2 +- Documentation/scripts/update-gems.sh | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/Documentation/scripts/build.sh b/Documentation/scripts/build.sh index 85bd5a5..e6a2cd8 100755 --- a/Documentation/scripts/build.sh +++ b/Documentation/scripts/build.sh @@ -4,7 +4,7 @@ cd "$(dirname "$0")/.." -if [ ! -w . -a ! -w .jekyll-cache ]; then +if [ ! -w . ] && [ ! -w .jekyll-cache ]; then JEKYLLFLAGS=--disable-disk-cache fi diff --git a/Documentation/scripts/update-gems.sh b/Documentation/scripts/update-gems.sh index 7ba98d8..d70f55f 100755 --- a/Documentation/scripts/update-gems.sh +++ b/Documentation/scripts/update-gems.sh @@ -1,8 +1,11 @@ #! /usr/bin/env nix-shell #! nix-shell -i bash -p bundler bundix + # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ +# shellcheck shell=bash + set -euo pipefail bundle lock --update -- 2.35.1

3 4

[PATCH] Documentation: rename "UUID" to "UUID Reference"
by Alyssa Ross 18 Nov '22

18 Nov '22

The purpose of this document is specifically to provide a reference table so that developers working with Spectrum can easily look up a given UUID to find out what it means. I think "UUID Reference" makes it clearer what the reader will find in this document, as I'd expect a document just called "UUID" to be something that explained e.g. what a UUID is. (I don't think we need to explain that at all in the Spectrum documentation, as we link to Wikipedia, which explains it well, and it's background knowledge that most developers looking to work with Spectrum will likely already have.) Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> Cc: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/appendices/uuid-reference.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/appendices/uuid-reference.adoc b/Documentation/appendices/uuid-reference.adoc index 146c068..5099ec3 100644 --- a/Documentation/appendices/uuid-reference.adoc +++ b/Documentation/appendices/uuid-reference.adoc @@ -1,4 +1,4 @@ -= Appendix B. UUID += Appendix B. UUID Reference :page-parent: Appendices :toclevels: 1 :page-nav_order: 2 base-commit: 395346bd3341fd0468bc4f8a9c427440c5d05438 -- 2.35.1

3 2

[PATCH] Documentation: rename "Getting Started" to "Using Spectrum"
by Alyssa Ross 18 Nov '22

18 Nov '22

I clicked on "Getting Started" by mistake expecting to find the documentation about obtaining and installing a Spectrum image, but it actually contains documentation that explains how to use the system once it's installed. So I think it would be clearer if it were named "Using Spectrum". Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> Cc: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- .../{getting-started => using-spectrum}/creating-vms.adoc | 2 +- .../{getting-started => using-spectrum}/index.adoc | 6 +++--- .../{getting-started => using-spectrum}/running-vms.adoc | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) rename Documentation/{getting-started => using-spectrum}/creating-vms.adoc (98%) rename Documentation/{getting-started => using-spectrum}/index.adoc (65%) rename Documentation/{getting-started => using-spectrum}/running-vms.adoc (93%) diff --git a/Documentation/getting-started/creating-vms.adoc b/Documentation/using-spectrum/creating-vms.adoc similarity index 98% rename from Documentation/getting-started/creating-vms.adoc rename to Documentation/using-spectrum/creating-vms.adoc index a2ad952..6847249 100644 --- a/Documentation/getting-started/creating-vms.adoc +++ b/Documentation/using-spectrum/creating-vms.adoc @@ -1,5 +1,5 @@ = Creating VMs -:page-parent: Getting Started +:page-parent: Using Spectrum :page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> diff --git a/Documentation/getting-started/index.adoc b/Documentation/using-spectrum/index.adoc similarity index 65% rename from Documentation/getting-started/index.adoc rename to Documentation/using-spectrum/index.adoc index 10dde7f..5ee5043 100644 --- a/Documentation/getting-started/index.adoc +++ b/Documentation/using-spectrum/index.adoc @@ -1,4 +1,4 @@ -= Getting Started += Using Spectrum :description: Exploring Spectrum OS. Using (=How-To-Guides), Configuring (adding smth). :page-nav_order: 3 :page-has_children: true @@ -9,5 +9,5 @@ Ready to get started with Spectrum? Here is what you can do next: -* xref:../getting-started/creating-vms.adoc[Create your own VM] to use others applications. -* xref:../getting-started/running-vms.adoc[Start some applications]. +* xref:creating-vms.adoc[Create your own VM] to use others applications. +* xref:running-vms.adoc[Start some applications]. diff --git a/Documentation/getting-started/running-vms.adoc b/Documentation/using-spectrum/running-vms.adoc similarity index 93% rename from Documentation/getting-started/running-vms.adoc rename to Documentation/using-spectrum/running-vms.adoc index ca70835..518b00e 100644 --- a/Documentation/getting-started/running-vms.adoc +++ b/Documentation/using-spectrum/running-vms.adoc @@ -1,5 +1,5 @@ = Running VMs -:page-parent: Getting Started +:page-parent: Using Spectrum :page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> base-commit: 395346bd3341fd0468bc4f8a9c427440c5d05438 -- 2.35.1

3 2

[PATCH 1/2] Documentation/scripts: fix shellcheck issues
by Alyssa Ross 16 Nov '22

16 Nov '22

There's an open issue about teaching shellcheck to automatically recognize nix-shell shebangs: https://github.com/koalaman/shellcheck/issues/1210 Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> --- Documentation/scripts/build.sh | 2 +- Documentation/scripts/update-gems.sh | 3 +++ 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/Documentation/scripts/build.sh b/Documentation/scripts/build.sh index 85bd5a5..e6a2cd8 100755 --- a/Documentation/scripts/build.sh +++ b/Documentation/scripts/build.sh @@ -4,7 +4,7 @@ cd "$(dirname "$0")/.." -if [ ! -w . -a ! -w .jekyll-cache ]; then +if [ ! -w . ] && [ ! -w .jekyll-cache ]; then JEKYLLFLAGS=--disable-disk-cache fi diff --git a/Documentation/scripts/update-gems.sh b/Documentation/scripts/update-gems.sh index 7ba98d8..d70f55f 100755 --- a/Documentation/scripts/update-gems.sh +++ b/Documentation/scripts/update-gems.sh @@ -1,8 +1,11 @@ #! /usr/bin/env nix-shell #! nix-shell -i bash -p bundler bundix + # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ +# shellcheck shell=bash + set -euo pipefail bundle lock --update -- 2.35.1

2 3

[PATCH v2 0/6] Introduce a shared base for application VMs
by Alyssa Ross 14 Nov '22

14 Nov '22

This series was originally developed for some work I'm finishing up now for NLnet, for letting Spectrum users build VMs on the system with Nix, so it's time for it to get another round. Changes since v1: • make-vm.nix only generates the VM's configuration directory, not a whole data/$name hierarchy that needs to be merged. • vm-lib/make-vm.nix and vm/make-vm.nix are separated, so system-provided VMs can deduplicate against the base image, while user-defined VMs can't so they're independently upgradeable. v1: https://spectrum-os.org/lists/archives/spectrum-devel/20220919073659.170327… The idea here is to reduce duplication between application VMs, both in terms of source code size and output size. After this change, creating a new VM just requires writing a very small Nix file like this: { config ? import ../../../nix/eval-config.nix {} }: import ../../vm-lib/make-vm.nix { inherit config; } { name = "appvm-lynx"; providers.net = [ "netvm" ]; run = config.pkgs.pkgsStatic.callPackage ( { writeScript, lynx }: writeScript "run-lynx" '' #!/bin/execlineb -P ${lynx}/bin/lynx https://spectrum-os.org '' ) { }; } Rather than a whole big source tree as before, most of which was duplicated with every other application VM. When a VM generated this way is started, it gets two disk images. One is the shared base image, which is part of the Spectrum base system, and the other contains only the application-specific stuff: the run script, and any store path dependencies that are not already present in the base image. This means that the amount of storage required for each new application VM is substantially reduced. Of course, this isn't the only way to generate VMs. Monolithic VMs like we had before would still work, with some small adjustments for the new disk layout. I also see this fitting well into making it possible to configure extra VMs at build time. It doesn't directly help with that, but making it so that each VM doesn't need to provide everything itself will make creating external VMs easier when it does happen. In future we might want to apply a similar mechanism to service VMs, like netvm, but since we only have one of those so far, it's not clear which parts exactly would be duplicated, so I'm leaving it for now. Other future work is considering the impacts of the shared base image on guest isolation. Can guests observe whether reads of the shared base image hit the host page cache, or even an internal disk cache? At the moment I suspect that the base image doesn't have enough specialised code in it that there would be any interesting results, but it's worth thinking about if the shared image grows new functionality, whether it would be interesting to another guest to be able to observe whether those resources have previously been loaded or not. If this _does_ turn out to be a concern, it could be mitigated by simply copying the base image to temporary storage before booting a VM, and then booting the VM from the copy. Alyssa Ross (6): host/start-vm: support multiple block devices scripts/make-gpt.sh: add support for labels vm: build GPT images host/start-vm: boot using partition label release: rename from "img" img/app: extract from appvm-{lynx,catgirl} Documentation/creating-vms.adoc | 8 +- Documentation/getting-spectrum.adoc | 2 +- host/initramfs/extfs.nix | 19 +-- host/rootfs/default.nix | 11 +- host/start-vm/lib.rs | 38 +++++- host/start-vm/tests/vm_command-basic.rs | 6 +- {vm/app/lynx => img/app}/Makefile | 57 ++++---- {vm/app/catgirl => img/app}/bin | 0 {vm/app/lynx => img/app}/default.nix | 22 ++-- img/app/etc/fstab | 8 ++ {vm/app/catgirl => img/app}/etc/init | 0 {vm/app/catgirl => img/app}/etc/mdev.conf | 0 {vm/app/lynx => img/app}/etc/mdev/iface | 2 +- {vm/app/catgirl => img/app}/etc/passwd | 0 .../catgirl => img/app}/etc/passwd.license | 0 {vm/app/catgirl => img/app}/etc/resolv.conf | 0 .../app}/etc/s6-linux-init/scripts/rc.init | 1 + .../s6-rc/lynx => img/app/etc/s6-rc/app}/run | 3 +- .../catgirl => img/app/etc/s6-rc/app}/type | 0 .../app/etc/s6-rc/app}/type.license | 0 .../etc/s6-rc/mdevd-coldplug/dependencies | 0 .../app}/etc/s6-rc/mdevd-coldplug/type | 0 .../etc/s6-rc/mdevd-coldplug/type.license | 0 .../app}/etc/s6-rc/mdevd-coldplug/up | 0 .../app}/etc/s6-rc/mdevd/notification-fd | 0 .../etc/s6-rc/mdevd/notification-fd.license | 0 .../catgirl => img/app}/etc/s6-rc/mdevd/run | 0 .../catgirl => img/app}/etc/s6-rc/mdevd/type | 0 .../app}/etc/s6-rc/mdevd/type.license | 0 .../app}/etc/s6-rc/ok-all/contents | 0 .../catgirl => img/app}/etc/s6-rc/ok-all/type | 0 .../app}/etc/s6-rc/ok-all/type.license | 0 .../app}/etc/ssl/certs/ca-certificates.crt | 0 {vm/app/lynx => img/app}/shell.nix | 11 +- release.nix | 2 +- {img => release}/combined/default.nix | 0 {img => release}/combined/eosimages.nix | 0 {img => release}/combined/grub.cfg.in | 0 {img => release}/combined/run-vm.nix | 0 ...ble-gpt-partition-attribute-55-check.patch | 0 ...pt-disable-partition-table-CRC-check.patch | 0 .../0003-install-remove-Endless-OS-ad.patch | 0 ...4-finished-don-t-run-eos-diagnostics.patch | 0 ...omote-spectrum-not-the-Endless-forum.patch | 0 {img => release}/installer/app/default.nix | 0 .../installer/app/vendor-customer-support.ini | 0 {img => release}/installer/configuration.nix | 0 {img => release}/installer/default.nix | 0 {img => release}/installer/run-vm.nix | 0 {img => release}/installer/seat.rules | 0 {img => release}/live/Makefile | 0 {img => release}/live/default.nix | 0 {img => release}/live/shell.nix | 0 scripts/make-gpt.sh | 4 +- scripts/sfdisk-field.awk | 2 +- vm-lib/make-vm.nix | 51 ++++++++ vm/app/catgirl.nix | 17 +++ vm/app/catgirl/Makefile | 123 ------------------ vm/app/catgirl/default.nix | 92 ------------- vm/app/catgirl/etc/fstab | 6 - vm/app/catgirl/etc/mdev/iface | 36 ----- .../catgirl/etc/s6-linux-init/scripts/rc.init | 10 -- vm/app/catgirl/etc/s6-rc/catgirl/run | 31 ----- .../data/appvm-catgirl/providers/net/netvm | 0 vm/app/catgirl/shell.nix | 17 --- vm/app/lynx.nix | 15 +++ vm/app/lynx/bin | 1 - vm/app/lynx/etc/fstab | 6 - vm/app/lynx/etc/init | 5 - vm/app/lynx/etc/mdev.conf | 5 - vm/app/lynx/etc/passwd | 1 - vm/app/lynx/etc/passwd.license | 2 - vm/app/lynx/etc/resolv.conf | 4 - vm/app/lynx/etc/s6-rc/lynx/type | 1 - vm/app/lynx/etc/s6-rc/lynx/type.license | 2 - .../etc/s6-rc/mdevd-coldplug/dependencies | 4 - vm/app/lynx/etc/s6-rc/mdevd-coldplug/type | 1 - .../etc/s6-rc/mdevd-coldplug/type.license | 2 - vm/app/lynx/etc/s6-rc/mdevd-coldplug/up | 4 - vm/app/lynx/etc/s6-rc/mdevd/notification-fd | 1 - .../etc/s6-rc/mdevd/notification-fd.license | 2 - vm/app/lynx/etc/s6-rc/mdevd/run | 5 - vm/app/lynx/etc/s6-rc/mdevd/type | 1 - vm/app/lynx/etc/s6-rc/mdevd/type.license | 2 - vm/app/lynx/etc/s6-rc/ok-all/contents | 4 - vm/app/lynx/etc/s6-rc/ok-all/type | 1 - vm/app/lynx/etc/s6-rc/ok-all/type.license | 2 - vm/app/lynx/etc/ssl/certs/ca-certificates.crt | 1 - .../host/data/appvm-lynx/providers/net/netvm | 0 vm/make-vm.nix | 9 ++ vm/sys/net/Makefile | 23 ++-- vm/sys/net/default.nix | 10 +- 92 files changed, 236 insertions(+), 457 deletions(-) rename {vm/app/lynx => img/app}/Makefile (66%) rename {vm/app/catgirl => img/app}/bin (100%) rename {vm/app/lynx => img/app}/default.nix (77%) create mode 100644 img/app/etc/fstab rename {vm/app/catgirl => img/app}/etc/init (100%) rename {vm/app/catgirl => img/app}/etc/mdev.conf (100%) rename {vm/app/lynx => img/app}/etc/mdev/iface (98%) rename {vm/app/catgirl => img/app}/etc/passwd (100%) rename {vm/app/catgirl => img/app}/etc/passwd.license (100%) rename {vm/app/catgirl => img/app}/etc/resolv.conf (100%) rename {vm/app/lynx => img/app}/etc/s6-linux-init/scripts/rc.init (90%) rename {vm/app/lynx/etc/s6-rc/lynx => img/app/etc/s6-rc/app}/run (80%) rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type (100%) rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/dependencies (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/up (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/run (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/contents (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type.license (100%) rename {vm/app/catgirl => img/app}/etc/ssl/certs/ca-certificates.crt (100%) rename {vm/app/lynx => img/app}/shell.nix (51%) rename {img => release}/combined/default.nix (100%) rename {img => release}/combined/eosimages.nix (100%) rename {img => release}/combined/grub.cfg.in (100%) rename {img => release}/combined/run-vm.nix (100%) rename {img => release}/installer/app/0001-gpt-disable-gpt-partition-attribute-55-check.patch (100%) rename {img => release}/installer/app/0002-gpt-disable-partition-table-CRC-check.patch (100%) rename {img => release}/installer/app/0003-install-remove-Endless-OS-ad.patch (100%) rename {img => release}/installer/app/0004-finished-don-t-run-eos-diagnostics.patch (100%) rename {img => release}/installer/app/0005-finished-promote-spectrum-not-the-Endless-forum.patch (100%) rename {img => release}/installer/app/default.nix (100%) rename {img => release}/installer/app/vendor-customer-support.ini (100%) rename {img => release}/installer/configuration.nix (100%) rename {img => release}/installer/default.nix (100%) rename {img => release}/installer/run-vm.nix (100%) rename {img => release}/installer/seat.rules (100%) rename {img => release}/live/Makefile (100%) rename {img => release}/live/default.nix (100%) rename {img => release}/live/shell.nix (100%) create mode 100644 vm-lib/make-vm.nix create mode 100644 vm/app/catgirl.nix delete mode 100644 vm/app/catgirl/Makefile delete mode 100644 vm/app/catgirl/default.nix delete mode 100644 vm/app/catgirl/etc/fstab delete mode 100755 vm/app/catgirl/etc/mdev/iface delete mode 100755 vm/app/catgirl/etc/s6-linux-init/scripts/rc.init delete mode 100755 vm/app/catgirl/etc/s6-rc/catgirl/run delete mode 100644 vm/app/catgirl/host/data/appvm-catgirl/providers/net/netvm delete mode 100644 vm/app/catgirl/shell.nix create mode 100644 vm/app/lynx.nix delete mode 120000 vm/app/lynx/bin delete mode 100644 vm/app/lynx/etc/fstab delete mode 100755 vm/app/lynx/etc/init delete mode 100644 vm/app/lynx/etc/mdev.conf delete mode 100644 vm/app/lynx/etc/passwd delete mode 100644 vm/app/lynx/etc/passwd.license delete mode 100644 vm/app/lynx/etc/resolv.conf delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/dependencies delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/up delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/run delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/contents delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type.license delete mode 120000 vm/app/lynx/etc/ssl/certs/ca-certificates.crt delete mode 100644 vm/app/lynx/host/data/appvm-lynx/providers/net/netvm create mode 100644 vm/make-vm.nix base-commit: 7a6d44e24ddcc9cba73deed25fb85038b7c3d823 -- 2.37.1

2 12

[PATCH 0/2] IOMMU fixes
by Alyssa Ross 14 Nov '22

14 Nov '22

Vadim found that it was not possible to test networking in the live image's make run target. This turned out to be for two separate reasons. One was that we didn't have an IOMMU on that QEMU machine, but even adding that wasn't enough, because Intel IOMMUs also just didn't work at all on the live image, due to a missing kernel command line parameter. Alyssa Ross (2): Set intel_iommu=on in kernel command line Add IOMMU to other host test targets host/initramfs/Makefile | 4 +++- img/combined/run-vm.nix | 3 ++- img/live/Makefile | 4 +++- 3 files changed, 8 insertions(+), 3 deletions(-) base-commit: 3aa2f8aa7423cdf36c797943ad7d5a9756290909 -- 2.37.1

2 4

[PATCH 1/2] scripts: fix shellcheck warnings
by Alyssa Ross 11 Nov '22

11 Nov '22

Signed-off-by: Alyssa Ross <alyssa.ross(a)unikie.com> --- Henri, I've CCed you on this series, in case you'd like to review. We have some documentation on reviewing patches here: https://spectrum-os.org/doc/development/reviewing-patches.html scripts/format-uuid.sh | 12 ++++++------ scripts/make-gpt.sh | 6 +++--- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh index 6758088..441ed64 100755 --- a/scripts/format-uuid.sh +++ b/scripts/format-uuid.sh @@ -8,12 +8,12 @@ substr () { str=$1 beg=$2 end=$3 - echo $str | cut -c $beg-$end + echo "$str" | cut -c "$beg-$end" } -u1=$(substr $1 1 8) -u2=$(substr $1 9 12) -u3=$(substr $1 13 16) -u4=$(substr $1 17 20) -u5=$(substr $1 21 32) +u1=$(substr "$1" 1 8) +u2=$(substr "$1" 9 12) +u3=$(substr "$1" 13 16) +u4=$(substr "$1" 17 20) +u5=$(substr "$1" 21 32) printf "%s\n" "$u1-$u2-$u3-$u4-$u5" diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 3215edb..554182f 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -23,7 +23,7 @@ sizeMiB() { fillPartition() { sfdisk -J "$1" | jq -r --argjson index "$2" \ '.partitiontable.partitions[$index] | "\(.start) \(.size)"' | - (read start size; + (read -r start size; dd if="$3" of="$1" seek="$start" count="$size" conv=notrunc) } @@ -48,7 +48,7 @@ gptBytes=$TWO_MiB for partition; do sizeMiB="$(sizeMiB "$(partitionPath "$partition")")" table="$table${nl}size=${sizeMiB}MiB,$(awk -f "$scriptsDir/sfdisk-field.awk" -v partition="$partition")" - gptBytes="$(expr "$gptBytes" + "$sizeMiB" \* $ONE_MiB)" + gptBytes="$((gptBytes + sizeMiB * ONE_MiB))" done rm -f "$out" @@ -60,5 +60,5 @@ EOF n=0 for partition; do fillPartition "$out" "$n" "$(partitionPath "$partition")" - n="$(expr "$n" + 1)" + n="$((n + 1))" done -- 2.35.1

3 5

[PATCH v3] Remove bashisms from spectrum shell scripts
by Henri Rosten 10 Nov '22

10 Nov '22

This commit removes bashisms from spectrum shell scripts. This change is needed to be able to use the scripts from POSIX-compliant shells which are not bash compatible - such as dash. Signed-off-by: Henri Rosten <henri.rosten(a)unikie.com> --- Documentation/scripts/undocumented-uuids.sh | 9 +++++++-- scripts/format-uuid.sh | 15 ++++++++++++++- scripts/make-gpt.sh | 4 +++- 3 files changed, 24 insertions(+), 4 deletions(-) diff --git a/Documentation/scripts/undocumented-uuids.sh b/Documentation/scripts/undocumented-uuids.sh index 34c2d22..c75bfd2 100755 --- a/Documentation/scripts/undocumented-uuids.sh +++ b/Documentation/scripts/undocumented-uuids.sh @@ -1,14 +1,19 @@ #!/bin/sh -eu # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ cd "$(dirname "$0")/../.." PATTERN='\b[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}\b' -UUID_REFERENCE_PATH=Documentation/uuid-reference.adoc +UUID_REFERENCE_PATH=Documentation/appendices/uuid-reference.adoc +documented_uuids=$(mktemp) +trap 'rm -f -- "$documented_uuids"' EXIT + +grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u >"$documented_uuids" git ls-files -coz --exclude-standard | grep -Fxvz "$UUID_REFERENCE_PATH" | xargs -0 git grep -Ehio --no-index --no-line-number "$PATTERN" -- | sort -u | - comm -23 - <(grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u) + comm -23 - "$documented_uuids" diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh index fa07eb9..6758088 100755 --- a/scripts/format-uuid.sh +++ b/scripts/format-uuid.sh @@ -1,6 +1,19 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ -printf "%s\n" "${1:0:8}-${1:8:4}-${1:12:4}-${1:16:4}-${1:20}" +substr () { + str=$1 + beg=$2 + end=$3 + echo $str | cut -c $beg-$end +} + +u1=$(substr $1 1 8) +u2=$(substr $1 9 12) +u3=$(substr $1 13 16) +u4=$(substr $1 17 20) +u5=$(substr $1 21 32) +printf "%s\n" "$u1-$u2-$u3-$u4-$u5" diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 56076d3..3215edb 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,6 +1,7 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... @@ -38,7 +39,8 @@ scriptsDir="$(dirname "$0")" out="$1" shift -nl=$'\n' +nl=' +' table="label: gpt" # Keep 1MiB free at the start, and 1MiB free at the end. -- 2.25.1

2 1

[PATCH v2] Remove bashisms from spectrum shell scripts
by Henri Rosten 09 Nov '22

09 Nov '22

This commit removes bashisms from spectrum shell scripts. This change is needed to be able to use the scripts from POSIX-compliant shells which are not bash compatible - such as dash. Signed-off-by: Henri Rosten <henri.rosten(a)unikie.com> --- Documentation/scripts/undocumented-uuids.sh | 7 ++++++- scripts/format-uuid.sh | 15 ++++++++++++++- scripts/make-gpt.sh | 7 +++---- 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/Documentation/scripts/undocumented-uuids.sh b/Documentation/scripts/undocumented-uuids.sh index 34c2d22..fa21366 100755 --- a/Documentation/scripts/undocumented-uuids.sh +++ b/Documentation/scripts/undocumented-uuids.sh @@ -1,14 +1,19 @@ #!/bin/sh -eu # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ +trap 'rm -f -- "$documented_uuids"' EXIT + cd "$(dirname "$0")/../.." PATTERN='\b[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}\b' UUID_REFERENCE_PATH=Documentation/uuid-reference.adoc +documented_uuids=$(mktemp) +grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u >$documented_uuids git ls-files -coz --exclude-standard | grep -Fxvz "$UUID_REFERENCE_PATH" | xargs -0 git grep -Ehio --no-index --no-line-number "$PATTERN" -- | sort -u | - comm -23 - <(grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u) + comm -23 - $documented_uuids diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh index fa07eb9..6758088 100755 --- a/scripts/format-uuid.sh +++ b/scripts/format-uuid.sh @@ -1,6 +1,19 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ -printf "%s\n" "${1:0:8}-${1:8:4}-${1:12:4}-${1:16:4}-${1:20}" +substr () { + str=$1 + beg=$2 + end=$3 + echo $str | cut -c $beg-$end +} + +u1=$(substr $1 1 8) +u2=$(substr $1 9 12) +u3=$(substr $1 13 16) +u4=$(substr $1 17 20) +u5=$(substr $1 21 32) +printf "%s\n" "$u1-$u2-$u3-$u4-$u5" diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 56076d3..351aa69 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,6 +1,7 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" out="$1" shift -nl=$'\n' +nl='\n' table="label: gpt" # Keep 1MiB free at the start, and 1MiB free at the end. @@ -51,9 +52,7 @@ done rm -f "$out" truncate -s "$gptBytes" "$out" -sfdisk "$out" <<EOF -$table -EOF +printf "$table" | sfdisk "$out" n=0 for partition; do -- 2.25.1

2 2

[PATCH] Remove bashisms from spectrum shell scripts
by Henri Rosten 09 Nov '22

09 Nov '22

This commit removes bashisms from spectrum shell scripts. This change is needed to be able to use the scripts from POSIX-compliant shells which are not bash compatible - such as dash. Signed-off-by: Henri Rosten <henri.rosten(a)unikie.com> --- Documentation/scripts/undocumented-uuids.sh | 6 +++++- scripts/format-uuid.sh | 15 ++++++++++++++- scripts/make-gpt.sh | 7 +++---- 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/Documentation/scripts/undocumented-uuids.sh b/Documentation/scripts/undocumented-uuids.sh index 34c2d22..0434407 100755 --- a/Documentation/scripts/undocumented-uuids.sh +++ b/Documentation/scripts/undocumented-uuids.sh @@ -1,5 +1,6 @@ #!/bin/sh -eu # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ cd "$(dirname "$0")/../.." @@ -7,8 +8,11 @@ cd "$(dirname "$0")/../.." PATTERN='\b[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}\b' UUID_REFERENCE_PATH=Documentation/uuid-reference.adoc +tmp=$(mktemp) +grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u >$tmp git ls-files -coz --exclude-standard | grep -Fxvz "$UUID_REFERENCE_PATH" | xargs -0 git grep -Ehio --no-index --no-line-number "$PATTERN" -- | sort -u | - comm -23 - <(grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u) + comm -23 - $tmp +rm -f $tmp diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh index fa07eb9..c7b94da 100755 --- a/scripts/format-uuid.sh +++ b/scripts/format-uuid.sh @@ -1,6 +1,19 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ -printf "%s\n" "${1:0:8}-${1:8:4}-${1:12:4}-${1:16:4}-${1:20}" +substr () { + str=$1 + beg=$2 + end=$3 + echo $(echo $str | cut -c $beg-$end) +} + +u1=$(substr $1 1 8) +u2=$(substr $1 9 12) +u3=$(substr $1 13 16) +u4=$(substr $1 17 20) +u5=$(substr $1 21 32) +printf "%s\n" "$u1-$u2-$u3-$u4-$u5" diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 56076d3..351aa69 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,6 +1,7 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" out="$1" shift -nl=$'\n' +nl='\n' table="label: gpt" # Keep 1MiB free at the start, and 1MiB free at the end. @@ -51,9 +52,7 @@ done rm -f "$out" truncate -s "$gptBytes" "$out" -sfdisk "$out" <<EOF -$table -EOF +printf "$table" | sfdisk "$out" n=0 for partition; do -- 2.25.1

2 4

[PATCH v4] Docs: new structure
by Jenni Nikolaenko 08 Nov '22

08 Nov '22

Create separate folders for new parent pages, update Introduction page, remove a and the articles from titles, quick check text for simple english Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/{ => about}/architecture.adoc | 51 +++++++++--------- Documentation/about/index.adoc | 32 ++++++++++++ Documentation/appendices/index.adoc | 13 +++++ Documentation/appendices/user-partition.adoc | 17 ++++++ .../{ => appendices}/uuid-reference.adoc | 8 +-- Documentation/building-documentation.adoc | 52 ------------------- .../decisions/001-host-update-mechanism.adoc | 4 +- .../decisions/002-install-options.adoc | 4 +- Documentation/decisions/003-partitioning.adoc | 2 +- .../004-data-at-rest-encryption.adoc | 4 +- .../005-virtual-machine-monitor.adoc | 2 +- .../decisions/006-drivers-on-host.adoc | 2 +- .../decisions/007-usb-virtual-machines.adoc | 2 +- ...008-inter-vm-communication-mechanisms.adoc | 2 +- Documentation/decisions/index.adoc | 2 +- Documentation/{ => development}/b4.adoc | 6 ++- .../build-configuration.adoc | 13 +++-- .../development/building-documentation.adoc | 52 +++++++++++++++++++ .../{ => development}/debugging.adoc | 12 ++--- .../{ => development}/first-patch.adoc | 32 ++++++------ Documentation/development/index.adoc | 27 ++++++++++ Documentation/{ => development}/replying.adoc | 13 ++--- .../{ => development}/reviewing-patches.adoc | 4 +- .../{ => development}/testing-patches.adoc | 28 +++++----- .../development/working-with-patches.adoc | 12 +++++ Documentation/explanation.adoc | 6 --- .../{ => getting-started}/creating-vms.adoc | 3 +- Documentation/getting-started/index.adoc | 13 +++++ .../{ => getting-started}/running-vms.adoc | 3 +- Documentation/how-to.adoc | 6 --- Documentation/index.adoc | 29 +++++++++-- .../{ => installation}/binary-cache.adoc | 22 +++++--- .../{ => installation}/getting-spectrum.adoc | 16 +++--- Documentation/installation/index.adoc | 23 ++++++++ Documentation/reference.adoc | 6 --- Documentation/tutorials.adoc | 6 --- Documentation/user-partition.adoc | 12 ----- 37 files changed, 341 insertions(+), 200 deletions(-) rename Documentation/{ => about}/architecture.adoc (56%) create mode 100644 Documentation/about/index.adoc create mode 100644 Documentation/appendices/index.adoc create mode 100644 Documentation/appendices/user-partition.adoc rename Documentation/{ => appendices}/uuid-reference.adoc (96%) delete mode 100644 Documentation/building-documentation.adoc rename Documentation/{ => development}/b4.adoc (91%) rename Documentation/{ => development}/build-configuration.adoc (73%) create mode 100644 Documentation/development/building-documentation.adoc rename Documentation/{ => development}/debugging.adoc (79%) rename Documentation/{ => development}/first-patch.adoc (80%) create mode 100644 Documentation/development/index.adoc rename Documentation/{ => development}/replying.adoc (68%) rename Documentation/{ => development}/reviewing-patches.adoc (88%) rename Documentation/{ => development}/testing-patches.adoc (82%) create mode 100644 Documentation/development/working-with-patches.adoc delete mode 100644 Documentation/explanation.adoc rename Documentation/{ => getting-started}/creating-vms.adoc (98%) create mode 100644 Documentation/getting-started/index.adoc rename Documentation/{ => getting-started}/running-vms.adoc (90%) delete mode 100644 Documentation/how-to.adoc rename Documentation/{ => installation}/binary-cache.adoc (74%) rename Documentation/{ => installation}/getting-spectrum.adoc (79%) create mode 100644 Documentation/installation/index.adoc delete mode 100644 Documentation/reference.adoc delete mode 100644 Documentation/tutorials.adoc delete mode 100644 Documentation/user-partition.adoc diff --git a/Documentation/architecture.adoc b/Documentation/about/architecture.adoc similarity index 56% rename from Documentation/architecture.adoc rename to Documentation/about/architecture.adoc index 1c4307b..b311e77 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/about/architecture.adoc @@ -1,32 +1,31 @@ = Architecture -:page-parent: Explanation +:page-parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Introduction +Spectrum is based on the principle of security by compartmentalization. -Spectrum is based on the principle of security by -compartmentalization. The high level stack is illustrated in the -following diagram: +The high level stack is illustrated in the following diagram: -image::diagrams/stack.svg[] +image::../diagrams/stack.svg[] The default set of virtual machines includes two application VMs, _appvm-catgirl_ (an IRC client) and _appvm-lynx_ (a text-based web browser); and a system VM, _netvm_ (which handles hardware network -devices and provides network services to application VMs). Refer to -xref:creating-vms.adoc[Creating VMs] and xref:running-vms.adoc[Running +devices and provides network services to application VMs). +Refer to xref:../getting-started/creating-vms.adoc[Creating VMs] and +xref:../getting-started/running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. == Architecture Decision Records (ADRs) https://en.wikipedia.org/wiki/Architectural_decision[Architecturally significant -decisions] are xref:decisions/index.adoc[recorded] as lightweight -https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions/[ADRs]. +decisions] are xref:../decisions/index.adoc[recorded] as lightweight +https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions[ADRs]. -== The Spectrum host system +== Spectrum Host System Compartmentalization is implemented using https://cloud-hypervisor.org/[cloud-hypervisor] virtual machines. @@ -35,8 +34,8 @@ https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[Kernel-based Virtual Machine] (KVM) to provide lightweight, hardware-accelerated VMs. While Linux (including KVM) is portable between many hardware architectures, -cloud-hypervisor supports only x86_64 and aarch64. Spectrum currently only -works on x86_64, but aarch64 support is in development. +cloud-hypervisor supports only x86_64 and aarch64. +Spectrum currently only works on x86_64, but aarch64 support is in development. https://skarnet.org/software/s6-rc/overview.html[s6-rc] is used for service management. Utilities like kmod, util-linux and busybox are installed for boot @@ -44,20 +43,22 @@ and service scripts. https://wayland.freedesktop.org/[Wayland] is used for window management and display. The Wayland architecture is well documented -https://wayland.freedesktop.org/architecture.html[here]. The host provides only -a Wayland terminal client, https://codeberg.org/dnkl/foot/[foot], which is used -for interacting with VM consoles. In future it will be possible for application -VMs to display windows on the single Wayland compositor on the host system, -using Wayland virtio-gpu contexts. X11 applications are not currently -supported, but could be in future. +https://wayland.freedesktop.org/architecture.html[here]. +The host provides onlya Wayland terminal client, +https://codeberg.org/dnkl/foot/[foot], which is used for interacting with VM +consoles. +In future it will be possible for application VMs to display windows on the +single Wayland compositor on the host system, using Wayland virtio-gpu +contexts. X11 applications are not currently supported, but could be in future. All host user space components are built with the https://musl.libc.org/[musl C -library]. In comparison to other libc implementations, musl has +library]. +In comparison to other libc implementations, musl has https://www.etalabs.net/compare_libcs.html[added safety on resource exhaustion -and security hardening on memory allocation]. Kernel hardening will be -investigated in future. +and security hardening on memory allocation]. +Kernel hardening will be investigated in future. -== Exploring the Spectrum dependency tree +== Spectrum Dependency Tree For a detailed, interactive view of dependencies, use https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: @@ -66,5 +67,5 @@ https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: [listing] nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree -https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[See video of Spectrum live image -interactive analysis with nix-tree] +See the https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[video] of Spectrum live +image interactive analysis with nix-tree. diff --git a/Documentation/about/index.adoc b/Documentation/about/index.adoc new file mode 100644 index 0000000..6961b6a --- /dev/null +++ b/Documentation/about/index.adoc @@ -0,0 +1,32 @@ += About Spectrum +:description: Some words about Spectrum as the operating system, not a project. Highlights the differences between common Linux distributions and Spectrum. +:page-nav_order: 1 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is a Linux-based system that uses the +https://github.com/NixOS/nix[Nix package manager] and +the https://github.com/NixOS/nixpkgs[Nix Packages collection] (Nixpkgs). + +This gives an actively-developed base with good +hardware support, powerful and optimised compartmentalization primitives in +https://www.linux-kvm.org/page/Main_Page[KVM], and the reproducible packaging +and configuration system that is important +for a maintainable compartmentalized system. + +== Why Spectrum + +* User data and applications are managed centrally while remaining isolated. +That means that the system can be backed up and managed as a whole, rather than +mixed up in several dozen VMs. + +* The host system and isolated environments are managed declaratively and +reproducibly using the Nix package manager. +This can save the user the burden of maintaining many different virtual +computers, allowing finer-grained resource access controls and making it +possible to verify the software running across all environments. + +TIP: If you are interested in why we do something _this_ way instead of _that_ +way, see xref:../decisions/index.adoc[Architecture Decision Records]. diff --git a/Documentation/appendices/index.adoc b/Documentation/appendices/index.adoc new file mode 100644 index 0000000..ebacda7 --- /dev/null +++ b/Documentation/appendices/index.adoc @@ -0,0 +1,13 @@ += Appendices +:description: Additional information. +:page-nav_order: 5 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +This section provides additional information on how to work with: + +* xref:../appendices/user-partition.adoc[user partition] +* xref:../appendices/uuid-reference.adoc[universally unique identifier (UUID)]. diff --git a/Documentation/appendices/user-partition.adoc b/Documentation/appendices/user-partition.adoc new file mode 100644 index 0000000..4121340 --- /dev/null +++ b/Documentation/appendices/user-partition.adoc @@ -0,0 +1,17 @@ += Appendix A. User Partition +:page-parent: Appendices +:page-nav_order: 1 + +// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +The Spectrum host system is immutable, so configuration and user data +lives on a separate partition. + +The host system discovers the user +partition by looking for the special partition type +`9293e1ff-cee4-4658-88be-898ec863944f`. + +This behavior can be overridden with the `ext` parameter on the host's kernel +command line, which works in a similar way to the standard Linux `root` +parameter. diff --git a/Documentation/uuid-reference.adoc b/Documentation/appendices/uuid-reference.adoc similarity index 96% rename from Documentation/uuid-reference.adoc rename to Documentation/appendices/uuid-reference.adoc index 4b0b481..146c068 100644 --- a/Documentation/uuid-reference.adoc +++ b/Documentation/appendices/uuid-reference.adoc @@ -1,7 +1,7 @@ -= UUID Reference -:page-parent: Reference -:toc: preamble += Appendix B. UUID +:page-parent: Appendices :toclevels: 1 +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -40,7 +40,7 @@ xref:user-partition.adoc[Spectrum user partition]. === `56a3bbc3-aefa-43d9-a64d-7b3fd59bbc4e` -https://github.com/endlessm/eos-installer["eosimages"] partition on the +https://github.com/endlessm/eos-installer["eosimages"] partition on the Spectrum combined live system / installer image. == Combined Image Partition IDs diff --git a/Documentation/building-documentation.adoc b/Documentation/building-documentation.adoc deleted file mode 100644 index b491105..0000000 --- a/Documentation/building-documentation.adoc +++ /dev/null @@ -1,52 +0,0 @@ -= Building the Documentation -:page-parent: Tutorials - -// SPDX-FileCopyrightText: 2022 Unikie -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -This tutorial assumes that you have https://nixos.org/[Nix] installed. -You may also want to xref:binary-cache.adoc[configure the Spectrum -binary cache], to avoid having to wait for dependencies to compile on -your local system. - -1. Get a copy of the Spectrum source code: -+ -[source,shell] ----- -git clone https://spectrum-os.org/git/spectrum ----- -2. Enter the documentation directory: -+ -[source,shell] ----- -cd spectrum/Documentation ----- -3. Enter the development environment: -+ -[source,shell] ----- -nix-shell -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz ----- -4. In the development shell, do an initial build of the documentation -site: -+ -[source,shell] ----- -scripts/build.sh ----- -5. Run a development server for previewing changes locally: -+ -[source,shell] ----- -jekyll serve ----- -+ -This will serve a local copy of the documentation at http://localhost:4000/. -+ -IMPORTANT: Jekyll doesn't handle rendering of the draw.io diagrams, so -if you modify any of those, or add new ones, you'll have to run -`scripts/build.sh` again to do a full rebuild of the site. - -Once you've made your changes to the documentation, see -xref:first-patch.adoc[Sending Your First Patch] for information -about how to submit them for review. diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc index 574deb4..39f9f28 100644 --- a/Documentation/decisions/001-host-update-mechanism.adoc +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -1,6 +1,6 @@ = 001 Host Update Mechanism :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -11,7 +11,7 @@ Proposed == Context -Spectrum currently has no implementation for software update. The host -- +Spectrum currently has no implementation for software update. The host -- consisting of the Linux kernel, KVM, cloud-hypervisor and minimal user space tools -- will require software updates to support feature development and security fixes. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc index 4412b53..4a745eb 100644 --- a/Documentation/decisions/002-install-options.adoc +++ b/Documentation/decisions/002-install-options.adoc @@ -1,6 +1,6 @@ -= 002 Install options += 002 Install Options :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc index 8494ea4..a13b8cc 100644 --- a/Documentation/decisions/003-partitioning.adoc +++ b/Documentation/decisions/003-partitioning.adoc @@ -1,6 +1,6 @@ = 003 Partitioning :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc index 26fe273..5b0f518 100644 --- a/Documentation/decisions/004-data-at-rest-encryption.adoc +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -1,6 +1,6 @@ = 004 Data at Rest Encryption :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -20,5 +20,5 @@ User data is encrypted. == Consequences Spectrum needs to come with enough software to get the encryption key -via different methods (password, usb, fido, etc.) Can we use dm-crypt +via different methods (password, usb, fido, etc.). Can we use dm-crypt for everything instead of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-monitor.adoc b/Documentation/decisions/005-virtual-machine-monitor.adoc index db81c72..df1b501 100644 --- a/Documentation/decisions/005-virtual-machine-monitor.adoc +++ b/Documentation/decisions/005-virtual-machine-monitor.adoc @@ -1,6 +1,6 @@ = 005 Virtual Machine Monitor :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc index 872044e..b92d863 100644 --- a/Documentation/decisions/006-drivers-on-host.adoc +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -1,6 +1,6 @@ = 006 Drivers on Host :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/007-usb-virtual-machines.adoc b/Documentation/decisions/007-usb-virtual-machines.adoc index 3bdf78b..d832691 100644 --- a/Documentation/decisions/007-usb-virtual-machines.adoc +++ b/Documentation/decisions/007-usb-virtual-machines.adoc @@ -1,6 +1,6 @@ = 007 USB Virtual Machine :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc index a1b7d49..c1e5b87 100644 --- a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc +++ b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc @@ -1,6 +1,6 @@ = 008 Inter-VM Communication Mechanisms :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/index.adoc b/Documentation/decisions/index.adoc index 772f382..a022239 100644 --- a/Documentation/decisions/index.adoc +++ b/Documentation/decisions/index.adoc @@ -1,6 +1,6 @@ = Architecture Decision Records :page-has_children: true -:page-parent: Explanation +:page-parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/b4.adoc b/Documentation/development/b4.adoc similarity index 91% rename from Documentation/b4.adoc rename to Documentation/development/b4.adoc index 489ced4..5f141e9 100644 --- a/Documentation/b4.adoc +++ b/Documentation/development/b4.adoc @@ -1,5 +1,7 @@ = Installing and Configuring b4 -:page-parent: Tutorials +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-FileCopyrightText: 2022 Unikie @@ -19,7 +21,7 @@ of the Spectrum root's nix-shell. You should be able to install b4 from your package manager. -Using Nix, you can start a shell with b4 available by running +Using Nix, you can start a shell with b4 available by running: [listing] [source,shell] diff --git a/Documentation/build-configuration.adoc b/Documentation/development/build-configuration.adoc similarity index 73% rename from Documentation/build-configuration.adoc rename to Documentation/development/build-configuration.adoc index b89575f..c9a8c99 100644 --- a/Documentation/build-configuration.adoc +++ b/Documentation/development/build-configuration.adoc @@ -1,19 +1,22 @@ = Configuring the Build -:page-parent: How-to Guides +:page-parent: Development +:page-nav_order: 1 :example-caption: Test // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 Some aspects of a Spectrum build can be customised using a build -configuration file. By default, this configuration file should be -called config.nix and located in the root of the Spectrum source tree, -but this can be overridden by setting `spectrum-config` in the +configuration file. + +By default, this configuration file should be called config.nix and located in +the root of the Spectrum source tree, but this can be overridden by setting +`spectrum-config` in the https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PAT… to the path of the configuration file. The configuration file should contain an attribute set. The only -currently allowed attribute name is `pkgs`, which allows using a +currently allowed attribute name is `pkgs`. It allows using a custom Nixpkgs to evaluate Spectrum. .config.nix to build Spectrum with a https://nixos.org/manual/nixpkgs/unstable/#sec-overlays-definition[Nixpkgs overlay] diff --git a/Documentation/development/building-documentation.adoc b/Documentation/development/building-documentation.adoc new file mode 100644 index 0000000..4f464f5 --- /dev/null +++ b/Documentation/development/building-documentation.adoc @@ -0,0 +1,52 @@ += Building Documentation +:page-parent: Development +:page-nav_order: 5 + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Make sure you have https://nixos.org/download.html[Nix] installed. +You may also want to xref:../installation/binary-cache.adoc[configure the Spectrum +binary cache], to avoid having to wait for dependencies to compile on +your local system. + +. Get a copy of the Spectrum source code: ++ +[source,shell] +---- +git clone https://spectrum-os.org/git/spectrum +---- +. Enter the documentation directory: ++ +[source,shell] +---- +cd spectrum/Documentation +---- +. Enter the development environment: ++ +[source,shell] +---- +nix-shell -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz +---- +. In the development shell, do an initial build of the documentation +site: ++ +[source,shell] +---- +scripts/build.sh +---- +. Run a development server for previewing changes locally: ++ +[source,shell] +---- +jekyll serve +---- ++ +This will serve a local copy of the documentation at http://localhost:4000/. ++ +IMPORTANT: Jekyll does not handle rendering of the draw.io diagrams. If you +modify any of those, or add new ones, run `scripts/build.sh` again to do a full +rebuild of the site. + +After making changes to the documentation, see how to +xref:first-patch.adoc[send your patch] and submit the changes for review. diff --git a/Documentation/debugging.adoc b/Documentation/development/debugging.adoc similarity index 79% rename from Documentation/debugging.adoc rename to Documentation/development/debugging.adoc index 3871a7c..f1bbf01 100644 --- a/Documentation/debugging.adoc +++ b/Documentation/development/debugging.adoc @@ -1,7 +1,6 @@ -= Debugging Spectrum -:page-parent: Explanation -:toc: -:toclevels: 1 += Debugging +:page-parent: Development +:page-nav_order: 4 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -9,8 +8,9 @@ == Extracting core dumps when running Spectrum in a VM When using a VM to run the Spectrum host system, a special mechanism -is available to enable easy introspection of core files. When a -program on the Spectrum host system dumps core, the system will +is available to enable easy introspection of core files. + +When a program on the Spectrum host system dumps core, the system will attempt to upload the core file to _its_ host (i.e. the system running Spectrum in a VM) using the vsock(7) protocol, on port 1129271877. diff --git a/Documentation/first-patch.adoc b/Documentation/development/first-patch.adoc similarity index 80% rename from Documentation/first-patch.adoc rename to Documentation/development/first-patch.adoc index 30672b9..aa6ebee 100644 --- a/Documentation/first-patch.adoc +++ b/Documentation/development/first-patch.adoc @@ -1,11 +1,11 @@ -= Sending Your First Patch -:page-parent: Tutorials += Sending Your Patch +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Prerequisites - This tutorial assumes that you already have basic https://git-scm.com/[git] experience. @@ -14,19 +14,18 @@ https://spectrum-os.org/git/[Spectrum source tree]. You'll also need to have configured `git send-email` — a guide for this can be found at https://git-send-email.io/. -== Making your changes +== Making Changes -If you've worked on any git repository before, the process for making -your changes will probably be very familiar. +The process of making changes is similar to working on any git repository. -1. Create a branch for your changes: +. Create a branch for your changes: + [source,shell] ---- git checkout -b fix-docs # for example ---- -2. Make changes in your editor. -3. Stage and commit your changes: +. Make changes in your editor. +. Stage and commit your changes: + [source,shell] ---- @@ -39,7 +38,7 @@ indicates your acceptance of the https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin], which is mandatory for Spectrum patches. -== Submitting changes +== Submitting Changes Once you're happy with how the commits on your branch look, run: @@ -64,13 +63,14 @@ message that will be sent before all of your patches. Once your patch has been submitted, wait for it to be reviewed. Feedback, if any, will be sent as email replies to your submitted -patch. You can respond to feedback in your mail client. Please use -the Reply All button to ensure that your messages are sent to the +patch. You can respond to feedback in your mail client. + +Use the *Reply All* button to sent your messages to the mailing list as well as to the person who sent the feedback. -If you need to make changes to your patch, and submit a new version, +If you need to make changes to your patch and submit a new version, use https://git-rebase.io/[`git rebase`] to create a new version of -your patch(es), and submit it like this: +your patch(es) and then submit it like this: [source,shell] ---- @@ -81,7 +81,7 @@ The added `-v2` flag indicates that this is version two of your patch set. If your patches require more rounds of changes, submit subsequent rounds with `-v3`, `-v4`, etc. as appropriate. -If you'd like to describe what has changed from the previous version +If you would like to describe what has changed from the previous version of your patches, you can do so in a xref:cover-letter[cover letter] as described above. diff --git a/Documentation/development/index.adoc b/Documentation/development/index.adoc new file mode 100644 index 0000000..1fccdd5 --- /dev/null +++ b/Documentation/development/index.adoc @@ -0,0 +1,27 @@ += Development +:description: Development progress, general development practices +:page-nav_order: 4 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is free software, currently under active development. + + +== Developer Setup + +* https://git.kernel.org/pub/scm/utils/b4/b4.git/about/[b4] +* https://nixos.org/manual/nix/stable/introduction.html[Nix package manager] +* https://docs.asciidoctor.org/[AsciiDoc] for writing the documentation + + +== Mailing Lists + +The Spectrum project runs several +https://spectrum-os.org/participating.html#mailing-lists[mailing lists] on +which you can ask your questions or help other people with the questions they +have. All the Spectrum developers as well as many long time Linux and Spectrum users are on the lists. + +For real-time feedback, use +https://spectrum-os.org/participating.html#irc[IRC/Matrix channel]. diff --git a/Documentation/replying.adoc b/Documentation/development/replying.adoc similarity index 68% rename from Documentation/replying.adoc rename to Documentation/development/replying.adoc index bb8e31a..a1ad394 100644 --- a/Documentation/replying.adoc +++ b/Documentation/development/replying.adoc @@ -1,5 +1,7 @@ = Replying to Messages in the Mailing List Archives -:page-parent: Tutorials +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -9,14 +11,13 @@ https://spectrum-os.org/participating.html#mailing-lists[mailing lists]. Make sure to Reply All when replying to messages, so that the message -is sent to the mailing list and not just to the person you're -replying to. +is sent to the mailing list and not just to the person you reply to. == Getting a copy of a message -You may want to reply to a mailing list message that you don't have a -copy of in your mail client, because you aren't subscribed to the -list, or because you subscribed after the message was sent. +You may want to reply to a mailing list message that you do not have a +copy of in your mail client, because you are not subscribed to the +list or because you subscribed after the message was sent. To do this, find the message you want to reply to in the https://spectrum-os.org/lists/archives[public-inbox list archives], diff --git a/Documentation/reviewing-patches.adoc b/Documentation/development/reviewing-patches.adoc similarity index 88% rename from Documentation/reviewing-patches.adoc rename to Documentation/development/reviewing-patches.adoc index 63ff24e..ba47f25 100644 --- a/Documentation/reviewing-patches.adoc +++ b/Documentation/development/reviewing-patches.adoc @@ -1,5 +1,7 @@ = Reviewing Patches -:page-parent: How-to Guides +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/testing-patches.adoc b/Documentation/development/testing-patches.adoc similarity index 82% rename from Documentation/testing-patches.adoc rename to Documentation/development/testing-patches.adoc index 743cd6e..0c72c93 100644 --- a/Documentation/testing-patches.adoc +++ b/Documentation/development/testing-patches.adoc @@ -1,5 +1,7 @@ = Testing Patches -:page-parent: How-to Guides +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-FileCopyrightText: 2022 Unikie @@ -9,7 +11,7 @@ Potential changes to Spectrum are posted to and discussed on the https://spectrum-os.org/participating.html#spectrum-devel[devel@spectrum-os… mailing list. -== Apply the patch +== Apply Patch . Find the patch series you want to test on https://spectrum-os.org/lists/archives/spectrum-devel/[public-inbox]. @@ -33,15 +35,15 @@ of the Spectrum root's nix-shell, which allows you to skip this step. ---- b4 am 20220511092352.70E54C980(a)atuin.qyliss.net ---- - -. b4 will indicate the file name it has downloaded the patches into - with a line like: +b4 will indicate the file name it has downloaded the patches into with a line +like: + [example] [listing] +---- Writing ./20220424_hi_host_rootfs_fix_weston_hotplugging.mbx -+ -Run `git am` on that file to apply the patches, for example: +---- +. Run `git am` on that file to apply the patches. For example: + [example] [source,shell] @@ -49,14 +51,12 @@ Run `git am` on that file to apply the patches, for example: git am 20220424_hi_host_rootfs_fix_weston_hotplugging.mbx ---- -== Post your test results +== Post Your Results -When you've tested a patch, it's really helpful to +When you tested a patch, it is helpful to xref:replying.adoc[reply] with your test results. -If the patch worked for you, please reply to it and include a line -like the following, separated from any reply text: - +If the patch worked for you, please reply to it and include a line like the following, separated from any reply text: ---- Tested-by: John Smith <john(a)example.com> ---- @@ -66,10 +66,10 @@ patch replies will be automatically included in the commit message when a patch is applied. It's also helpful to explain in your reply how you tested the patch, -but you don't have to if it's obvious. (For example, if a patch is +but you don't have to if it's obvious. For example, if a patch is supposed to fix a bug, and you verified that after applying the patch the bug is fixed, just the Tested-by line on its own is enough to -indicate that.) +indicate that. If you found an issue with the patch, do not include a Tested-by line, and instead reply to the patch explaining what you tested, what you diff --git a/Documentation/development/working-with-patches.adoc b/Documentation/development/working-with-patches.adoc new file mode 100644 index 0000000..a2694ed --- /dev/null +++ b/Documentation/development/working-with-patches.adoc @@ -0,0 +1,12 @@ += Working with Patches +:page-parent: Development +:page-nav_order: 3 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Patches are the way for contributors to submit code to the Spectrum project. + +Make sure to xref:../development/b4.adoc[install and configure the b4 utility] +for applying and testing other people's patches. diff --git a/Documentation/explanation.adoc b/Documentation/explanation.adoc deleted file mode 100644 index b39cc6d..0000000 --- a/Documentation/explanation.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Explanation -:page-has_children: true -:page-nav_order: 4 - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/creating-vms.adoc b/Documentation/getting-started/creating-vms.adoc similarity index 98% rename from Documentation/creating-vms.adoc rename to Documentation/getting-started/creating-vms.adoc index d967098..725316c 100644 --- a/Documentation/creating-vms.adoc +++ b/Documentation/getting-started/creating-vms.adoc @@ -1,5 +1,6 @@ = Creating VMs -:page-parent: Reference +:page-parent: Getting Started +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/getting-started/index.adoc b/Documentation/getting-started/index.adoc new file mode 100644 index 0000000..10dde7f --- /dev/null +++ b/Documentation/getting-started/index.adoc @@ -0,0 +1,13 @@ += Getting Started +:description: Exploring Spectrum OS. Using (=How-To-Guides), Configuring (adding smth). +:page-nav_order: 3 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Ready to get started with Spectrum? Here is what you can do next: + +* xref:../getting-started/creating-vms.adoc[Create your own VM] to use others applications. +* xref:../getting-started/running-vms.adoc[Start some applications]. diff --git a/Documentation/running-vms.adoc b/Documentation/getting-started/running-vms.adoc similarity index 90% rename from Documentation/running-vms.adoc rename to Documentation/getting-started/running-vms.adoc index d0d3f99..ca70835 100644 --- a/Documentation/running-vms.adoc +++ b/Documentation/getting-started/running-vms.adoc @@ -1,5 +1,6 @@ = Running VMs -:page-parent: Reference +:page-parent: Getting Started +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/how-to.adoc b/Documentation/how-to.adoc deleted file mode 100644 index f43fa13..0000000 --- a/Documentation/how-to.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= How-to Guides -:page-has_children: true -:page-nav_order: 2 - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/index.adoc b/Documentation/index.adoc index 3079847..e0a45c7 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -1,13 +1,32 @@ -= Spectrum Docs += Spectrum Documentation :page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -Spectrum is a compartmentalized operating system. -If you'd like to try Spectrum, see xref:getting-spectrum.adoc[Getting -Spectrum]. +Spectrum is an open-source project that aims to create a computer operating +system, based on the principle of security by compartmentalization, that has a +lower barrier to entry and is easy to use and maintain. + +== Using Spectrum To learn about what Spectrum is and how it's implemented, start with -the xref:architecture.adoc[architecture overview]. +the xref:about/architecture.adoc[architecture overview]. + +If you want to try Spectrum, see xref:../installation/index.adoc[Build and Run] + to setup a development environment. + + +== Developing and Contributing + +Spectrum is made of free and open-source software. It is free for anyone to + use, modify, and distribute. + +Once you are up and running, see + xref:../development/index.adoc[Development] to understand how to work with + Spectrum's code and participate in the collaborative development process. + +If you are thinking of contributing to Spectrum docs, see + xref:../development/building-documentation.adoc[Building Documentation]. diff --git a/Documentation/binary-cache.adoc b/Documentation/installation/binary-cache.adoc similarity index 74% rename from Documentation/binary-cache.adoc rename to Documentation/installation/binary-cache.adoc index 6e69b39..ab53666 100644 --- a/Documentation/binary-cache.adoc +++ b/Documentation/installation/binary-cache.adoc @@ -1,5 +1,6 @@ -= Setting Up the Binary Cache -:page-parent: How-to Guides += Setting Up Binary Cache +:page-parent: Build and Run +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -20,10 +21,9 @@ encounter any trouble with it. The binary cache is currently not able to provide logs, due to a https://github.com/NixOS/nix/pull/6051[Nix bug]. -== On NixOS +== For NixOS -The following configuration adds the Spectrum binary cache as a -substituter, and tells Nix to trust builds signed with its public key. +Add the following configuration to /etc/nixos/configuration.nix: [source,nix] ---- @@ -38,7 +38,13 @@ substituter, and tells Nix to trust builds signed with its public key. } ---- -== On Non-NixOS systems +This configuration adds the Spectrum binary cache as a substituter and makes +Nix trust builds signed with its public key. + +To apply changes, rebuild your system with the https://nixos.wiki/wiki/Nixos-rebuild[nixos-rebuild] command. + + +== For Non-NixOS Systems Add the following configuration to /etc/nix/nix.conf: @@ -48,6 +54,10 @@ substituters = https://cache.dataaturservice.se/spectrum/ https://cache.nixos.or trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ= ---- +Run `systemctl restart nix-daemon.service` after making any changes in the Nix +configuration. After that the binary cache will be used automatically when +`nix-build` is run. + If you have other binary caches configured besides cache.nixos.org, make sure to add the Spectrum cache URL and public key to the existing configuration lines in your nix.conf file. diff --git a/Documentation/getting-spectrum.adoc b/Documentation/installation/getting-spectrum.adoc similarity index 79% rename from Documentation/getting-spectrum.adoc rename to Documentation/installation/getting-spectrum.adoc index b422a94..392cab5 100644 --- a/Documentation/getting-spectrum.adoc +++ b/Documentation/installation/getting-spectrum.adoc @@ -1,5 +1,6 @@ = Getting Spectrum -:page-parent: Tutorials +:page-parent: Build and Run +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -8,7 +9,7 @@ To get Spectrum, you need to build it from source. As long as you're running Linuxfootnote:[Building from other operating systems might work, but hasn't been tested. Patches are welcome to support building from other operating systems, as long as they're not too invasive.] -and have https://nixos.org/[Nix] and https://git-scm.com/[git] +and have https://nixos.org/download.html[Nix] and https://git-scm.com/[git] installed, it's very easy to do. TIP: Before you start, consider setting up the @@ -34,7 +35,7 @@ nix-shell -I nixpkgs=../../../nixpkgs-spectrum --run 'make run' This builds just enough of Spectrum to try it out in a VM, but it will still take a very long time. -== Building an installer +== Building Installer [source,shell] ---- @@ -47,8 +48,7 @@ If you haven't set up the xref:binary-cache.adoc[binary cache], this will take a very long time. When it's done, a symbolic link named "result" will appear, pointing to a Spectrum USB installer image. -CAUTION: Spectrum is not yet suitable for real-world use. Do not use -your Spectrum system for anything important or sensitive. Spectrum is -currently missing many important security properties, and there is no -procedure for updating to new versions of Spectrum -- you have to -reinstall. +CAUTION: Do not use Spectrum for anything important or sensitive as it is not +yet suitable for real-world use. Many important security properties are +currently missing, and there is no procedure for updating to +new versions—you have to reinstall the OS. diff --git a/Documentation/installation/index.adoc b/Documentation/installation/index.adoc new file mode 100644 index 0000000..d67c88d --- /dev/null +++ b/Documentation/installation/index.adoc @@ -0,0 +1,23 @@ += Build and Run +:description: How to download and install Spectrum OS. +:page-nav_order: 2 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + + +To start working with Spectrum, you need: + +* xref:../installation/binary-cache.adoc[Set up the binary cache] to speed up the build process. +* xref:../installation/getting-spectrum.adoc[Build Spectrum] from the source. + +TIP: At this stage, Spectrum works only on x86-64. AArch64 support is in +development. + +== Uninstalling and Updating + +Currently, there is no implementation for a software update. + +You can replace Spectrum by installing another OS. diff --git a/Documentation/reference.adoc b/Documentation/reference.adoc deleted file mode 100644 index 44b359d..0000000 --- a/Documentation/reference.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Reference -:page-has_children: true -:page-nav_order: 3 - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/tutorials.adoc b/Documentation/tutorials.adoc deleted file mode 100644 index cd1fb12..0000000 --- a/Documentation/tutorials.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Tutorials -:page-nav_order: 1 -:page-has_children: true - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/user-partition.adoc b/Documentation/user-partition.adoc deleted file mode 100644 index 73bc0d0..0000000 --- a/Documentation/user-partition.adoc +++ /dev/null @@ -1,12 +0,0 @@ -= The User Partition -:page-parent: Explanation - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -The Spectrum host system is immutable, so configuration and user data -lives on a separate partition. The host system discovers the user -partition by looking for the special partition type -`9293e1ff-cee4-4658-88be-898ec863944f`. This behavior can be -overridden with the `ext` parameter on the host's kernel command line, -which works in a similar way to the standard Linux `root` parameter. -- 2.34.1

2 2

[PATCH] Docs: new structure
by Jenni Nikolaenko 08 Nov '22

08 Nov '22

Create separate folders for new parent pages, update Introduction page, remove a and the articles from titles, quick check text for simple english Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/{ => about}/architecture.adoc | 20 ++++--- Documentation/about/index.adoc | 30 +++++++++++ Documentation/building-documentation.adoc | 52 ------------------ .../decisions/001-host-update-mechanism.adoc | 4 +- .../decisions/002-install-options.adoc | 4 +- Documentation/decisions/003-partitioning.adoc | 2 +- .../004-data-at-rest-encryption.adoc | 4 +- .../005-virtual-machine-monitor.adoc | 2 +- .../decisions/006-drivers-on-host.adoc | 2 +- .../decisions/007-usb-virtual-machines.adoc | 2 +- ...008-inter-vm-communication-mechanisms.adoc | 2 +- Documentation/decisions/index.adoc | 2 +- .../build-configuration.adoc | 13 +++-- .../development/building-documentation.adoc | 53 +++++++++++++++++++ .../{ => development}/debugging.adoc | 12 ++--- .../{ => development}/first-patch.adoc | 32 +++++------ Documentation/development/index.adoc | 31 +++++++++++ Documentation/{ => development}/replying.adoc | 13 ++--- .../{ => development}/reviewing-patches.adoc | 4 +- .../{ => development}/testing-patches.adoc | 34 ++++++------ .../{ => development}/uuid-reference.adoc | 5 +- Documentation/explanation.adoc | 3 +- .../{ => getting-started}/creating-vms.adoc | 2 +- Documentation/getting-started/index.adoc | 14 +++++ .../{ => getting-started}/running-vms.adoc | 2 +- .../getting-started/user-partition.adoc | 16 ++++++ Documentation/how-to.adoc | 1 + Documentation/index.adoc | 29 ++++++++-- Documentation/{ => installation}/b4.adoc | 5 +- .../{ => installation}/binary-cache.adoc | 18 ++++--- .../{ => installation}/getting-spectrum.adoc | 14 ++--- Documentation/installation/index.adoc | 26 +++++++++ Documentation/reference.adoc | 3 +- Documentation/tutorials.adoc | 3 +- Documentation/user-partition.adoc | 12 ----- 35 files changed, 302 insertions(+), 169 deletions(-) rename Documentation/{ => about}/architecture.adoc (84%) create mode 100644 Documentation/about/index.adoc delete mode 100644 Documentation/building-documentation.adoc rename Documentation/{ => development}/build-configuration.adoc (73%) create mode 100644 Documentation/development/building-documentation.adoc rename Documentation/{ => development}/debugging.adoc (79%) rename Documentation/{ => development}/first-patch.adoc (80%) create mode 100644 Documentation/development/index.adoc rename Documentation/{ => development}/replying.adoc (68%) rename Documentation/{ => development}/reviewing-patches.adoc (88%) rename Documentation/{ => development}/testing-patches.adoc (81%) rename Documentation/{ => development}/uuid-reference.adoc (97%) rename Documentation/{ => getting-started}/creating-vms.adoc (98%) create mode 100644 Documentation/getting-started/index.adoc rename Documentation/{ => getting-started}/running-vms.adoc (93%) create mode 100644 Documentation/getting-started/user-partition.adoc rename Documentation/{ => installation}/b4.adoc (93%) rename Documentation/{ => installation}/binary-cache.adoc (80%) rename Documentation/{ => installation}/getting-spectrum.adoc (82%) create mode 100644 Documentation/installation/index.adoc delete mode 100644 Documentation/user-partition.adoc diff --git a/Documentation/architecture.adoc b/Documentation/about/architecture.adoc similarity index 84% rename from Documentation/architecture.adoc rename to Documentation/about/architecture.adoc index 1c4307b..c1af320 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/about/architecture.adoc @@ -1,17 +1,15 @@ = Architecture -:page-parent: Explanation +:page-parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Introduction +Spectrum is based on the principle of security by compartmentalization. -Spectrum is based on the principle of security by -compartmentalization. The high level stack is illustrated in the -following diagram: +The high level stack is illustrated in the following diagram: -image::diagrams/stack.svg[] +image::../diagrams/stack.svg[] The default set of virtual machines includes two application VMs, _appvm-catgirl_ (an IRC client) and _appvm-lynx_ (a text-based web @@ -26,7 +24,7 @@ https://en.wikipedia.org/wiki/Architectural_decision[Architecturally significant decisions] are xref:decisions/index.adoc[recorded] as lightweight https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions/[A…. -== The Spectrum host system +== Spectrum Host System Compartmentalization is implemented using https://cloud-hypervisor.org/[cloud-hypervisor] virtual machines. @@ -35,7 +33,7 @@ https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[Kernel-based Virtual Machine] (KVM) to provide lightweight, hardware-accelerated VMs. While Linux (including KVM) is portable between many hardware architectures, -cloud-hypervisor supports only x86_64 and aarch64. Spectrum currently only +cloud-hypervisor supports only x86_64 and aarch64. Spectrum currently only works on x86_64, but aarch64 support is in development. https://skarnet.org/software/s6-rc/overview.html[s6-rc] is used for service @@ -44,7 +42,7 @@ and service scripts. https://wayland.freedesktop.org/[Wayland] is used for window management and display. The Wayland architecture is well documented -https://wayland.freedesktop.org/architecture.html[here]. The host provides only +https://wayland.freedesktop.org/architecture.html[here]. The host provides only a Wayland terminal client, https://codeberg.org/dnkl/foot/[foot], which is used for interacting with VM consoles. In future it will be possible for application VMs to display windows on the single Wayland compositor on the host system, @@ -57,7 +55,7 @@ https://www.etalabs.net/compare_libcs.html[added safety on resource exhaustion and security hardening on memory allocation]. Kernel hardening will be investigated in future. -== Exploring the Spectrum dependency tree +== Spectrum Dependency Tree For a detailed, interactive view of dependencies, use https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: @@ -67,4 +65,4 @@ https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[See video of Spectrum live image -interactive analysis with nix-tree] +interactive analysis with nix-tree]. diff --git a/Documentation/about/index.adoc b/Documentation/about/index.adoc new file mode 100644 index 0000000..9e2a47e --- /dev/null +++ b/Documentation/about/index.adoc @@ -0,0 +1,30 @@ += About Spectrum +:description: Some words about Spectrum as the operating system, not a project. Highlights the differences between common Linux distributions and Spectrum. +:page-nav_order: 1 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is a Linux-based system that uses Nix package manager and Nixpkgs +collection of packages. + +This gives an actively-developed base with good +hardware support, powerful and optimised compartmentalization primitives in +KVM, and the reproducible packaging and configuration system that is important +for a maintainable compartmentalized system. + +== Why Spectrum + +* User data and applications are managed centrally while remaining isolated. +That means that the system can be backed up and managed as a whole, rather than +mixed up in several dozen VMs. + +* The host system and isolated environments are managed declaratively and +reproducibly using the Nix package manager. +This can save the user the burden of maintaining many different virtual +computers, allowing finer-grained resource access controls and making it +possible to verify the software running across all environments. + +TIP: If you are interested in why we do something _this_ way instead of _that_ +way, see xref:../decisions/index.adoc[Architecture Decision Records]. diff --git a/Documentation/building-documentation.adoc b/Documentation/building-documentation.adoc deleted file mode 100644 index b491105..0000000 --- a/Documentation/building-documentation.adoc +++ /dev/null @@ -1,52 +0,0 @@ -= Building the Documentation -:page-parent: Tutorials - -// SPDX-FileCopyrightText: 2022 Unikie -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -This tutorial assumes that you have https://nixos.org/[Nix] installed. -You may also want to xref:binary-cache.adoc[configure the Spectrum -binary cache], to avoid having to wait for dependencies to compile on -your local system. - -1. Get a copy of the Spectrum source code: -+ -[source,shell] ----- -git clone https://spectrum-os.org/git/spectrum ----- -2. Enter the documentation directory: -+ -[source,shell] ----- -cd spectrum/Documentation ----- -3. Enter the development environment: -+ -[source,shell] ----- -nix-shell -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz ----- -4. In the development shell, do an initial build of the documentation -site: -+ -[source,shell] ----- -scripts/build.sh ----- -5. Run a development server for previewing changes locally: -+ -[source,shell] ----- -jekyll serve ----- -+ -This will serve a local copy of the documentation at http://localhost:4000/. -+ -IMPORTANT: Jekyll doesn't handle rendering of the draw.io diagrams, so -if you modify any of those, or add new ones, you'll have to run -`scripts/build.sh` again to do a full rebuild of the site. - -Once you've made your changes to the documentation, see -xref:first-patch.adoc[Sending Your First Patch] for information -about how to submit them for review. diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc index 574deb4..39f9f28 100644 --- a/Documentation/decisions/001-host-update-mechanism.adoc +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -1,6 +1,6 @@ = 001 Host Update Mechanism :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -11,7 +11,7 @@ Proposed == Context -Spectrum currently has no implementation for software update. The host -- +Spectrum currently has no implementation for software update. The host -- consisting of the Linux kernel, KVM, cloud-hypervisor and minimal user space tools -- will require software updates to support feature development and security fixes. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc index 4412b53..4a745eb 100644 --- a/Documentation/decisions/002-install-options.adoc +++ b/Documentation/decisions/002-install-options.adoc @@ -1,6 +1,6 @@ -= 002 Install options += 002 Install Options :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc index 8494ea4..a13b8cc 100644 --- a/Documentation/decisions/003-partitioning.adoc +++ b/Documentation/decisions/003-partitioning.adoc @@ -1,6 +1,6 @@ = 003 Partitioning :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc index 26fe273..5b0f518 100644 --- a/Documentation/decisions/004-data-at-rest-encryption.adoc +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -1,6 +1,6 @@ = 004 Data at Rest Encryption :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -20,5 +20,5 @@ User data is encrypted. == Consequences Spectrum needs to come with enough software to get the encryption key -via different methods (password, usb, fido, etc.) Can we use dm-crypt +via different methods (password, usb, fido, etc.). Can we use dm-crypt for everything instead of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-monitor.adoc b/Documentation/decisions/005-virtual-machine-monitor.adoc index db81c72..df1b501 100644 --- a/Documentation/decisions/005-virtual-machine-monitor.adoc +++ b/Documentation/decisions/005-virtual-machine-monitor.adoc @@ -1,6 +1,6 @@ = 005 Virtual Machine Monitor :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc index 872044e..b92d863 100644 --- a/Documentation/decisions/006-drivers-on-host.adoc +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -1,6 +1,6 @@ = 006 Drivers on Host :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/007-usb-virtual-machines.adoc b/Documentation/decisions/007-usb-virtual-machines.adoc index 3bdf78b..d832691 100644 --- a/Documentation/decisions/007-usb-virtual-machines.adoc +++ b/Documentation/decisions/007-usb-virtual-machines.adoc @@ -1,6 +1,6 @@ = 007 USB Virtual Machine :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc index a1b7d49..c1e5b87 100644 --- a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc +++ b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc @@ -1,6 +1,6 @@ = 008 Inter-VM Communication Mechanisms :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/index.adoc b/Documentation/decisions/index.adoc index 772f382..a022239 100644 --- a/Documentation/decisions/index.adoc +++ b/Documentation/decisions/index.adoc @@ -1,6 +1,6 @@ = Architecture Decision Records :page-has_children: true -:page-parent: Explanation +:page-parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/build-configuration.adoc b/Documentation/development/build-configuration.adoc similarity index 73% rename from Documentation/build-configuration.adoc rename to Documentation/development/build-configuration.adoc index b89575f..c9a8c99 100644 --- a/Documentation/build-configuration.adoc +++ b/Documentation/development/build-configuration.adoc @@ -1,19 +1,22 @@ = Configuring the Build -:page-parent: How-to Guides +:page-parent: Development +:page-nav_order: 1 :example-caption: Test // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 Some aspects of a Spectrum build can be customised using a build -configuration file. By default, this configuration file should be -called config.nix and located in the root of the Spectrum source tree, -but this can be overridden by setting `spectrum-config` in the +configuration file. + +By default, this configuration file should be called config.nix and located in +the root of the Spectrum source tree, but this can be overridden by setting +`spectrum-config` in the https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PAT… to the path of the configuration file. The configuration file should contain an attribute set. The only -currently allowed attribute name is `pkgs`, which allows using a +currently allowed attribute name is `pkgs`. It allows using a custom Nixpkgs to evaluate Spectrum. .config.nix to build Spectrum with a https://nixos.org/manual/nixpkgs/unstable/#sec-overlays-definition[Nixpkgs overlay] diff --git a/Documentation/development/building-documentation.adoc b/Documentation/development/building-documentation.adoc new file mode 100644 index 0000000..d83c343 --- /dev/null +++ b/Documentation/development/building-documentation.adoc @@ -0,0 +1,53 @@ += Building Documentation +:page-parent: Development +:page-nav_order: 4 + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Make sure you have https://nixos.org/[Nix] installed. +You may also want to xref:binary-cache.adoc[configure the Spectrum +binary cache], to avoid having to wait for dependencies to compile on +your local system. + +. Get a copy of the Spectrum source code: ++ +[source,shell] +---- +git clone https://spectrum-os.org/git/spectrum +---- +. Enter the documentation directory: ++ +[source,shell] +---- +cd spectrum/Documentation +---- +. Enter the development environment: ++ +[source,shell] +---- +nix-shell -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz +---- +. In the development shell, do an initial build of the documentation +site: ++ +[source,shell] +---- +scripts/build.sh +---- +. Run a development server for previewing changes locally: ++ +[source,shell] +---- +jekyll serve +---- ++ +This will serve a local copy of the documentation at http://localhost:4000/. ++ +IMPORTANT: Jekyll does not handle rendering of the draw.io diagrams. If you +modify any of those, or add new ones, run `scripts/build.sh` again to do a full +rebuild of the site. + +After making changes to the documentation, see +xref:first-patch.adoc[Working with Patches] for information +on how to submit your changes for review. diff --git a/Documentation/debugging.adoc b/Documentation/development/debugging.adoc similarity index 79% rename from Documentation/debugging.adoc rename to Documentation/development/debugging.adoc index 3871a7c..d342294 100644 --- a/Documentation/debugging.adoc +++ b/Documentation/development/debugging.adoc @@ -1,7 +1,6 @@ -= Debugging Spectrum -:page-parent: Explanation -:toc: -:toclevels: 1 += Debugging +:page-parent: Development +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -9,8 +8,9 @@ == Extracting core dumps when running Spectrum in a VM When using a VM to run the Spectrum host system, a special mechanism -is available to enable easy introspection of core files. When a -program on the Spectrum host system dumps core, the system will +is available to enable easy introspection of core files. + +When a program on the Spectrum host system dumps core, the system will attempt to upload the core file to _its_ host (i.e. the system running Spectrum in a VM) using the vsock(7) protocol, on port 1129271877. diff --git a/Documentation/first-patch.adoc b/Documentation/development/first-patch.adoc similarity index 80% rename from Documentation/first-patch.adoc rename to Documentation/development/first-patch.adoc index 30672b9..cf19b37 100644 --- a/Documentation/first-patch.adoc +++ b/Documentation/development/first-patch.adoc @@ -1,11 +1,11 @@ -= Sending Your First Patch -:page-parent: Tutorials += Working with Patches +:page-parent: Development +:page-nav_order: 1 +:page-has_children: true // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Prerequisites - This tutorial assumes that you already have basic https://git-scm.com/[git] experience. @@ -14,19 +14,18 @@ https://spectrum-os.org/git/[Spectrum source tree]. You'll also need to have configured `git send-email` — a guide for this can be found at https://git-send-email.io/. -== Making your changes +== Making Changes -If you've worked on any git repository before, the process for making -your changes will probably be very familiar. +The process of making changes is similar to working on any git repository. -1. Create a branch for your changes: +. Create a branch for your changes: + [source,shell] ---- git checkout -b fix-docs # for example ---- -2. Make changes in your editor. -3. Stage and commit your changes: +. Make changes in your editor. +. Stage and commit your changes: + [source,shell] ---- @@ -39,7 +38,7 @@ indicates your acceptance of the https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin], which is mandatory for Spectrum patches. -== Submitting changes +== Submitting Changes Once you're happy with how the commits on your branch look, run: @@ -64,13 +63,14 @@ message that will be sent before all of your patches. Once your patch has been submitted, wait for it to be reviewed. Feedback, if any, will be sent as email replies to your submitted -patch. You can respond to feedback in your mail client. Please use -the Reply All button to ensure that your messages are sent to the +patch. You can respond to feedback in your mail client. + +Use the *Reply All* button to sent your messages to the mailing list as well as to the person who sent the feedback. -If you need to make changes to your patch, and submit a new version, +If you need to make changes to your patch and submit a new version, use https://git-rebase.io/[`git rebase`] to create a new version of -your patch(es), and submit it like this: +your patch(es) and then submit it like this: [source,shell] ---- @@ -81,7 +81,7 @@ The added `-v2` flag indicates that this is version two of your patch set. If your patches require more rounds of changes, submit subsequent rounds with `-v3`, `-v4`, etc. as appropriate. -If you'd like to describe what has changed from the previous version +If you would like to describe what has changed from the previous version of your patches, you can do so in a xref:cover-letter[cover letter] as described above. diff --git a/Documentation/development/index.adoc b/Documentation/development/index.adoc new file mode 100644 index 0000000..1fe4d8b --- /dev/null +++ b/Documentation/development/index.adoc @@ -0,0 +1,31 @@ += Development +:description: Development progress, general development practices +:page-nav_order: 4 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is free software, currently under active development. + +If you made changes in a source code, +https://spectrum-os.org/installation/getting-spectrum.html#building-an-installer[rebuild Spectrum], +xref:../development/testing-patches.adoc[test your patch] and +then https://spectrum-os.org/development/first-patch.html#submitting-changes[sub… your changes for review]. + +== Developer Setup + +Before starting, make sure you are familiar with +https://git.kernel.org/pub/scm/utils/b4/b4.git/about/[b4] and the +https://nixos.org/manual/nix/stable/introduction.html[Nix package manager]. + +== Mailing Lists + +The Spectrum project runs several +https://spectrum-os.org/mailman3/lists/?all-lists[mailing lists] on which you +can ask your questions or help other people with the questions they have. All +the Spectrum developers as well as many long time Linux and Spectrum users are +on the lists. + +For real-time feedback, use +https://spectrum-os.org/participating.html#irc[IRC/Matrix channel]. diff --git a/Documentation/replying.adoc b/Documentation/development/replying.adoc similarity index 68% rename from Documentation/replying.adoc rename to Documentation/development/replying.adoc index bb8e31a..a1ad394 100644 --- a/Documentation/replying.adoc +++ b/Documentation/development/replying.adoc @@ -1,5 +1,7 @@ = Replying to Messages in the Mailing List Archives -:page-parent: Tutorials +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -9,14 +11,13 @@ https://spectrum-os.org/participating.html#mailing-lists[mailing lists]. Make sure to Reply All when replying to messages, so that the message -is sent to the mailing list and not just to the person you're -replying to. +is sent to the mailing list and not just to the person you reply to. == Getting a copy of a message -You may want to reply to a mailing list message that you don't have a -copy of in your mail client, because you aren't subscribed to the -list, or because you subscribed after the message was sent. +You may want to reply to a mailing list message that you do not have a +copy of in your mail client, because you are not subscribed to the +list or because you subscribed after the message was sent. To do this, find the message you want to reply to in the https://spectrum-os.org/lists/archives[public-inbox list archives], diff --git a/Documentation/reviewing-patches.adoc b/Documentation/development/reviewing-patches.adoc similarity index 88% rename from Documentation/reviewing-patches.adoc rename to Documentation/development/reviewing-patches.adoc index 63ff24e..ba47f25 100644 --- a/Documentation/reviewing-patches.adoc +++ b/Documentation/development/reviewing-patches.adoc @@ -1,5 +1,7 @@ = Reviewing Patches -:page-parent: How-to Guides +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/testing-patches.adoc b/Documentation/development/testing-patches.adoc similarity index 81% rename from Documentation/testing-patches.adoc rename to Documentation/development/testing-patches.adoc index 743cd6e..600cf0f 100644 --- a/Documentation/testing-patches.adoc +++ b/Documentation/development/testing-patches.adoc @@ -1,5 +1,7 @@ = Testing Patches -:page-parent: How-to Guides +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-FileCopyrightText: 2022 Unikie @@ -9,7 +11,7 @@ Potential changes to Spectrum are posted to and discussed on the https://spectrum-os.org/participating.html#spectrum-devel[devel@spectrum-os… mailing list. -== Apply the patch +== Apply Patch . Find the patch series you want to test on https://spectrum-os.org/lists/archives/spectrum-devel/[public-inbox]. @@ -28,35 +30,31 @@ of the Spectrum root's nix-shell, which allows you to skip this step. with the patch's Message-Id to download all the patches in the series into a file. + -[example] [source,shell] ---- b4 am 20220511092352.70E54C980(a)atuin.qyliss.net ---- - -. b4 will indicate the file name it has downloaded the patches into - with a line like: +b4 will indicate the file name it has downloaded the patches into with a line +like: + -[example] -[listing] +[source,shell] +---- Writing ./20220424_hi_host_rootfs_fix_weston_hotplugging.mbx +---- +. Run `git am` on that file to apply the patches. For example: + -Run `git am` on that file to apply the patches, for example: -+ -[example] [source,shell] ---- git am 20220424_hi_host_rootfs_fix_weston_hotplugging.mbx ---- -== Post your test results +== Post Your Results -When you've tested a patch, it's really helpful to +When you tested a patch, it is helpful to xref:replying.adoc[reply] with your test results. -If the patch worked for you, please reply to it and include a line -like the following, separated from any reply text: - +If the patch worked for you, please reply to it and include a line like the following, separated from any reply text: +[source,shell] ---- Tested-by: John Smith <john(a)example.com> ---- @@ -66,10 +64,10 @@ patch replies will be automatically included in the commit message when a patch is applied. It's also helpful to explain in your reply how you tested the patch, -but you don't have to if it's obvious. (For example, if a patch is +but you don't have to if it's obvious. For example, if a patch is supposed to fix a bug, and you verified that after applying the patch the bug is fixed, just the Tested-by line on its own is enough to -indicate that.) +indicate that. If you found an issue with the patch, do not include a Tested-by line, and instead reply to the patch explaining what you tested, what you diff --git a/Documentation/uuid-reference.adoc b/Documentation/development/uuid-reference.adoc similarity index 97% rename from Documentation/uuid-reference.adoc rename to Documentation/development/uuid-reference.adoc index 4b0b481..61c602f 100644 --- a/Documentation/uuid-reference.adoc +++ b/Documentation/development/uuid-reference.adoc @@ -1,7 +1,8 @@ = UUID Reference -:page-parent: Reference +:page-parent: Development :toc: preamble :toclevels: 1 +:page-nav_order: 5 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -40,7 +41,7 @@ xref:user-partition.adoc[Spectrum user partition]. === `56a3bbc3-aefa-43d9-a64d-7b3fd59bbc4e` -https://github.com/endlessm/eos-installer["eosimages"] partition on the +https://github.com/endlessm/eos-installer["eosimages"] partition on the Spectrum combined live system / installer image. == Combined Image Partition IDs diff --git a/Documentation/explanation.adoc b/Documentation/explanation.adoc index b39cc6d..f682129 100644 --- a/Documentation/explanation.adoc +++ b/Documentation/explanation.adoc @@ -1,6 +1,5 @@ = Explanation -:page-has_children: true -:page-nav_order: 4 +:page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/creating-vms.adoc b/Documentation/getting-started/creating-vms.adoc similarity index 98% rename from Documentation/creating-vms.adoc rename to Documentation/getting-started/creating-vms.adoc index d967098..e06be85 100644 --- a/Documentation/creating-vms.adoc +++ b/Documentation/getting-started/creating-vms.adoc @@ -1,5 +1,5 @@ = Creating VMs -:page-parent: Reference +:page-parent: Getting Started // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/getting-started/index.adoc b/Documentation/getting-started/index.adoc new file mode 100644 index 0000000..639e0a6 --- /dev/null +++ b/Documentation/getting-started/index.adoc @@ -0,0 +1,14 @@ += Getting Started +:description: Exploring Spectrum OS. Using (=How-To-Guides), Configuring (adding smth). Ready to get started with Spectrum OS? After installing you can create VMs and then configure each one. +:page-nav_order: 3 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Ready to get started with Spectrum? Here is what you can do next: + +* xref:../getting-started/creating-vms.adoc[Create your own VM] to use others applications. +* xref:../getting-started/running-vms.adoc[Start some applications]. +* If needed, xref:../getting-started/user-partition.adoc[change the user partition type]. diff --git a/Documentation/running-vms.adoc b/Documentation/getting-started/running-vms.adoc similarity index 93% rename from Documentation/running-vms.adoc rename to Documentation/getting-started/running-vms.adoc index d0d3f99..9073e3c 100644 --- a/Documentation/running-vms.adoc +++ b/Documentation/getting-started/running-vms.adoc @@ -1,5 +1,5 @@ = Running VMs -:page-parent: Reference +:page-parent: Getting Started // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/getting-started/user-partition.adoc b/Documentation/getting-started/user-partition.adoc new file mode 100644 index 0000000..7c142b8 --- /dev/null +++ b/Documentation/getting-started/user-partition.adoc @@ -0,0 +1,16 @@ += User Partition +:page-parent: Getting Started + +// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +The Spectrum host system is immutable, so configuration and user data +lives on a separate partition. + +The host system discovers the user +partition by looking for the special partition type +`9293e1ff-cee4-4658-88be-898ec863944f`. + +This behavior can be overridden with the `ext` parameter on the host's kernel +command line, which works in a similar way to the standard Linux `root` +parameter. diff --git a/Documentation/how-to.adoc b/Documentation/how-to.adoc index f43fa13..98cc842 100644 --- a/Documentation/how-to.adoc +++ b/Documentation/how-to.adoc @@ -1,4 +1,5 @@ = How-to Guides +:page-nav_exclude: true :page-has_children: true :page-nav_order: 2 diff --git a/Documentation/index.adoc b/Documentation/index.adoc index 3079847..ea28533 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -1,13 +1,32 @@ -= Spectrum Docs += Spectrum Documentation :page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -Spectrum is a compartmentalized operating system. -If you'd like to try Spectrum, see xref:getting-spectrum.adoc[Getting -Spectrum]. +Spectrum is an open-source project that aims to create a computer operating +system, based on the principle of security by compartmentalization, that has a +lower barrier to entry and is easy to use and maintain. + +== Using Spectrum To learn about what Spectrum is and how it's implemented, start with -the xref:architecture.adoc[architecture overview]. +the xref:about/architecture.adoc[architecture overview]. + +If you want to try Spectrum, see xref:../installation/index.adoc[Build and Run] + to setup a development environment. + + +== Developing and Contributing + +Spectrum is made of free and open-source software. It is free for anyone to + use, modify, and distribute. + +Once you are up and rinning, see + xref:../development/index.adoc[Development] to understand how to work with + patches, debug the system or build the documentation. + +If you are thinking of contributing to Spectrum docs, see + xref:../development/building-documentation.adoc[Building Documentation]. diff --git a/Documentation/b4.adoc b/Documentation/installation/b4.adoc similarity index 93% rename from Documentation/b4.adoc rename to Documentation/installation/b4.adoc index 489ced4..58efd0e 100644 --- a/Documentation/b4.adoc +++ b/Documentation/installation/b4.adoc @@ -1,5 +1,6 @@ = Installing and Configuring b4 -:page-parent: Tutorials +:page-parent: Build and Run +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-FileCopyrightText: 2022 Unikie @@ -19,7 +20,7 @@ of the Spectrum root's nix-shell. You should be able to install b4 from your package manager. -Using Nix, you can start a shell with b4 available by running +Using Nix, you can start a shell with b4 available by running: [listing] [source,shell] diff --git a/Documentation/binary-cache.adoc b/Documentation/installation/binary-cache.adoc similarity index 80% rename from Documentation/binary-cache.adoc rename to Documentation/installation/binary-cache.adoc index 6e69b39..dff12e1 100644 --- a/Documentation/binary-cache.adoc +++ b/Documentation/installation/binary-cache.adoc @@ -1,5 +1,6 @@ -= Setting Up the Binary Cache -:page-parent: How-to Guides += Setting Up Binary Cache +:page-parent: Build and Run +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -20,10 +21,9 @@ encounter any trouble with it. The binary cache is currently not able to provide logs, due to a https://github.com/NixOS/nix/pull/6051[Nix bug]. -== On NixOS +== For NixOS -The following configuration adds the Spectrum binary cache as a -substituter, and tells Nix to trust builds signed with its public key. +Add the following configuration to /etc/nixos/configuration.nix: [source,nix] ---- @@ -38,7 +38,13 @@ substituter, and tells Nix to trust builds signed with its public key. } ---- -== On Non-NixOS systems +This configuration adds the Spectrum binary cache as a substituter and makes +Nix to trust builds signed with its public key. + +To apply changes, rebuild your system with the https://nixos.wiki/wiki/Nixos-rebuild[nixos-rebuild] command. + + +== For Non-NixOS Systems Add the following configuration to /etc/nix/nix.conf: diff --git a/Documentation/getting-spectrum.adoc b/Documentation/installation/getting-spectrum.adoc similarity index 82% rename from Documentation/getting-spectrum.adoc rename to Documentation/installation/getting-spectrum.adoc index b3fa1ef..c365a93 100644 --- a/Documentation/getting-spectrum.adoc +++ b/Documentation/installation/getting-spectrum.adoc @@ -1,5 +1,6 @@ = Getting Spectrum -:page-parent: Tutorials +:page-parent: Build and Run +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -34,7 +35,7 @@ nix-shell -I nixpkgs=../../../nixpkgs-spectrum --run 'make run' This builds just enough of Spectrum to try it out in a VM, but it will still take a very long time. -== Building an installer +== Building Installer [source,shell] ---- @@ -47,8 +48,7 @@ This will take a very long time, but when it's done, a symbolic link named "result" will appear, pointing to a Spectrum USB installer image. -CAUTION: Spectrum is not yet suitable for real-world use. Do not use -your Spectrum system for anything important or sensitive. Spectrum is -currently missing many important security properties, and there is no -procedure for updating to new versions of Spectrum -- you have to -reinstall. +CAUTION: Do not use Spectrum for anything important or sensitive as it is not +yet suitable for real-world use. Many important security properties are +currently missing, and there is no procedure for updating to +new versions—you have to reinstall the OS. diff --git a/Documentation/installation/index.adoc b/Documentation/installation/index.adoc new file mode 100644 index 0000000..2c4e9c7 --- /dev/null +++ b/Documentation/installation/index.adoc @@ -0,0 +1,26 @@ += Build and Run +:description: How to download and install Spectrum OS. +:page-nav_order: 2 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + + +To start working with Spectrum, you need: + +* xref:../installation/binary-cache.adoc[Set up the binary cache] to speed up the build process. +* xref:../installation/getting-spectrum.adoc[Build Spectrum] from the source. +* xref:../installation/b4.adoc[Install and configure the b4 utility] to be able to work with patches. + +TIP: Currently, Spectrum works only on x86-64. AAarch64 support is in +development. If you have the configuration layer, you can +xref:../development/build-configuration.adoc[configure the build] to be able to +work with specific devices. + +== Uninstalling and Updating + +Currently, there is no implementation for a software update. + +You can replace Spectrum by installing another OS. diff --git a/Documentation/reference.adoc b/Documentation/reference.adoc index 44b359d..55259ea 100644 --- a/Documentation/reference.adoc +++ b/Documentation/reference.adoc @@ -1,6 +1,5 @@ = Reference -:page-has_children: true -:page-nav_order: 3 +:page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/tutorials.adoc b/Documentation/tutorials.adoc index cd1fb12..fcef31b 100644 --- a/Documentation/tutorials.adoc +++ b/Documentation/tutorials.adoc @@ -1,6 +1,5 @@ = Tutorials -:page-nav_order: 1 -:page-has_children: true +:page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/user-partition.adoc b/Documentation/user-partition.adoc deleted file mode 100644 index 73bc0d0..0000000 --- a/Documentation/user-partition.adoc +++ /dev/null @@ -1,12 +0,0 @@ -= The User Partition -:page-parent: Explanation - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -The Spectrum host system is immutable, so configuration and user data -lives on a separate partition. The host system discovers the user -partition by looking for the special partition type -`9293e1ff-cee4-4658-88be-898ec863944f`. This behavior can be -overridden with the `ext` parameter on the host's kernel command line, -which works in a similar way to the standard Linux `root` parameter. -- 2.34.1

3 6

[PATCH v3] Docs: new structure
by Jenni Nikolaenko 08 Nov '22

08 Nov '22

Create separate folders for new parent pages, update Introduction page, remove a and the articles from titles, quick check text for simple english Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/{ => about}/architecture.adoc | 27 ++++----- Documentation/about/index.adoc | 32 ++++++++++ Documentation/building-documentation.adoc | 52 ---------------- .../decisions/001-host-update-mechanism.adoc | 4 +- .../decisions/002-install-options.adoc | 4 +- Documentation/decisions/003-partitioning.adoc | 2 +- .../004-data-at-rest-encryption.adoc | 4 +- .../005-virtual-machine-monitor.adoc | 2 +- .../decisions/006-drivers-on-host.adoc | 2 +- .../decisions/007-usb-virtual-machines.adoc | 2 +- ...008-inter-vm-communication-mechanisms.adoc | 2 +- Documentation/decisions/index.adoc | 2 +- Documentation/{ => development}/b4.adoc | 6 +- .../build-configuration.adoc | 13 ++-- .../development/building-documentation.adoc | 52 ++++++++++++++++ .../{ => development}/creating-vms.adoc | 4 +- .../{ => development}/debugging.adoc | 12 ++-- .../{ => development}/first-patch.adoc | 32 +++++----- Documentation/development/index.adoc | 59 +++++++++++++++++++ Documentation/development/managing-vms.adoc | 14 +++++ Documentation/{ => development}/replying.adoc | 13 ++-- .../{ => development}/reviewing-patches.adoc | 4 +- .../{ => development}/running-vms.adoc | 4 +- .../{ => development}/testing-patches.adoc | 28 ++++----- Documentation/development/user-partition.adoc | 18 ++++++ .../{ => development}/uuid-reference.adoc | 5 +- .../development/working-with-patces.adoc | 12 ++++ Documentation/explanation.adoc | 6 -- Documentation/how-to.adoc | 6 -- Documentation/index.adoc | 29 +++++++-- .../{ => installation}/binary-cache.adoc | 22 +++++-- .../{ => installation}/getting-spectrum.adoc | 16 ++--- Documentation/installation/index.adoc | 23 ++++++++ Documentation/reference.adoc | 6 -- Documentation/tutorials.adoc | 6 -- Documentation/user-partition.adoc | 12 ---- 36 files changed, 350 insertions(+), 187 deletions(-) rename Documentation/{ => about}/architecture.adoc (79%) create mode 100644 Documentation/about/index.adoc delete mode 100644 Documentation/building-documentation.adoc rename Documentation/{ => development}/b4.adoc (91%) rename Documentation/{ => development}/build-configuration.adoc (73%) create mode 100644 Documentation/development/building-documentation.adoc rename Documentation/{ => development}/creating-vms.adoc (97%) rename Documentation/{ => development}/debugging.adoc (79%) rename Documentation/{ => development}/first-patch.adoc (80%) create mode 100644 Documentation/development/index.adoc create mode 100644 Documentation/development/managing-vms.adoc rename Documentation/{ => development}/replying.adoc (68%) rename Documentation/{ => development}/reviewing-patches.adoc (88%) rename Documentation/{ => development}/running-vms.adoc (85%) rename Documentation/{ => development}/testing-patches.adoc (82%) create mode 100644 Documentation/development/user-partition.adoc rename Documentation/{ => development}/uuid-reference.adoc (97%) create mode 100644 Documentation/development/working-with-patces.adoc delete mode 100644 Documentation/explanation.adoc delete mode 100644 Documentation/how-to.adoc rename Documentation/{ => installation}/binary-cache.adoc (74%) rename Documentation/{ => installation}/getting-spectrum.adoc (79%) create mode 100644 Documentation/installation/index.adoc delete mode 100644 Documentation/reference.adoc delete mode 100644 Documentation/tutorials.adoc delete mode 100644 Documentation/user-partition.adoc diff --git a/Documentation/architecture.adoc b/Documentation/about/architecture.adoc similarity index 79% rename from Documentation/architecture.adoc rename to Documentation/about/architecture.adoc index 1c4307b..1237577 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/about/architecture.adoc @@ -1,32 +1,31 @@ = Architecture -:page-parent: Explanation +:page-parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Introduction +Spectrum is based on the principle of security by compartmentalization. -Spectrum is based on the principle of security by -compartmentalization. The high level stack is illustrated in the -following diagram: +The high level stack is illustrated in the following diagram: -image::diagrams/stack.svg[] +image::../diagrams/stack.svg[] The default set of virtual machines includes two application VMs, _appvm-catgirl_ (an IRC client) and _appvm-lynx_ (a text-based web browser); and a system VM, _netvm_ (which handles hardware network devices and provides network services to application VMs). Refer to -xref:creating-vms.adoc[Creating VMs] and xref:running-vms.adoc[Running +xref:../development/creating-vms.adoc[Creating VMs] and +xref:../development/running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. == Architecture Decision Records (ADRs) https://en.wikipedia.org/wiki/Architectural_decision[Architecturally significant -decisions] are xref:decisions/index.adoc[recorded] as lightweight -https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions/[ADRs]. +decisions] are xref:../decisions/index.adoc[recorded] as lightweight +https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions[ADRs]. -== The Spectrum host system +== Spectrum Host System Compartmentalization is implemented using https://cloud-hypervisor.org/[cloud-hypervisor] virtual machines. @@ -44,7 +43,7 @@ and service scripts. https://wayland.freedesktop.org/[Wayland] is used for window management and display. The Wayland architecture is well documented -https://wayland.freedesktop.org/architecture.html[here]. The host provides only +https://wayland.freedesktop.org/architecture.html[here]. The host provides only a Wayland terminal client, https://codeberg.org/dnkl/foot/[foot], which is used for interacting with VM consoles. In future it will be possible for application VMs to display windows on the single Wayland compositor on the host system, @@ -57,7 +56,7 @@ https://www.etalabs.net/compare_libcs.html[added safety on resource exhaustion and security hardening on memory allocation]. Kernel hardening will be investigated in future. -== Exploring the Spectrum dependency tree +== Spectrum Dependency Tree For a detailed, interactive view of dependencies, use https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: @@ -66,5 +65,5 @@ https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: [listing] nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree -https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[See video of Spectrum live image -interactive analysis with nix-tree] +See the https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[video] of Spectrum live +image interactive analysis with nix-tree. diff --git a/Documentation/about/index.adoc b/Documentation/about/index.adoc new file mode 100644 index 0000000..6961b6a --- /dev/null +++ b/Documentation/about/index.adoc @@ -0,0 +1,32 @@ += About Spectrum +:description: Some words about Spectrum as the operating system, not a project. Highlights the differences between common Linux distributions and Spectrum. +:page-nav_order: 1 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is a Linux-based system that uses the +https://github.com/NixOS/nix[Nix package manager] and +the https://github.com/NixOS/nixpkgs[Nix Packages collection] (Nixpkgs). + +This gives an actively-developed base with good +hardware support, powerful and optimised compartmentalization primitives in +https://www.linux-kvm.org/page/Main_Page[KVM], and the reproducible packaging +and configuration system that is important +for a maintainable compartmentalized system. + +== Why Spectrum + +* User data and applications are managed centrally while remaining isolated. +That means that the system can be backed up and managed as a whole, rather than +mixed up in several dozen VMs. + +* The host system and isolated environments are managed declaratively and +reproducibly using the Nix package manager. +This can save the user the burden of maintaining many different virtual +computers, allowing finer-grained resource access controls and making it +possible to verify the software running across all environments. + +TIP: If you are interested in why we do something _this_ way instead of _that_ +way, see xref:../decisions/index.adoc[Architecture Decision Records]. diff --git a/Documentation/building-documentation.adoc b/Documentation/building-documentation.adoc deleted file mode 100644 index b491105..0000000 --- a/Documentation/building-documentation.adoc +++ /dev/null @@ -1,52 +0,0 @@ -= Building the Documentation -:page-parent: Tutorials - -// SPDX-FileCopyrightText: 2022 Unikie -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -This tutorial assumes that you have https://nixos.org/[Nix] installed. -You may also want to xref:binary-cache.adoc[configure the Spectrum -binary cache], to avoid having to wait for dependencies to compile on -your local system. - -1. Get a copy of the Spectrum source code: -+ -[source,shell] ----- -git clone https://spectrum-os.org/git/spectrum ----- -2. Enter the documentation directory: -+ -[source,shell] ----- -cd spectrum/Documentation ----- -3. Enter the development environment: -+ -[source,shell] ----- -nix-shell -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz ----- -4. In the development shell, do an initial build of the documentation -site: -+ -[source,shell] ----- -scripts/build.sh ----- -5. Run a development server for previewing changes locally: -+ -[source,shell] ----- -jekyll serve ----- -+ -This will serve a local copy of the documentation at http://localhost:4000/. -+ -IMPORTANT: Jekyll doesn't handle rendering of the draw.io diagrams, so -if you modify any of those, or add new ones, you'll have to run -`scripts/build.sh` again to do a full rebuild of the site. - -Once you've made your changes to the documentation, see -xref:first-patch.adoc[Sending Your First Patch] for information -about how to submit them for review. diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc index 574deb4..39f9f28 100644 --- a/Documentation/decisions/001-host-update-mechanism.adoc +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -1,6 +1,6 @@ = 001 Host Update Mechanism :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -11,7 +11,7 @@ Proposed == Context -Spectrum currently has no implementation for software update. The host -- +Spectrum currently has no implementation for software update. The host -- consisting of the Linux kernel, KVM, cloud-hypervisor and minimal user space tools -- will require software updates to support feature development and security fixes. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc index 4412b53..4a745eb 100644 --- a/Documentation/decisions/002-install-options.adoc +++ b/Documentation/decisions/002-install-options.adoc @@ -1,6 +1,6 @@ -= 002 Install options += 002 Install Options :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc index 8494ea4..a13b8cc 100644 --- a/Documentation/decisions/003-partitioning.adoc +++ b/Documentation/decisions/003-partitioning.adoc @@ -1,6 +1,6 @@ = 003 Partitioning :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc index 26fe273..5b0f518 100644 --- a/Documentation/decisions/004-data-at-rest-encryption.adoc +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -1,6 +1,6 @@ = 004 Data at Rest Encryption :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -20,5 +20,5 @@ User data is encrypted. == Consequences Spectrum needs to come with enough software to get the encryption key -via different methods (password, usb, fido, etc.) Can we use dm-crypt +via different methods (password, usb, fido, etc.). Can we use dm-crypt for everything instead of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-monitor.adoc b/Documentation/decisions/005-virtual-machine-monitor.adoc index db81c72..df1b501 100644 --- a/Documentation/decisions/005-virtual-machine-monitor.adoc +++ b/Documentation/decisions/005-virtual-machine-monitor.adoc @@ -1,6 +1,6 @@ = 005 Virtual Machine Monitor :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc index 872044e..b92d863 100644 --- a/Documentation/decisions/006-drivers-on-host.adoc +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -1,6 +1,6 @@ = 006 Drivers on Host :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/007-usb-virtual-machines.adoc b/Documentation/decisions/007-usb-virtual-machines.adoc index 3bdf78b..d832691 100644 --- a/Documentation/decisions/007-usb-virtual-machines.adoc +++ b/Documentation/decisions/007-usb-virtual-machines.adoc @@ -1,6 +1,6 @@ = 007 USB Virtual Machine :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc index a1b7d49..c1e5b87 100644 --- a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc +++ b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc @@ -1,6 +1,6 @@ = 008 Inter-VM Communication Mechanisms :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/index.adoc b/Documentation/decisions/index.adoc index 772f382..a022239 100644 --- a/Documentation/decisions/index.adoc +++ b/Documentation/decisions/index.adoc @@ -1,6 +1,6 @@ = Architecture Decision Records :page-has_children: true -:page-parent: Explanation +:page-parent: About Spectrum // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/b4.adoc b/Documentation/development/b4.adoc similarity index 91% rename from Documentation/b4.adoc rename to Documentation/development/b4.adoc index 489ced4..5f141e9 100644 --- a/Documentation/b4.adoc +++ b/Documentation/development/b4.adoc @@ -1,5 +1,7 @@ = Installing and Configuring b4 -:page-parent: Tutorials +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-FileCopyrightText: 2022 Unikie @@ -19,7 +21,7 @@ of the Spectrum root's nix-shell. You should be able to install b4 from your package manager. -Using Nix, you can start a shell with b4 available by running +Using Nix, you can start a shell with b4 available by running: [listing] [source,shell] diff --git a/Documentation/build-configuration.adoc b/Documentation/development/build-configuration.adoc similarity index 73% rename from Documentation/build-configuration.adoc rename to Documentation/development/build-configuration.adoc index b89575f..c9a8c99 100644 --- a/Documentation/build-configuration.adoc +++ b/Documentation/development/build-configuration.adoc @@ -1,19 +1,22 @@ = Configuring the Build -:page-parent: How-to Guides +:page-parent: Development +:page-nav_order: 1 :example-caption: Test // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 Some aspects of a Spectrum build can be customised using a build -configuration file. By default, this configuration file should be -called config.nix and located in the root of the Spectrum source tree, -but this can be overridden by setting `spectrum-config` in the +configuration file. + +By default, this configuration file should be called config.nix and located in +the root of the Spectrum source tree, but this can be overridden by setting +`spectrum-config` in the https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PAT… to the path of the configuration file. The configuration file should contain an attribute set. The only -currently allowed attribute name is `pkgs`, which allows using a +currently allowed attribute name is `pkgs`. It allows using a custom Nixpkgs to evaluate Spectrum. .config.nix to build Spectrum with a https://nixos.org/manual/nixpkgs/unstable/#sec-overlays-definition[Nixpkgs overlay] diff --git a/Documentation/development/building-documentation.adoc b/Documentation/development/building-documentation.adoc new file mode 100644 index 0000000..4f464f5 --- /dev/null +++ b/Documentation/development/building-documentation.adoc @@ -0,0 +1,52 @@ += Building Documentation +:page-parent: Development +:page-nav_order: 5 + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Make sure you have https://nixos.org/download.html[Nix] installed. +You may also want to xref:../installation/binary-cache.adoc[configure the Spectrum +binary cache], to avoid having to wait for dependencies to compile on +your local system. + +. Get a copy of the Spectrum source code: ++ +[source,shell] +---- +git clone https://spectrum-os.org/git/spectrum +---- +. Enter the documentation directory: ++ +[source,shell] +---- +cd spectrum/Documentation +---- +. Enter the development environment: ++ +[source,shell] +---- +nix-shell -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz +---- +. In the development shell, do an initial build of the documentation +site: ++ +[source,shell] +---- +scripts/build.sh +---- +. Run a development server for previewing changes locally: ++ +[source,shell] +---- +jekyll serve +---- ++ +This will serve a local copy of the documentation at http://localhost:4000/. ++ +IMPORTANT: Jekyll does not handle rendering of the draw.io diagrams. If you +modify any of those, or add new ones, run `scripts/build.sh` again to do a full +rebuild of the site. + +After making changes to the documentation, see how to +xref:first-patch.adoc[send your patch] and submit the changes for review. diff --git a/Documentation/creating-vms.adoc b/Documentation/development/creating-vms.adoc similarity index 97% rename from Documentation/creating-vms.adoc rename to Documentation/development/creating-vms.adoc index d967098..8785190 100644 --- a/Documentation/creating-vms.adoc +++ b/Documentation/development/creating-vms.adoc @@ -1,5 +1,7 @@ = Creating VMs -:page-parent: Reference +:page-parent: Managing VMs +:page-grand_parent: Development +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/debugging.adoc b/Documentation/development/debugging.adoc similarity index 79% rename from Documentation/debugging.adoc rename to Documentation/development/debugging.adoc index 3871a7c..f1bbf01 100644 --- a/Documentation/debugging.adoc +++ b/Documentation/development/debugging.adoc @@ -1,7 +1,6 @@ -= Debugging Spectrum -:page-parent: Explanation -:toc: -:toclevels: 1 += Debugging +:page-parent: Development +:page-nav_order: 4 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -9,8 +8,9 @@ == Extracting core dumps when running Spectrum in a VM When using a VM to run the Spectrum host system, a special mechanism -is available to enable easy introspection of core files. When a -program on the Spectrum host system dumps core, the system will +is available to enable easy introspection of core files. + +When a program on the Spectrum host system dumps core, the system will attempt to upload the core file to _its_ host (i.e. the system running Spectrum in a VM) using the vsock(7) protocol, on port 1129271877. diff --git a/Documentation/first-patch.adoc b/Documentation/development/first-patch.adoc similarity index 80% rename from Documentation/first-patch.adoc rename to Documentation/development/first-patch.adoc index 30672b9..aa6ebee 100644 --- a/Documentation/first-patch.adoc +++ b/Documentation/development/first-patch.adoc @@ -1,11 +1,11 @@ -= Sending Your First Patch -:page-parent: Tutorials += Sending Your Patch +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Prerequisites - This tutorial assumes that you already have basic https://git-scm.com/[git] experience. @@ -14,19 +14,18 @@ https://spectrum-os.org/git/[Spectrum source tree]. You'll also need to have configured `git send-email` — a guide for this can be found at https://git-send-email.io/. -== Making your changes +== Making Changes -If you've worked on any git repository before, the process for making -your changes will probably be very familiar. +The process of making changes is similar to working on any git repository. -1. Create a branch for your changes: +. Create a branch for your changes: + [source,shell] ---- git checkout -b fix-docs # for example ---- -2. Make changes in your editor. -3. Stage and commit your changes: +. Make changes in your editor. +. Stage and commit your changes: + [source,shell] ---- @@ -39,7 +38,7 @@ indicates your acceptance of the https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin], which is mandatory for Spectrum patches. -== Submitting changes +== Submitting Changes Once you're happy with how the commits on your branch look, run: @@ -64,13 +63,14 @@ message that will be sent before all of your patches. Once your patch has been submitted, wait for it to be reviewed. Feedback, if any, will be sent as email replies to your submitted -patch. You can respond to feedback in your mail client. Please use -the Reply All button to ensure that your messages are sent to the +patch. You can respond to feedback in your mail client. + +Use the *Reply All* button to sent your messages to the mailing list as well as to the person who sent the feedback. -If you need to make changes to your patch, and submit a new version, +If you need to make changes to your patch and submit a new version, use https://git-rebase.io/[`git rebase`] to create a new version of -your patch(es), and submit it like this: +your patch(es) and then submit it like this: [source,shell] ---- @@ -81,7 +81,7 @@ The added `-v2` flag indicates that this is version two of your patch set. If your patches require more rounds of changes, submit subsequent rounds with `-v3`, `-v4`, etc. as appropriate. -If you'd like to describe what has changed from the previous version +If you would like to describe what has changed from the previous version of your patches, you can do so in a xref:cover-letter[cover letter] as described above. diff --git a/Documentation/development/index.adoc b/Documentation/development/index.adoc new file mode 100644 index 0000000..4925570 --- /dev/null +++ b/Documentation/development/index.adoc @@ -0,0 +1,59 @@ += Development +:description: Development progress, general development practices +:page-nav_order: 4 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is free software, currently under active development. + + +== General Workflow + +Spectrum is a very small and not changeable sort of Linux distro. It does +not have a package manager for users. If you faced a missing feature or a bug, or you want to suggest some improvements, please consider the following: + +* Get the latest version of the +https://spectrum-os.org/git/spectrum[source code] and make sure your problem +was not fixed. +* xref:../installation/getting-spectrum.adoc[Build Spectrum] and make changes +in the source code. In addition: +** If you need to customize the build to be able to use a +vendor kernel, you can +xref:../development/build-configuration.adoc[configure the build]. +** You can xref:../development/testing-patches.adoc[test patches] +subbmited by others. +* If you would like to share your code with the community, +you can send the changes to the maintainers for possible inclusion in +the Spectrum source code as a new patch. +** xref:../development/b4.adoc[Install and configure the b4 utility] to be able +to work with patches. +** xref:../development/first-patch.adoc[Submit the changes for review]. +Keep your git commits clean and make sure they meet general guidelines. +** Wait for approval from the maintainers' side. Detailed updates are posted in +Spectrum Development +https://spectrum-os.org/lists/hyperkitty/list/devel@spectrum-os.org/[threads]. +** Update documentation with the code. For more information, see +xref:../development/building-documentation.adoc[Building Documentation]. + +For additional information, see +https://spectrum-os.org/contributing.html[Contributing to Spectrum]. + + +== Developer Setup + +* https://git.kernel.org/pub/scm/utils/b4/b4.git/about/[b4] +* https://nixos.org/manual/nix/stable/introduction.html[Nix package manager] +* https://docs.asciidoctor.org/[AsciiDoc] for writing the documentation + + +== Mailing Lists + +The Spectrum project runs several +https://spectrum-os.org/participating.html#mailing-lists[mailing lists] on +which you can ask your questions or help other people with the questions they +have. All the Spectrum developers as well as many long time Linux and Spectrum users are on the lists. + +For real-time feedback, use +https://spectrum-os.org/participating.html#irc[IRC/Matrix channel]. diff --git a/Documentation/development/managing-vms.adoc b/Documentation/development/managing-vms.adoc new file mode 100644 index 0000000..d0a3f09 --- /dev/null +++ b/Documentation/development/managing-vms.adoc @@ -0,0 +1,14 @@ += Managing VMs +:page-parent: Development +:page-nav_order: 2 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Ready to get started with Spectrum? Here is what you can do next: + +* xref:../development/creating-vms.adoc[Create your own VM] to use others applications. +* xref:../development/running-vms.adoc[Start some applications]. +* If needed, xref:../development/user-partition.adoc[change the user partition type]. diff --git a/Documentation/replying.adoc b/Documentation/development/replying.adoc similarity index 68% rename from Documentation/replying.adoc rename to Documentation/development/replying.adoc index bb8e31a..a1ad394 100644 --- a/Documentation/replying.adoc +++ b/Documentation/development/replying.adoc @@ -1,5 +1,7 @@ = Replying to Messages in the Mailing List Archives -:page-parent: Tutorials +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -9,14 +11,13 @@ https://spectrum-os.org/participating.html#mailing-lists[mailing lists]. Make sure to Reply All when replying to messages, so that the message -is sent to the mailing list and not just to the person you're -replying to. +is sent to the mailing list and not just to the person you reply to. == Getting a copy of a message -You may want to reply to a mailing list message that you don't have a -copy of in your mail client, because you aren't subscribed to the -list, or because you subscribed after the message was sent. +You may want to reply to a mailing list message that you do not have a +copy of in your mail client, because you are not subscribed to the +list or because you subscribed after the message was sent. To do this, find the message you want to reply to in the https://spectrum-os.org/lists/archives[public-inbox list archives], diff --git a/Documentation/reviewing-patches.adoc b/Documentation/development/reviewing-patches.adoc similarity index 88% rename from Documentation/reviewing-patches.adoc rename to Documentation/development/reviewing-patches.adoc index 63ff24e..ba47f25 100644 --- a/Documentation/reviewing-patches.adoc +++ b/Documentation/development/reviewing-patches.adoc @@ -1,5 +1,7 @@ = Reviewing Patches -:page-parent: How-to Guides +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/running-vms.adoc b/Documentation/development/running-vms.adoc similarity index 85% rename from Documentation/running-vms.adoc rename to Documentation/development/running-vms.adoc index d0d3f99..7c474d5 100644 --- a/Documentation/running-vms.adoc +++ b/Documentation/development/running-vms.adoc @@ -1,5 +1,7 @@ = Running VMs -:page-parent: Reference +:page-parent: Managing VMs +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/testing-patches.adoc b/Documentation/development/testing-patches.adoc similarity index 82% rename from Documentation/testing-patches.adoc rename to Documentation/development/testing-patches.adoc index 743cd6e..0c72c93 100644 --- a/Documentation/testing-patches.adoc +++ b/Documentation/development/testing-patches.adoc @@ -1,5 +1,7 @@ = Testing Patches -:page-parent: How-to Guides +:page-parent: Working with Patches +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-FileCopyrightText: 2022 Unikie @@ -9,7 +11,7 @@ Potential changes to Spectrum are posted to and discussed on the https://spectrum-os.org/participating.html#spectrum-devel[devel@spectrum-os… mailing list. -== Apply the patch +== Apply Patch . Find the patch series you want to test on https://spectrum-os.org/lists/archives/spectrum-devel/[public-inbox]. @@ -33,15 +35,15 @@ of the Spectrum root's nix-shell, which allows you to skip this step. ---- b4 am 20220511092352.70E54C980(a)atuin.qyliss.net ---- - -. b4 will indicate the file name it has downloaded the patches into - with a line like: +b4 will indicate the file name it has downloaded the patches into with a line +like: + [example] [listing] +---- Writing ./20220424_hi_host_rootfs_fix_weston_hotplugging.mbx -+ -Run `git am` on that file to apply the patches, for example: +---- +. Run `git am` on that file to apply the patches. For example: + [example] [source,shell] @@ -49,14 +51,12 @@ Run `git am` on that file to apply the patches, for example: git am 20220424_hi_host_rootfs_fix_weston_hotplugging.mbx ---- -== Post your test results +== Post Your Results -When you've tested a patch, it's really helpful to +When you tested a patch, it is helpful to xref:replying.adoc[reply] with your test results. -If the patch worked for you, please reply to it and include a line -like the following, separated from any reply text: - +If the patch worked for you, please reply to it and include a line like the following, separated from any reply text: ---- Tested-by: John Smith <john(a)example.com> ---- @@ -66,10 +66,10 @@ patch replies will be automatically included in the commit message when a patch is applied. It's also helpful to explain in your reply how you tested the patch, -but you don't have to if it's obvious. (For example, if a patch is +but you don't have to if it's obvious. For example, if a patch is supposed to fix a bug, and you verified that after applying the patch the bug is fixed, just the Tested-by line on its own is enough to -indicate that.) +indicate that. If you found an issue with the patch, do not include a Tested-by line, and instead reply to the patch explaining what you tested, what you diff --git a/Documentation/development/user-partition.adoc b/Documentation/development/user-partition.adoc new file mode 100644 index 0000000..3fa6d01 --- /dev/null +++ b/Documentation/development/user-partition.adoc @@ -0,0 +1,18 @@ += User Partition +:page-parent: Managing VMs +:page-grand_parent: Development +:page-nav_order: 3 + +// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +The Spectrum host system is immutable, so configuration and user data +lives on a separate partition. + +The host system discovers the user +partition by looking for the special partition type +`9293e1ff-cee4-4658-88be-898ec863944f`. + +This behavior can be overridden with the `ext` parameter on the host's kernel +command line, which works in a similar way to the standard Linux `root` +parameter. diff --git a/Documentation/uuid-reference.adoc b/Documentation/development/uuid-reference.adoc similarity index 97% rename from Documentation/uuid-reference.adoc rename to Documentation/development/uuid-reference.adoc index 4b0b481..0bbded5 100644 --- a/Documentation/uuid-reference.adoc +++ b/Documentation/development/uuid-reference.adoc @@ -1,7 +1,8 @@ = UUID Reference -:page-parent: Reference +:page-parent: Development :toc: preamble :toclevels: 1 +:page-nav_order: 6 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -40,7 +41,7 @@ xref:user-partition.adoc[Spectrum user partition]. === `56a3bbc3-aefa-43d9-a64d-7b3fd59bbc4e` -https://github.com/endlessm/eos-installer["eosimages"] partition on the +https://github.com/endlessm/eos-installer["eosimages"] partition on the Spectrum combined live system / installer image. == Combined Image Partition IDs diff --git a/Documentation/development/working-with-patces.adoc b/Documentation/development/working-with-patces.adoc new file mode 100644 index 0000000..f73734f --- /dev/null +++ b/Documentation/development/working-with-patces.adoc @@ -0,0 +1,12 @@ += Working with Patches +:page-parent: Development +:page-nav_order: 3 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Patches are the way for contributors to submit code to the Spectrum project. + +Make sure to xref:../development/b4.adoc[install and configure the b4 utility] +before starting. diff --git a/Documentation/explanation.adoc b/Documentation/explanation.adoc deleted file mode 100644 index b39cc6d..0000000 --- a/Documentation/explanation.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Explanation -:page-has_children: true -:page-nav_order: 4 - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/how-to.adoc b/Documentation/how-to.adoc deleted file mode 100644 index f43fa13..0000000 --- a/Documentation/how-to.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= How-to Guides -:page-has_children: true -:page-nav_order: 2 - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/index.adoc b/Documentation/index.adoc index 3079847..e0a45c7 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -1,13 +1,32 @@ -= Spectrum Docs += Spectrum Documentation :page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -Spectrum is a compartmentalized operating system. -If you'd like to try Spectrum, see xref:getting-spectrum.adoc[Getting -Spectrum]. +Spectrum is an open-source project that aims to create a computer operating +system, based on the principle of security by compartmentalization, that has a +lower barrier to entry and is easy to use and maintain. + +== Using Spectrum To learn about what Spectrum is and how it's implemented, start with -the xref:architecture.adoc[architecture overview]. +the xref:about/architecture.adoc[architecture overview]. + +If you want to try Spectrum, see xref:../installation/index.adoc[Build and Run] + to setup a development environment. + + +== Developing and Contributing + +Spectrum is made of free and open-source software. It is free for anyone to + use, modify, and distribute. + +Once you are up and running, see + xref:../development/index.adoc[Development] to understand how to work with + Spectrum's code and participate in the collaborative development process. + +If you are thinking of contributing to Spectrum docs, see + xref:../development/building-documentation.adoc[Building Documentation]. diff --git a/Documentation/binary-cache.adoc b/Documentation/installation/binary-cache.adoc similarity index 74% rename from Documentation/binary-cache.adoc rename to Documentation/installation/binary-cache.adoc index 6e69b39..ab53666 100644 --- a/Documentation/binary-cache.adoc +++ b/Documentation/installation/binary-cache.adoc @@ -1,5 +1,6 @@ -= Setting Up the Binary Cache -:page-parent: How-to Guides += Setting Up Binary Cache +:page-parent: Build and Run +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -20,10 +21,9 @@ encounter any trouble with it. The binary cache is currently not able to provide logs, due to a https://github.com/NixOS/nix/pull/6051[Nix bug]. -== On NixOS +== For NixOS -The following configuration adds the Spectrum binary cache as a -substituter, and tells Nix to trust builds signed with its public key. +Add the following configuration to /etc/nixos/configuration.nix: [source,nix] ---- @@ -38,7 +38,13 @@ substituter, and tells Nix to trust builds signed with its public key. } ---- -== On Non-NixOS systems +This configuration adds the Spectrum binary cache as a substituter and makes +Nix trust builds signed with its public key. + +To apply changes, rebuild your system with the https://nixos.wiki/wiki/Nixos-rebuild[nixos-rebuild] command. + + +== For Non-NixOS Systems Add the following configuration to /etc/nix/nix.conf: @@ -48,6 +54,10 @@ substituters = https://cache.dataaturservice.se/spectrum/ https://cache.nixos.or trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ= ---- +Run `systemctl restart nix-daemon.service` after making any changes in the Nix +configuration. After that the binary cache will be used automatically when +`nix-build` is run. + If you have other binary caches configured besides cache.nixos.org, make sure to add the Spectrum cache URL and public key to the existing configuration lines in your nix.conf file. diff --git a/Documentation/getting-spectrum.adoc b/Documentation/installation/getting-spectrum.adoc similarity index 79% rename from Documentation/getting-spectrum.adoc rename to Documentation/installation/getting-spectrum.adoc index b422a94..392cab5 100644 --- a/Documentation/getting-spectrum.adoc +++ b/Documentation/installation/getting-spectrum.adoc @@ -1,5 +1,6 @@ = Getting Spectrum -:page-parent: Tutorials +:page-parent: Build and Run +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -8,7 +9,7 @@ To get Spectrum, you need to build it from source. As long as you're running Linuxfootnote:[Building from other operating systems might work, but hasn't been tested. Patches are welcome to support building from other operating systems, as long as they're not too invasive.] -and have https://nixos.org/[Nix] and https://git-scm.com/[git] +and have https://nixos.org/download.html[Nix] and https://git-scm.com/[git] installed, it's very easy to do. TIP: Before you start, consider setting up the @@ -34,7 +35,7 @@ nix-shell -I nixpkgs=../../../nixpkgs-spectrum --run 'make run' This builds just enough of Spectrum to try it out in a VM, but it will still take a very long time. -== Building an installer +== Building Installer [source,shell] ---- @@ -47,8 +48,7 @@ If you haven't set up the xref:binary-cache.adoc[binary cache], this will take a very long time. When it's done, a symbolic link named "result" will appear, pointing to a Spectrum USB installer image. -CAUTION: Spectrum is not yet suitable for real-world use. Do not use -your Spectrum system for anything important or sensitive. Spectrum is -currently missing many important security properties, and there is no -procedure for updating to new versions of Spectrum -- you have to -reinstall. +CAUTION: Do not use Spectrum for anything important or sensitive as it is not +yet suitable for real-world use. Many important security properties are +currently missing, and there is no procedure for updating to +new versions—you have to reinstall the OS. diff --git a/Documentation/installation/index.adoc b/Documentation/installation/index.adoc new file mode 100644 index 0000000..d67c88d --- /dev/null +++ b/Documentation/installation/index.adoc @@ -0,0 +1,23 @@ += Build and Run +:description: How to download and install Spectrum OS. +:page-nav_order: 2 +:page-has_children: true +:page-has_toc: false + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + + +To start working with Spectrum, you need: + +* xref:../installation/binary-cache.adoc[Set up the binary cache] to speed up the build process. +* xref:../installation/getting-spectrum.adoc[Build Spectrum] from the source. + +TIP: At this stage, Spectrum works only on x86-64. AArch64 support is in +development. + +== Uninstalling and Updating + +Currently, there is no implementation for a software update. + +You can replace Spectrum by installing another OS. diff --git a/Documentation/reference.adoc b/Documentation/reference.adoc deleted file mode 100644 index 44b359d..0000000 --- a/Documentation/reference.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Reference -:page-has_children: true -:page-nav_order: 3 - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/tutorials.adoc b/Documentation/tutorials.adoc deleted file mode 100644 index cd1fb12..0000000 --- a/Documentation/tutorials.adoc +++ /dev/null @@ -1,6 +0,0 @@ -= Tutorials -:page-nav_order: 1 -:page-has_children: true - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/user-partition.adoc b/Documentation/user-partition.adoc deleted file mode 100644 index 73bc0d0..0000000 --- a/Documentation/user-partition.adoc +++ /dev/null @@ -1,12 +0,0 @@ -= The User Partition -:page-parent: Explanation - -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 - -The Spectrum host system is immutable, so configuration and user data -lives on a separate partition. The host system discovers the user -partition by looking for the special partition type -`9293e1ff-cee4-4658-88be-898ec863944f`. This behavior can be -overridden with the `ext` parameter on the host's kernel command line, -which works in a similar way to the standard Linux `root` parameter. -- 2.34.1

2 3

[PATCH] Docs: new structure
by Jenni Nikolaenko 28 Oct '22

28 Oct '22

Create separate folders for new parent pages, update Introduction page, remove a and the articles from titles, quick check text for simple english Signed-off-by: Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> --- Documentation/{ => about}/architecture.adoc | 19 ++++----- Documentation/about/index.adoc | 21 ++++++++++ .../decisions/001-host-update-mechanism.adoc | 2 +- .../decisions/002-install-options.adoc | 4 +- Documentation/decisions/003-partitioning.adoc | 2 +- .../004-data-at-rest-encryption.adoc | 2 +- .../005-virtual-machine-monitor.adoc | 2 +- .../decisions/006-drivers-on-host.adoc | 2 +- .../decisions/007-usb-virtual-machines.adoc | 2 +- ...008-inter-vm-communication-mechanisms.adoc | 2 +- Documentation/decisions/index.adoc | 2 +- .../building-documentation.adoc | 10 ++--- .../{ => development}/debugging.adoc | 7 ++-- .../{ => development}/first-patch.adoc | 28 +++++++------ Documentation/development/index.adoc | 17 ++++++++ Documentation/{ => development}/replying.adoc | 4 +- .../{ => development}/reviewing-patches.adoc | 4 +- .../{ => development}/testing-patches.adoc | 42 ++++++++----------- .../{ => development}/uuid-reference.adoc | 4 +- Documentation/explanation.adoc | 3 +- .../{ => getting-started}/creating-vms.adoc | 2 +- Documentation/getting-started/index.adoc | 7 ++++ .../{ => getting-started}/running-vms.adoc | 2 +- .../{ => getting-started}/user-partition.adoc | 8 ++-- Documentation/how-to.adoc | 1 + Documentation/index.adoc | 22 +++++++--- Documentation/{ => installation}/b4.adoc | 3 +- .../{ => installation}/binary-cache.adoc | 11 ++--- .../{ => installation}/getting-spectrum.adoc | 11 ++--- Documentation/installation/index.adoc | 18 ++++++++ Documentation/reference.adoc | 3 +- Documentation/tutorials.adoc | 3 +- 32 files changed, 171 insertions(+), 99 deletions(-) rename Documentation/{ => about}/architecture.adoc (84%) create mode 100644 Documentation/about/index.adoc rename Documentation/{ => development}/building-documentation.adoc (85%) rename Documentation/{ => development}/debugging.adoc (92%) rename Documentation/{ => development}/first-patch.adoc (83%) create mode 100644 Documentation/development/index.adoc rename Documentation/{ => development}/replying.adoc (93%) rename Documentation/{ => development}/reviewing-patches.adoc (89%) rename Documentation/{ => development}/testing-patches.adoc (62%) rename Documentation/{ => development}/uuid-reference.adoc (98%) rename Documentation/{ => getting-started}/creating-vms.adoc (98%) create mode 100644 Documentation/getting-started/index.adoc rename Documentation/{ => getting-started}/running-vms.adoc (93%) rename Documentation/{ => getting-started}/user-partition.adoc (80%) rename Documentation/{ => installation}/b4.adoc (96%) rename Documentation/{ => installation}/binary-cache.adoc (90%) rename Documentation/{ => installation}/getting-spectrum.adoc (85%) create mode 100644 Documentation/installation/index.adoc diff --git a/Documentation/architecture.adoc b/Documentation/about/architecture.adoc similarity index 84% rename from Documentation/architecture.adoc rename to Documentation/about/architecture.adoc index 1c4307b..db82d60 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/about/architecture.adoc @@ -1,17 +1,16 @@ = Architecture -:page-parent: Explanation +:page-parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie +// SPDX-FileCopyrightText: 2022 Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Introduction +Spectrum OS is based on the principle of security by compartmentalization. -Spectrum is based on the principle of security by -compartmentalization. The high level stack is illustrated in the -following diagram: +The high level stack is illustrated in the following diagram: -image::diagrams/stack.svg[] +image::../diagrams/stack.svg[] The default set of virtual machines includes two application VMs, _appvm-catgirl_ (an IRC client) and _appvm-lynx_ (a text-based web @@ -26,7 +25,7 @@ https://en.wikipedia.org/wiki/Architectural_decision[Architecturally significant decisions] are xref:decisions/index.adoc[recorded] as lightweight https://cognitect.com/blog/2011/11/15/documenting-architecture-decisions/[A…. -== The Spectrum host system +== Spectrum Host System Compartmentalization is implemented using https://cloud-hypervisor.org/[cloud-hypervisor] virtual machines. @@ -35,7 +34,7 @@ https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[Kernel-based Virtual Machine] (KVM) to provide lightweight, hardware-accelerated VMs. While Linux (including KVM) is portable between many hardware architectures, -cloud-hypervisor supports only x86_64 and aarch64. Spectrum currently only +cloud-hypervisor supports only x86_64 and aarch64. Spectrum currently only works on x86_64, but aarch64 support is in development. https://skarnet.org/software/s6-rc/overview.html[s6-rc] is used for service @@ -44,7 +43,7 @@ and service scripts. https://wayland.freedesktop.org/[Wayland] is used for window management and display. The Wayland architecture is well documented -https://wayland.freedesktop.org/architecture.html[here]. The host provides only +https://wayland.freedesktop.org/architecture.html[here]. The host provides only a Wayland terminal client, https://codeberg.org/dnkl/foot/[foot], which is used for interacting with VM consoles. In future it will be possible for application VMs to display windows on the single Wayland compositor on the host system, @@ -57,7 +56,7 @@ https://www.etalabs.net/compare_libcs.html[added safety on resource exhaustion and security hardening on memory allocation]. Kernel hardening will be investigated in future. -== Exploring the Spectrum dependency tree +== Spectrum Dependency Tree For a detailed, interactive view of dependencies, use https://github.com/utdemir/nix-tree[nix-tree] in the Spectrum repository: diff --git a/Documentation/about/index.adoc b/Documentation/about/index.adoc new file mode 100644 index 0000000..a882852 --- /dev/null +++ b/Documentation/about/index.adoc @@ -0,0 +1,21 @@ += About Spectrum OS +:description: Some words about Spectrum as the operating system, not a project. Highlights the differences between common Linux distributions and Spectrum. +:page-nav_order: 1 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Jenni Nikko <evgeniia.nikolaenko(a)unikie.com> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is a Linux-based system, derived from NixOS. This gives an actively-developed base with good hardware support, powerful and optimised compartmentalization primitives in KVM, and the reproducible packaging and configuration system that is important for a maintainable compartmentalized system. + +== Why Spectrum + +There are several features that make Spectrum OS unique: + +* User data and applications are managed centrally, while remaining isolated. +That means that the system can be backed up and managed as a whole, rather than mixed up in several dozen VMs. + +* The host system and isolated environments are managed declaratively and reproducibly using the Nix package manager. +This can save the user the burden of maintaining many different virtual computers, allowing finer-grained resource access controls and making it possible to verify the software running across all environments. + +TIP: If you are interested in why we do something _this_ way instead of _that_ way, see xref:../decisions/index.adoc[Architecture Decision Records]. diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc index 574deb4..7032146 100644 --- a/Documentation/decisions/001-host-update-mechanism.adoc +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -1,6 +1,6 @@ = 001 Host Update Mechanism :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc index 4412b53..a7c4175 100644 --- a/Documentation/decisions/002-install-options.adoc +++ b/Documentation/decisions/002-install-options.adoc @@ -1,6 +1,6 @@ -= 002 Install options += 002 Install Options :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc index 8494ea4..b00f528 100644 --- a/Documentation/decisions/003-partitioning.adoc +++ b/Documentation/decisions/003-partitioning.adoc @@ -1,6 +1,6 @@ = 003 Partitioning :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc index 26fe273..27323db 100644 --- a/Documentation/decisions/004-data-at-rest-encryption.adoc +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -1,6 +1,6 @@ = 004 Data at Rest Encryption :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/005-virtual-machine-monitor.adoc b/Documentation/decisions/005-virtual-machine-monitor.adoc index db81c72..df5a65e 100644 --- a/Documentation/decisions/005-virtual-machine-monitor.adoc +++ b/Documentation/decisions/005-virtual-machine-monitor.adoc @@ -1,6 +1,6 @@ = 005 Virtual Machine Monitor :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc index 872044e..86d3105 100644 --- a/Documentation/decisions/006-drivers-on-host.adoc +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -1,6 +1,6 @@ = 006 Drivers on Host :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/007-usb-virtual-machines.adoc b/Documentation/decisions/007-usb-virtual-machines.adoc index 3bdf78b..24dac65 100644 --- a/Documentation/decisions/007-usb-virtual-machines.adoc +++ b/Documentation/decisions/007-usb-virtual-machines.adoc @@ -1,6 +1,6 @@ = 007 USB Virtual Machine :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc index a1b7d49..9fce4ef 100644 --- a/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc +++ b/Documentation/decisions/008-inter-vm-communication-mechanisms.adoc @@ -1,6 +1,6 @@ = 008 Inter-VM Communication Mechanisms :page-parent: Architecture Decision Records -:page-grand_parent: Explanation +:page-grand_parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/decisions/index.adoc b/Documentation/decisions/index.adoc index 772f382..4f3a7e1 100644 --- a/Documentation/decisions/index.adoc +++ b/Documentation/decisions/index.adoc @@ -1,6 +1,6 @@ = Architecture Decision Records :page-has_children: true -:page-parent: Explanation +:page-parent: About Spectrum OS // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/building-documentation.adoc b/Documentation/development/building-documentation.adoc similarity index 85% rename from Documentation/building-documentation.adoc rename to Documentation/development/building-documentation.adoc index b491105..da5fa8c 100644 --- a/Documentation/building-documentation.adoc +++ b/Documentation/development/building-documentation.adoc @@ -1,5 +1,5 @@ -= Building the Documentation -:page-parent: Tutorials += Building Documentation +:page-parent: Development // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -47,6 +47,6 @@ IMPORTANT: Jekyll doesn't handle rendering of the draw.io diagrams, so if you modify any of those, or add new ones, you'll have to run `scripts/build.sh` again to do a full rebuild of the site. -Once you've made your changes to the documentation, see -xref:first-patch.adoc[Sending Your First Patch] for information -about how to submit them for review. +Once you made your changes to the documentation, see +xref:first-patch.adoc[Patching] for information +on how to submit your patch for review. diff --git a/Documentation/debugging.adoc b/Documentation/development/debugging.adoc similarity index 92% rename from Documentation/debugging.adoc rename to Documentation/development/debugging.adoc index 3871a7c..6e529a9 100644 --- a/Documentation/debugging.adoc +++ b/Documentation/development/debugging.adoc @@ -1,7 +1,6 @@ -= Debugging Spectrum -:page-parent: Explanation -:toc: -:toclevels: 1 += Debugging +:page-parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/first-patch.adoc b/Documentation/development/first-patch.adoc similarity index 83% rename from Documentation/first-patch.adoc rename to Documentation/development/first-patch.adoc index 30672b9..7b8dd1d 100644 --- a/Documentation/first-patch.adoc +++ b/Documentation/development/first-patch.adoc @@ -1,11 +1,12 @@ -= Sending Your First Patch -:page-parent: Tutorials += Patching +:page-parent: Development +:page-nav_order: 1 +:page-has_children: true +:toc: preamble // SPDX-FileCopyrightText: 2022 Unikie // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -== Prerequisites - This tutorial assumes that you already have basic https://git-scm.com/[git] experience. @@ -14,10 +15,10 @@ https://spectrum-os.org/git/[Spectrum source tree]. You'll also need to have configured `git send-email` — a guide for this can be found at https://git-send-email.io/. -== Making your changes +== Making Changes -If you've worked on any git repository before, the process for making -your changes will probably be very familiar. +If you worked on any git repository before, the process for making +your changes will be very familiar. 1. Create a branch for your changes: + @@ -39,7 +40,7 @@ indicates your acceptance of the https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin], which is mandatory for Spectrum patches. -== Submitting changes +== Submitting Changes Once you're happy with how the commits on your branch look, run: @@ -64,13 +65,14 @@ message that will be sent before all of your patches. Once your patch has been submitted, wait for it to be reviewed. Feedback, if any, will be sent as email replies to your submitted -patch. You can respond to feedback in your mail client. Please use -the Reply All button to ensure that your messages are sent to the +patch. You can respond to feedback in your mail client. + +Use the *Reply All* button to sent your messages to the mailing list as well as to the person who sent the feedback. -If you need to make changes to your patch, and submit a new version, +If you need to make changes to your patch and submit a new version, use https://git-rebase.io/[`git rebase`] to create a new version of -your patch(es), and submit it like this: +your patch(es) and then submit it like this: [source,shell] ---- @@ -81,7 +83,7 @@ The added `-v2` flag indicates that this is version two of your patch set. If your patches require more rounds of changes, submit subsequent rounds with `-v3`, `-v4`, etc. as appropriate. -If you'd like to describe what has changed from the previous version +If you would like to describe what has changed from the previous version of your patches, you can do so in a xref:cover-letter[cover letter] as described above. diff --git a/Documentation/development/index.adoc b/Documentation/development/index.adoc new file mode 100644 index 0000000..471daf7 --- /dev/null +++ b/Documentation/development/index.adoc @@ -0,0 +1,17 @@ += Development +:description: Development progress, general development practices +:page-nav_order: 4 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Jenni Nikko <evgeniia.nikolaenko(a)unikie.com> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +Spectrum is free software, currently under active development. + +== Developer Setup + +Before starting, make sure you are familiar with https://git.kernel.org/pub/scm/utils/b4/b4.git/about/[b4] and the https://nixos.org/manual/nix/stable/introduction.html[Nix package manager]. + +== Mailing Lists + +The Spectrum project runs several https://spectrum-os.org/mailman3/lists/?all-lists[mailing lists] on which you can ask your questions or help other people with the questions they have. All the Spectrum developers as well as many long time Linux and Spectrum users are on the lists. diff --git a/Documentation/replying.adoc b/Documentation/development/replying.adoc similarity index 93% rename from Documentation/replying.adoc rename to Documentation/development/replying.adoc index bb8e31a..05740a0 100644 --- a/Documentation/replying.adoc +++ b/Documentation/development/replying.adoc @@ -1,5 +1,7 @@ = Replying to Messages in the Mailing List Archives -:page-parent: Tutorials +:page-parent: Patching +:page-grand_parent: Development +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/reviewing-patches.adoc b/Documentation/development/reviewing-patches.adoc similarity index 89% rename from Documentation/reviewing-patches.adoc rename to Documentation/development/reviewing-patches.adoc index 63ff24e..c8c971f 100644 --- a/Documentation/reviewing-patches.adoc +++ b/Documentation/development/reviewing-patches.adoc @@ -1,5 +1,7 @@ = Reviewing Patches -:page-parent: How-to Guides +:page-parent: Patching +:page-grand_parent: Development +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/testing-patches.adoc b/Documentation/development/testing-patches.adoc similarity index 62% rename from Documentation/testing-patches.adoc rename to Documentation/development/testing-patches.adoc index 8ba7804..99adfd2 100644 --- a/Documentation/testing-patches.adoc +++ b/Documentation/development/testing-patches.adoc @@ -1,5 +1,8 @@ = Testing Patches -:page-parent: How-to Guides +:page-parent: Patching +:page-grand_parent: Development +:page-nav_order: 1 +:toc: preamble // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 @@ -8,52 +11,41 @@ Potential changes to Spectrum are posted to and discussed on the https://spectrum-os.org/participating.html#spectrum-devel[devel@spectrum-os… mailing list. -== Apply the patch +== Apply Patch If you haven't already, you'll first need to xref:b4.adoc[install and configure] https://git.kernel.org/pub/scm/utils/b4/b4.git/about/[b4]. Then: -. Find the patch series you want to test on - https://spectrum-os.org/lists/archives/spectrum-devel/[public-inbox]. -. Navigate to the "permalink" page for any patch in the series. -. Copy the Message-Id for the patch, as shown on the permalink page, e.g. - \20220511092352.70E54C980@atuin.qyliss.net. -. In a checkout of the appropriate git repository - (https://spectrum-os.org/git/spectrum[Spectrum] or - https://spectrum-os.org/git/nixpkgs[Spectrum Nixpkgs]), Run `b4 am` - with the patch's Message-Id to download all the patches in the - series into a file. +1. Find the patch series you want to test on https://spectrum-os.org/lists/archives/spectrum-devel/[public-inbox]. +2. Navigate to the "permalink" page for any patch in the series. +3. Copy the Message-Id for the patch, as shown on the permalink page, e.g. \20220511092352.70E54C980@atuin.qyliss.net. +4. In a checkout of the appropriate git repository (https://spectrum-os.org/git/spectrum[Spectrum] or https://spectrum-os.org/git/nixpkgs[Spectrum Nixpkgs]), run `b4 am` with the patch's Message-Id to download all the patches in the series into a file. + -[example] [source,shell] ---- b4 am 20220511092352.70E54C980(a)atuin.qyliss.net ---- - -. b4 will indicate the file name it has downloaded the patches into - with a line like: +b4 will indicate the file name it has downloaded the patches into with a line like: + -[example] -[listing] +[source,shell] +---- Writing ./20220424_hi_host_rootfs_fix_weston_hotplugging.mbx +---- +5. Run `git am` on that file to apply the patches, for example: + -Run `git am` on that file to apply the patches, for example: -+ -[example] [source,shell] ---- git am 20220424_hi_host_rootfs_fix_weston_hotplugging.mbx ---- -== Post your test results +== Post Your Results When you've tested a patch, it's really helpful to xref:replying.adoc[reply] with your test results. -If the patch worked for you, please reply to it and include a line -like the following, separated from any reply text: - +If the patch worked for you, please reply to it and include a line like the following, separated from any reply text: +[source,shell] ---- Tested-by: John Smith <john(a)example.com> ---- diff --git a/Documentation/uuid-reference.adoc b/Documentation/development/uuid-reference.adoc similarity index 98% rename from Documentation/uuid-reference.adoc rename to Documentation/development/uuid-reference.adoc index 4b0b481..0eccc82 100644 --- a/Documentation/uuid-reference.adoc +++ b/Documentation/development/uuid-reference.adoc @@ -1,5 +1,5 @@ = UUID Reference -:page-parent: Reference +:page-parent: Development :toc: preamble :toclevels: 1 @@ -40,7 +40,7 @@ xref:user-partition.adoc[Spectrum user partition]. === `56a3bbc3-aefa-43d9-a64d-7b3fd59bbc4e` -https://github.com/endlessm/eos-installer["eosimages"] partition on the +https://github.com/endlessm/eos-installer["eosimages"] partition on the Spectrum combined live system / installer image. == Combined Image Partition IDs diff --git a/Documentation/explanation.adoc b/Documentation/explanation.adoc index b39cc6d..f682129 100644 --- a/Documentation/explanation.adoc +++ b/Documentation/explanation.adoc @@ -1,6 +1,5 @@ = Explanation -:page-has_children: true -:page-nav_order: 4 +:page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/creating-vms.adoc b/Documentation/getting-started/creating-vms.adoc similarity index 98% rename from Documentation/creating-vms.adoc rename to Documentation/getting-started/creating-vms.adoc index d967098..e06be85 100644 --- a/Documentation/creating-vms.adoc +++ b/Documentation/getting-started/creating-vms.adoc @@ -1,5 +1,5 @@ = Creating VMs -:page-parent: Reference +:page-parent: Getting Started // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/getting-started/index.adoc b/Documentation/getting-started/index.adoc new file mode 100644 index 0000000..1b468ab --- /dev/null +++ b/Documentation/getting-started/index.adoc @@ -0,0 +1,7 @@ += Getting Started +:description: Exploring Spectrum OS. Using (=How-To-Guides), Configuring (adding smth). Ready to get started with Spectrum OS? After installing you can create VMs and then configure each one. +:page-nav_order: 3 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Jenni Nikko <evgeniia.nikolaenko(a)unikie.com> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/running-vms.adoc b/Documentation/getting-started/running-vms.adoc similarity index 93% rename from Documentation/running-vms.adoc rename to Documentation/getting-started/running-vms.adoc index d0d3f99..9073e3c 100644 --- a/Documentation/running-vms.adoc +++ b/Documentation/getting-started/running-vms.adoc @@ -1,5 +1,5 @@ = Running VMs -:page-parent: Reference +:page-parent: Getting Started // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/user-partition.adoc b/Documentation/getting-started/user-partition.adoc similarity index 80% rename from Documentation/user-partition.adoc rename to Documentation/getting-started/user-partition.adoc index 73bc0d0..a33d7fc 100644 --- a/Documentation/user-partition.adoc +++ b/Documentation/getting-started/user-partition.adoc @@ -1,11 +1,13 @@ -= The User Partition -:page-parent: Explanation += User Partition +:page-parent: Getting Started // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 The Spectrum host system is immutable, so configuration and user data -lives on a separate partition. The host system discovers the user +lives on a separate partition. + +The host system discovers the user partition by looking for the special partition type `9293e1ff-cee4-4658-88be-898ec863944f`. This behavior can be overridden with the `ext` parameter on the host's kernel command line, diff --git a/Documentation/how-to.adoc b/Documentation/how-to.adoc index f43fa13..98cc842 100644 --- a/Documentation/how-to.adoc +++ b/Documentation/how-to.adoc @@ -1,4 +1,5 @@ = How-to Guides +:page-nav_exclude: true :page-has_children: true :page-nav_order: 2 diff --git a/Documentation/index.adoc b/Documentation/index.adoc index 3079847..d26676b 100644 --- a/Documentation/index.adoc +++ b/Documentation/index.adoc @@ -1,13 +1,23 @@ -= Spectrum Docs += Spectrum Documentation :page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-FileCopyrightText: 2022 Jenni Nikolaenko <evgeniia.nikolaenko(a)unikie.com> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -Spectrum is a compartmentalized operating system. +== Spectrum Project -If you'd like to try Spectrum, see xref:getting-spectrum.adoc[Getting -Spectrum]. +Spectrum is an open source project that aims to create a computer operating system, based on the principle of security by compartmentalization, that has a lower barrier to entry and is easier to use and maintain than other such systems. For more information on the Spectrum project, see https://spectrum-os.org/. -To learn about what Spectrum is and how it's implemented, start with -the xref:architecture.adoc[architecture overview]. +Spectrum is made of free and open source software. It is free for anyone to use, modify, and distribute. If you want to be involved with the Spectrum project, see https://spectrum-os.org/contributing.html. + +The Spectrum project source code is https://spectrum-os.org/git/spectrum. + +== Spectrum OS + +Spectrum is an in-development operating system that aims to afford its users security by compartmentalization, while also improving upon other similar projects by maintaining a high level of usability. + +To learn about what Spectrum OS is and how it's implemented, start with +the xref:about/architecture.adoc[architecture overview]. + +If you want to try Spectrum, see xref:../installation/index.adoc[Build and Run]. diff --git a/Documentation/b4.adoc b/Documentation/installation/b4.adoc similarity index 96% rename from Documentation/b4.adoc rename to Documentation/installation/b4.adoc index 2519894..1ba87b2 100644 --- a/Documentation/b4.adoc +++ b/Documentation/installation/b4.adoc @@ -1,5 +1,6 @@ = Installing and Configuring b4 -:page-parent: Tutorials +:page-parent: Build and Run +:page-nav_order: 3 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/binary-cache.adoc b/Documentation/installation/binary-cache.adoc similarity index 90% rename from Documentation/binary-cache.adoc rename to Documentation/installation/binary-cache.adoc index 6e69b39..232f96c 100644 --- a/Documentation/binary-cache.adoc +++ b/Documentation/installation/binary-cache.adoc @@ -1,10 +1,11 @@ -= Setting Up the Binary Cache -:page-parent: How-to Guides += Setting Up Binary Cache +:page-parent: Build and Run +:page-nav_order: 1 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -Building Spectrum from source can take a very long time. To avoid +Building Spectrum OS from source can take a very long time. To avoid having to wait when building the system to try it out or test patches, an x86_64 binary cache service is available. If configured to do so, Nix will download build outputs from the cache, instead of building @@ -20,7 +21,7 @@ encounter any trouble with it. The binary cache is currently not able to provide logs, due to a https://github.com/NixOS/nix/pull/6051[Nix bug]. -== On NixOS +== For NixOS The following configuration adds the Spectrum binary cache as a substituter, and tells Nix to trust builds signed with its public key. @@ -38,7 +39,7 @@ substituter, and tells Nix to trust builds signed with its public key. } ---- -== On Non-NixOS systems +== For Non-NixOS Systems Add the following configuration to /etc/nix/nix.conf: diff --git a/Documentation/getting-spectrum.adoc b/Documentation/installation/getting-spectrum.adoc similarity index 85% rename from Documentation/getting-spectrum.adoc rename to Documentation/installation/getting-spectrum.adoc index b3fa1ef..a0ea1c4 100644 --- a/Documentation/getting-spectrum.adoc +++ b/Documentation/installation/getting-spectrum.adoc @@ -1,10 +1,11 @@ = Getting Spectrum -:page-parent: Tutorials +:page-parent: Build and Run +:page-nav_order: 2 // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -To get Spectrum, you need to build it from source. As long as you're +To get Spectrum OS, you need to build it from source. As long as you're running Linuxfootnote:[Building from other operating systems might work, but hasn't been tested. Patches are welcome to support building from other operating systems, as long as they're not too invasive.] @@ -17,7 +18,7 @@ lot of time waiting for builds. == Trying Spectrum -If you want to try Spectrum out to get a feel for it, without +If you want to try Spectrum OS out to get a feel for it, without installing it, you can run it in a development VM with some example applications. @@ -34,7 +35,7 @@ nix-shell -I nixpkgs=../../../nixpkgs-spectrum --run 'make run' This builds just enough of Spectrum to try it out in a VM, but it will still take a very long time. -== Building an installer +== Building Installer [source,shell] ---- @@ -48,7 +49,7 @@ named "result" will appear, pointing to a Spectrum USB installer image. CAUTION: Spectrum is not yet suitable for real-world use. Do not use -your Spectrum system for anything important or sensitive. Spectrum is +your Spectrum OS for anything important or sensitive. Spectrum is currently missing many important security properties, and there is no procedure for updating to new versions of Spectrum -- you have to reinstall. diff --git a/Documentation/installation/index.adoc b/Documentation/installation/index.adoc new file mode 100644 index 0000000..99e9723 --- /dev/null +++ b/Documentation/installation/index.adoc @@ -0,0 +1,18 @@ += Build and Run +:description: How to download and install Spectrum OS. +:page-nav_order: 2 +:page-has_children: true + +// SPDX-FileCopyrightText: 2022 Jenni Nikko <evgeniia.nikolaenko(a)unikie.com> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +To try Spectrum OS out or xref:../development/testing-patches.adoc[test patches], you need to build the system from the source. + +In order to speed up the build process, set up the binary cache. After building Spectrum OS, you can install and configure the b4 utility to be able to work with patches. + + +TIP: Note that Spectrum OS currently works only on x86-64. AAarch64 support is in development. + +Currently, there is no implementation for a software update. + +You can replace the installation with some other OS. diff --git a/Documentation/reference.adoc b/Documentation/reference.adoc index 44b359d..55259ea 100644 --- a/Documentation/reference.adoc +++ b/Documentation/reference.adoc @@ -1,6 +1,5 @@ = Reference -:page-has_children: true -:page-nav_order: 3 +:page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 diff --git a/Documentation/tutorials.adoc b/Documentation/tutorials.adoc index cd1fb12..fcef31b 100644 --- a/Documentation/tutorials.adoc +++ b/Documentation/tutorials.adoc @@ -1,6 +1,5 @@ = Tutorials -:page-nav_order: 1 -:page-has_children: true +:page-nav_exclude: true // SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> // SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 -- 2.34.1

4 4

Branches for in-progress Wayland work
by Alyssa Ross 25 Oct '22

25 Oct '22

I've published branches called "wayland" on both the spectrum and nixpkgs repositories, to make it easier to follow the current state of that work. I've tested that both hello-wayland and foot work in a Spectrum VM. (To test foot I just found-and-replaced hello-wayland with foot in hello-wayland.nix.)

1 0

I'll be on holiday for two weeks
by Alyssa Ross 30 Sep '22

30 Sep '22

I'm taking a holiday from my work for Unikie for two weeks starting now. (Back on Monday, 9th Oct.) I'll be spending some of the time finishing up some work for NLnet before Spectrum's grant expires, so you'll almost certainly still see some activity from me on the lists. But the rest of the time I'm going to try to spend away, relaxing, so anybody sending patches should not be alarmed if they go a couple of weeks before I take a look at them. :) Alyssa

1 1

[RFC PATCH nixpkgs 0/4] Wayland security-context support
by Puck Meerburg 30 Sep '22

30 Sep '22

NOTE: These patches are designed to apply on top of the previous Wayland support series at [1]. This series contains the patches necessary to build the demo repository for Wayland security-context[2] support. As the Spectrum support for Wayland is also very WIP, and uses a different WM than the one I was focused on (Weston versus sway), it's not yet integrated with Spectrum itself. Of course, my decision to use Sway in this demo isn't setting Spectrum's own window manager in stone; the hope is this protocol gets implemented into as many compositors (and sandboxes) as possible :) To try out the demo, see [3] for the repository and instructions. A few of these patches (wlroots, sway) have been sent upstream already[4][5]. The crosvm patches need a tiny bit of work before I'm completely confident sending them upstream. One major issue that is worked around but not entirely solved is a bit of a mystery to me: After a short amount of messages, the virtio-gpu driver stops sending and receiving Wayland messages. As far as I can tell, this is likely a quirk of running crosvm with only cross-domain enabled, but one I haven't been able to delve into the Linux source code for to figure out how to properly solve. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.158379… [2]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/68 [3]: https://puck.moe/git/security-context-demo [4]: https://gitlab.freedesktop.org/wlroots/wlroots/-/merge_requests/3589 [5]: https://github.com/swaywm/sway/pull/7187 Puck Meerburg (4): cloud-hypervisor: workaround keymap mmap wlroots: apply security-context patches sway: apply security-context patches crosvm: apply security-context patches ...ry-mapping-shared-memory-as-RO-if-RW.patch | 57 ++++++++ .../cloud-hypervisor/default.nix | 1 + .../virtualization/crosvm/default.nix | 10 +- .../window-managers/sway/default.nix | 22 +++ pkgs/development/libraries/wlroots/0.15.nix | 20 ++- .../libraries/wlroots/security-context-v1.xml | 131 ++++++++++++++++++ 6 files changed, 235 insertions(+), 6 deletions(-) create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0004-virtio-devices-try-mapping-shared-memory-as-RO-if-RW.patch create mode 100644 pkgs/development/libraries/wlroots/security-context-v1.xml -- 2.35.1

1 5

[RFC PATCH v2 00/10] Initial support for VM Wayland
by Alyssa Ross 30 Sep '22

30 Sep '22

IMPORTANT NOTE: this series should be applied on top of my previous series "Introduce a shared base for application VMs" [1]. This is a bit inconvenient I know, but I haven't committed the other series yet due to wanting to take a second pass at it. Changes since v1: - Fix syntax in img/app development Makefile target. - Enable virgl for virtio-gpu devices (see below). v1: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.158379… Introduction ------------ This series adds the Spectrum-side support for running VMs that can display Wayland windows on the host compositor, using virtio-gpu. There are various small things still to be resolved, but it's ready for other people to try out and test. The easiest way to test this out is to run "vm-start hello-wayland" on the Spectrum host. If everything goes well, a small window with a picture of a cat (hello-wayland) should appear. This window is running in a VM. Currently, virgl is enabled for all virtio-gpu devices. This means that all VMs will have GPU acceleration enabled. This is currently required to work around what appears to be a kernel bug that causes applications to freeze, but is likely not something we want to be enabling by default once that's fixed. I have tested with hello-wayland, as in the appvm provided as a demonstration here, and also with foot (by replacing hello-wayland with foot in that VM). I haven't done enough testing with this version of the code to guage how reliably they work yet. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20220919073659.170327… [2]: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.158379… Alyssa Ross (10): host/start-vm: use MAP_SHARED memory for VMs img/app: don't block app startup on network online img/app: add Wayland over virtio-gpu support to kernel vm-lib: add mesa drivers to VM img/app: add support for testing virtio-gpu img/app: add support for testing in crosvm host/start-vm: factor out VM definition path host: add support for Wayland in VMs vm/app: add hello-wayland demo VM host/start-vm: disable cloud-hypervisor sandbox Documentation/creating-vms.adoc | 5 +++ host/initramfs/extfs.nix | 3 ++ host/rootfs/Makefile | 4 +++ host/rootfs/default.nix | 4 +-- host/rootfs/etc/s6-rc/ext-rc-init/up | 8 +++++ host/rootfs/etc/template/gpu/data/check | 5 +++ host/rootfs/etc/template/gpu/notification-fd | 1 + .../etc/template/gpu/notification-fd.license | 2 ++ host/rootfs/etc/template/gpu/run | 9 ++++++ host/rootfs/etc/template/gpu/type | 1 + host/rootfs/etc/template/gpu/type.license | 2 ++ host/start-vm/start-vm.rs | 26 +++++++++------ img/app/Makefile | 32 +++++++++++++++++-- img/app/default.nix | 3 +- img/app/etc/mdev/iface | 2 +- img/app/etc/mdev/listen | 12 +++++++ img/app/etc/mdev/wait | 15 +++++++++ img/app/etc/s6-rc/ok-all/contents | 1 + img/app/shell.nix | 2 +- vm-lib/make-vm.nix | 14 ++++++-- vm/app/catgirl.nix | 1 + vm/app/hello-wayland.nix | 25 +++++++++++++++ vm/app/lynx.nix | 1 + 23 files changed, 159 insertions(+), 19 deletions(-) create mode 100755 host/rootfs/etc/template/gpu/data/check create mode 100644 host/rootfs/etc/template/gpu/notification-fd create mode 100644 host/rootfs/etc/template/gpu/notification-fd.license create mode 100755 host/rootfs/etc/template/gpu/run create mode 100644 host/rootfs/etc/template/gpu/type create mode 100644 host/rootfs/etc/template/gpu/type.license create mode 100755 img/app/etc/mdev/listen create mode 100755 img/app/etc/mdev/wait create mode 100644 vm/app/hello-wayland.nix -- 2.37.1

1 11

[RFC PATCH nixpkgs v2 0/9] virtio-gpu with crosvm and cloud-hypervisor
by Alyssa Ross 30 Sep '22

30 Sep '22

Changes since v1 ---------------- - Fixed the cloud-hypervisor seccomp sandbox for the GPU device. - Worked around crosvm not handling wlroots's read-only keymaps in cloud-hypervisor. (Thanks Puck.) - Switched to cherry-picks of sternenseemann's ocamlPackages.wayland and wayland-proxy-virtwl updates from upstream. v1: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.158379… Introduction ------------ This series contains all the changes needed to make it possible to run Wayland over cross-domain virtio-gpu using cloud-hypervisor. It only contains changes needed to packages in Nixpkgs, so there's no Spectrum integration here. That's a separate patchset, of which I'll also be sending v2 soon. There's also still some work to do here, hence being marked RFC — in the cloud-hypervisor patch you'll spot a few TODOs and FIXMEs — but it's ready for other people to have a look at and test. With these changes, it's possible to run a cloud-hypervisor VM with a GPU attached as follows: crosvm device gpu --socket vhost-user-gpu.sock ... cloud-hypervisor --gpu socket=vhost-user-gpu.sock ... Then in the guest, just run a Wayland application under wayland-proxy-virtwl: wayland-proxy-virtwl --virtio-gpu hello-wayland For Wayland over cross-domain virtio-gpu, which we're particularly interested in, the full crosvm command line would be something like: crosvm device gpu --socket vhost-user-gpu.sock \ --wayland-sock "$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY" \ --params '{"context-types": "virgl:virgl2:cross-domain"}' Overview -------- We start by upgrading crosvm to a beta version (since very recent changes changed their vhost-user implementation in a way that makes it significantly easier to interoperate with from cloud-hypervisor), then applying a few small fixes for cases where crosvm had made assumptions that were violated by cloud-hypervisor. Next, we apply patches to cloud-hypervisor and some of its dependencies, to implement the frontend of the GPU device. Finally, we upgrade wayland-proxy-virtwl (the program that speaks virtio-gpu and acts as the Wayland compositor inside the guest) to a version that supports virtio-gpu. I'm using wayland-proxy-virtwl over Sommelier here primary because Sommelier appears to have broken with Linux 5.19, but another nice benefit is that wayland-proxy-virtwl is maintained by somebody that we know and talk to sometimes on discuss(a)spectrum-os.org. Upstreamability --------------- A question anybody looking at the volume of changes here should be asking is: "To what extent is this upstreamable?". It's a bit of a mixed bag — most notably, cloud-hypervisor is not interested in virtio-gpu[1], so we'll have to maintain the cloud-hypervisor patches ourselves until we can come up with a more upstream-friendly solution (e.g. a vfio-user implementation of virtio-gpu). We also can't upstream the changes to the vhost crate, as the protocol we are using is crosvm specific, and so some standardisation work would need to happen there. But we can upstream the changes to crosvm, and the virtio-bindings crate. I've already started on the latter. As for the patches to Nixpkgs itself here, the updates to wayland-proxy-virtwl and the wayland library it depends on are already upstream (thanks sterni), I have an open PR[2] for the crosvm update (although to 105.0 rather than 106.3 than here, since the 106 series is still in beta), and the rustPlatform change is also upstreamable, but would require auditing Nixpkgs for cargoSha256 values that would change as a result. [1]: https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/3960#discu… [2]: https://github.com/NixOS/nixpkgs/pull/193746 Alyssa Ross (7): crosvm: switch back to old git repo URL crosvm.updateScript: update release branch format crosvm: 104.0 -> 106.2 crosvm.updateScript: don't vendor Cargo.lock crosvm: add fixes for cloud-hypervisor virtio-gpu rustPlatform: forward unpack hooks to cargo fetch cloud-hypervisor: add virtio-gpu support sternenseemann (2): ocamlPackages.wayland: 1.0 -> 1.1 wayland-proxy-virtwl: unstable-2021-12-05 -> unstable-2022-09-22 .../0001-build-use-local-vhost.patch | 39 + ...dings-regenerate-with-bindgen-0.60.1.patch | 2589 ++++++++++++ ...0002-build-use-local-virtio-bindings.patch | 39 + ...gs-remove-workaround-for-old-bindgen.patch | 28 + ...-bindings-regenerate-with-Glibc-2.36.patch | 247 ++ ...0003-virtio-devices-add-a-GPU-device.patch | 1279 ++++++ ...-bindings-regenerate-with-Linux-5.19.patch | 1067 +++++ ...tio-bindings-add-virtio-gpu-bindings.patch | 3587 +++++++++++++++++ .../cloud-hypervisor/default.nix | 64 +- ...ser-add-shared-memory-region-support.patch | 724 ++++ .../virtualization/crosvm/Cargo.lock | 2214 ---------- .../crosvm/default-seccomp-policy-dir.diff | 38 +- .../virtualization/crosvm/default.nix | 15 +- ...-consider-shm-buuffers-when-setting-.patch | 34 + ...t_user-loosen-expected-message-order.patch | 71 + ...ces-vhost_user-remove-spurious-check.patch | 42 + .../virtualization/crosvm/update.py | 15 +- .../rust/build-rust-package/default.nix | 4 +- .../ocaml-modules/wayland/default.nix | 8 +- .../wayland/wayland-proxy-virtwl/default.nix | 17 +- 20 files changed, 9857 insertions(+), 2264 deletions(-) create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-build-use-local-vhost.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-virtio-bindings-regenerate-with-bindgen-0.60.1.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-build-use-local-virtio-bindings.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-virtio-bindings-remove-workaround-for-old-bindgen.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-bindings-regenerate-with-Glibc-2.36.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-devices-add-a-GPU-device.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0004-virtio-bindings-regenerate-with-Linux-5.19.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0005-virtio-bindings-add-virtio-gpu-bindings.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/vhost-vhost_user-add-shared-memory-region-support.patch delete mode 100644 pkgs/applications/virtualization/crosvm/Cargo.lock create mode 100644 pkgs/applications/virtualization/crosvm/devices-properly-consider-shm-buuffers-when-setting-.patch create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-loosen-expected-message-order.patch create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-remove-spurious-check.patch base-commit: 768879638cfd6a54664b762ec98a7f8f45620c44 -- 2.37.1

1 9

[RFC PATCH 00/10] Initial support for VM Wayland
by Alyssa Ross 30 Sep '22

30 Sep '22

IMPORTANT NOTE: this series should be applied on top of my previous series "Introduce a shared base for application VMs" [1]. This is a bit inconvenient I know, but I haven't committed the other series yet due to wanting to take a second pass at it. This series adds the Spectrum-side support for running VMs that can display Wayland windows on the host compositor, using virtio-gpu. There are various small things still to be resolved, but it's ready for other people to try out and test. The easiest way to test this out is to run "vm-start hello-wayland" on the Spectrum host. If everything goes well, a small window with a picture of a cat (hello-wayland) should appear. This window is running in a VM. One particular known issue is that on one run of the new appvm-hello-wayland, I got this error: [12:11:52.107] libwayland: error in client communication (pid 863) I'm not yet aware of what would cause such an error. It went away after I rebooted and tried again. There are some other known issues with my cloud-hypervisor patch too. See the other thread for information on those. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20220919073659.170327… [2]: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.158379… Alyssa Ross (10): host/start-vm: use MAP_SHARED memory for VMs img/app: don't block app startup on network online img/app: add Wayland over virtio-gpu support to kernel vm-lib: add mesa drivers to VM img/app: add support for testing virtio-gpu img/app: add support for testing in crosvm host/start-vm: factor out VM definition path host: add support for Wayland in VMs vm/app: add hello-wayland demo VM host/start-vm: disable cloud-hypervisor sandbox Documentation/creating-vms.adoc | 5 +++ host/initramfs/extfs.nix | 3 ++ host/rootfs/Makefile | 4 +++ host/rootfs/default.nix | 4 +-- host/rootfs/etc/s6-rc/ext-rc-init/up | 8 +++++ host/rootfs/etc/template/gpu/data/check | 5 +++ host/rootfs/etc/template/gpu/notification-fd | 1 + .../etc/template/gpu/notification-fd.license | 2 ++ host/rootfs/etc/template/gpu/run | 9 ++++++ host/rootfs/etc/template/gpu/type | 1 + host/rootfs/etc/template/gpu/type.license | 2 ++ host/start-vm/start-vm.rs | 26 +++++++++------ img/app/Makefile | 32 +++++++++++++++++-- img/app/default.nix | 3 +- img/app/etc/mdev/iface | 2 +- img/app/etc/mdev/listen | 12 +++++++ img/app/etc/mdev/wait | 15 +++++++++ img/app/etc/s6-rc/ok-all/contents | 1 + img/app/shell.nix | 2 +- vm-lib/make-vm.nix | 14 ++++++-- vm/app/catgirl.nix | 1 + vm/app/hello-wayland.nix | 25 +++++++++++++++ vm/app/lynx.nix | 1 + 23 files changed, 159 insertions(+), 19 deletions(-) create mode 100755 host/rootfs/etc/template/gpu/data/check create mode 100644 host/rootfs/etc/template/gpu/notification-fd create mode 100644 host/rootfs/etc/template/gpu/notification-fd.license create mode 100755 host/rootfs/etc/template/gpu/run create mode 100644 host/rootfs/etc/template/gpu/type create mode 100644 host/rootfs/etc/template/gpu/type.license create mode 100755 img/app/etc/mdev/listen create mode 100755 img/app/etc/mdev/wait create mode 100644 vm/app/hello-wayland.nix -- 2.37.1

1 11

[PATCH v2] EFI Loader: use correct arch suffix
by Ivan Nikolaenko 29 Sep '22

29 Sep '22

Previously there were hardcoded "x64" in EFI loaders' filenames. Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- img/combined/default.nix | 8 +++++--- img/live/Makefile | 2 +- img/live/default.nix | 6 ++++-- 3 files changed, 10 insertions(+), 6 deletions(-) diff --git a/img/combined/default.nix b/img/combined/default.nix index 3989d55..f65f619 100644 --- a/img/combined/default.nix +++ b/img/combined/default.nix @@ -6,7 +6,7 @@ let inherit (builtins) storeDir; - inherit (pkgs.lib) removePrefix; + inherit (pkgs.lib) removePrefix toUpper; eosimages = import ./eosimages.nix { inherit config; }; @@ -31,6 +31,8 @@ let --owner 0 --group 0 | tar2sqfs $out ''; + efiArch = stdenv.hostPlatform.efiArch; + grub = grub2_efi; grubCfg = substituteAll { @@ -74,8 +76,8 @@ let done mcopy -i $out ${grub}/share/grub/unicode.pf2 ::/grub/fonts - grub-mkimage -o grubx64.efi -p "(hd0,gpt1)/grub" -O ${grub.grubTarget} part_gpt fat - mcopy -i $out grubx64.efi ::/EFI/BOOT/BOOTX64.EFI + grub-mkimage -o grub${efiArch}.efi -p "(hd0,gpt1)/grub" -O ${grub.grubTarget} part_gpt fat + mcopy -i $out grub${efiArch}.efi ::/EFI/BOOT/BOOT${toUpper efiArch}.EFI fsck.vfat -n $out ''; diff --git a/img/live/Makefile b/img/live/Makefile index 9815c88..e63c598 100644 --- a/img/live/Makefile +++ b/img/live/Makefile @@ -36,7 +36,7 @@ build/boot.fat: $(KERNEL) $(INITRAMFS) $(SYSTEMD_BOOT_EFI) build/spectrum.conf $(MCOPY) -i $@ build/spectrum.conf ::/loader/entries $(MCOPY) -i $@ $(KERNEL) ::/spectrum/linux $(MCOPY) -i $@ $(INITRAMFS) ::/spectrum/initrd - $(MCOPY) -i $@ $(SYSTEMD_BOOT_EFI) ::/EFI/BOOT/BOOTX64.EFI + $(MCOPY) -i $@ $(SYSTEMD_BOOT_EFI) ::/EFI/BOOT/$(EFINAME) # veritysetup format produces two files, but Make only (portably) # supports one output per rule, so we combine the two outputs then diff --git a/img/live/default.nix b/img/live/default.nix index 1fa8da3..4301ccc 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -9,7 +9,7 @@ let inherit (config) pkgs; - inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix; + inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix toUpper; extfs = pkgs.pkgsStatic.callPackage ../../host/initramfs/extfs.nix { inherit config; @@ -17,6 +17,7 @@ let rootfs = import ../../host/rootfs { inherit config; }; scripts = import ../../scripts { inherit config; }; initramfs = import ../../host/initramfs { inherit config rootfs; }; + efiArch = stdenv.hostPlatform.efiArch; in stdenvNoCC.mkDerivation { @@ -35,7 +36,8 @@ stdenvNoCC.mkDerivation { INITRAMFS = initramfs; KERNEL = "${rootfs.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; ROOT_FS = rootfs; - SYSTEMD_BOOT_EFI = "${systemd}/lib/systemd/boot/efi/systemd-bootx64.efi"; + SYSTEMD_BOOT_EFI = "${systemd}/lib/systemd/boot/efi/systemd-boot${efiArch}.efi"; + EFINAME = "BOOT${toUpper efiArch}.EFI"; buildFlags = [ "build/live.img" ]; makeFlags = [ "SCRIPTS=${scripts}" ]; -- 2.25.1

3 2

[RFC PATCH nixpkgs 0/9] virtio-gpu with crosvm and cloud-hypervisor
by Alyssa Ross 29 Sep '22

29 Sep '22

Introduction ------------ This series contains all the changes needed to make it possible to run Wayland over cross-domain virtio-gpu using cloud-hypervisor. It only contains changes needed to packages in Nixpkgs, so there's no Spectrum integration here. That'll be coming soon. There's also still some work to do here, hence being marked RFC — in the cloud-hypervisor patch you'll spot a few TODOs and FIXMEs — but it's ready for other people to have a look at and test. With these changes, it's possible to run a cloud-hypervisor VM with a GPU attached as follows: crosvm device gpu --socket vhost-user-gpu.sock ... cloud-hypervisor --gpu socket=vhost-user-gpu.sock ... Then in the guest, just run a Wayland application under wayland-proxy-virtwl: wayland-proxy-virtwl --virtio-gpu hello-wayland For Wayland over cross-domain virtio-gpu, which we're particularly interested in, the full crosvm command line would be something like: crosvm device gpu --socket vhost-user-gpu.sock \ --wayland-sock "$XDG_RUNTIME_DIR/$WAYLAND_DISPLAY" \ --params '{"context-types": "cross-domain"}' Overview -------- We start by upgrading crosvm to a beta version (since very recent changes changed their vhost-user implementation in a way that makes it significantly easier to interoperate with from cloud-hypervisor), then applying a few small fixes for cases where crosvm had made assumptions that were violated by cloud-hypervisor. Next, we apply patches to cloud-hypervisor and some of its dependencies, to implement the frontend of the GPU device. Finally, we upgrade wayland-proxy-virtwl (the program that speaks virtio-gpu and acts as the Wayland compositor inside the guest) to a version that supports virtio-gpu. I'm using wayland-proxy-virtwl over Sommelier here primary because Sommelier appears to have broken with Linux 5.19, but another nice benefit is that wayland-proxy-virtwl is maintained by somebody that we know and talk to sometimes on discuss(a)spectrum-os.org. Upstreamability --------------- A question anybody looking at the volume of changes here should be asking is: "To what extent is this upstreamable?". It's a bit of a mixed bag — most notably, cloud-hypervisor is not interested in virtio-gpu[1], so we'll have to maintain the cloud-hypervisor patches ourselves until we can come up with a more upstream-friendly solution (e.g. a vfio-user implementation of virtio-gpu). We also can't upstream the changes to the vhost crate, as the protocol we are using is crosvm specific, and so some standardisation work would need to happen there. But we can upstream the changes to crosvm, and the virtio-bindings crate. I've already started on the latter. As for the patches to Nixpkgs itself here, we can upstream the crosvm update (although should probably target 105 if I get round to it soon, since 106 is still beta), the rustPlatform change, and the updates to wayland-proxy-virtwl and the wayland library it depends on. [1]: https://github.com/cloud-hypervisor/cloud-hypervisor/discussions/3960#discu… Alyssa Ross (9): crosvm: switch back to old git repo URL crosvm.updateScript: update release branch format crosvm: 104.0 -> 106.2 crosvm.updateScript: don't vendor Cargo.lock crosvm: add fixes for cloud-hypervisor virtio-gpu rustPlatform: forward unpack hooks to cargo fetch cloud-hypervisor: add virtio-gpu support ocamlPackages.wayland: 1.0 -> unstable-2022-05-07 wayland-proxy-virtwl: unstable-2021-12-05 -> unstable-2022-08-06 .../0001-build-use-local-vhost.patch | 39 + ...dings-regenerate-with-bindgen-0.60.1.patch | 2589 ++++++++++++ ...0002-build-use-local-virtio-bindings.patch | 39 + ...gs-remove-workaround-for-old-bindgen.patch | 28 + ...-bindings-regenerate-with-Glibc-2.36.patch | 247 ++ ...-devices-add-a-vhost-user-gpu-device.patch | 1253 ++++++ ...-bindings-regenerate-with-Linux-5.19.patch | 1067 +++++ ...tio-bindings-add-virtio-gpu-bindings.patch | 3587 +++++++++++++++++ .../cloud-hypervisor/default.nix | 64 +- ...ser-add-shared-memory-region-support.patch | 724 ++++ .../virtualization/crosvm/Cargo.lock | 2214 ---------- .../crosvm/default-seccomp-policy-dir.diff | 38 +- .../virtualization/crosvm/default.nix | 15 +- ...-consider-shm-buuffers-when-setting-.patch | 34 + ...t_user-loosen-expected-message-order.patch | 71 + ...ces-vhost_user-remove-spurious-check.patch | 42 + .../virtualization/crosvm/update.py | 15 +- .../rust/build-rust-package/default.nix | 4 +- .../ocaml-modules/wayland/default.nix | 12 +- .../wayland/wayland-proxy-virtwl/default.nix | 14 +- 20 files changed, 9834 insertions(+), 2262 deletions(-) create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-build-use-local-vhost.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0001-virtio-bindings-regenerate-with-bindgen-0.60.1.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-build-use-local-virtio-bindings.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0002-virtio-bindings-remove-workaround-for-old-bindgen.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-bindings-regenerate-with-Glibc-2.36.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0003-virtio-devices-add-a-vhost-user-gpu-device.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0004-virtio-bindings-regenerate-with-Linux-5.19.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/0005-virtio-bindings-add-virtio-gpu-bindings.patch create mode 100644 pkgs/applications/virtualization/cloud-hypervisor/vhost-vhost_user-add-shared-memory-region-support.patch delete mode 100644 pkgs/applications/virtualization/crosvm/Cargo.lock create mode 100644 pkgs/applications/virtualization/crosvm/devices-properly-consider-shm-buuffers-when-setting-.patch create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-loosen-expected-message-order.patch create mode 100644 pkgs/applications/virtualization/crosvm/devices-vhost_user-remove-spurious-check.patch base-commit: 768879638cfd6a54664b762ec98a7f8f45620c44 -- 2.37.1

2 13

[PATCH] scripts/make-gpt.sh: make GPT offset configurable
by Ivan Nikolaenko 29 Sep '22

29 Sep '22

These changes can be used in an external configuration layer as follows: overlays = [ (self: super: { gptOffset = 9437184; }) ]; Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- This patch is not a solution but just a proposal about how things can be done. There is at least one another way of doing this (having our own make-gpt.sh script), but I thought that editing of the current one is better way. Also I don't know should we also consider other images and rootfs be fixed this way. img/live/Makefile | 3 ++- img/live/default.nix | 1 + scripts/make-gpt.sh | 22 ++++++++++++++++++++-- 3 files changed, 23 insertions(+), 3 deletions(-) diff --git a/img/live/Makefile b/img/live/Makefile index 9815c88..5a29156 100644 --- a/img/live/Makefile +++ b/img/live/Makefile @@ -19,7 +19,8 @@ build/live.img: $(SCRIPTS)/format-uuid.sh $(SCRIPTS)/make-gpt.sh build/boot.fat build/boot.fat:c12a7328-f81f-11d2-ba4b-00a0c93ec93b \ build/rootfs.verity.superblock:2c7357ed-ebd2-46d9-aec1-23d437ec2bf5:$$($(SCRIPTS)/format-uuid.sh "$$(dd if=build/rootfs.verity.roothash bs=32 skip=1 count=1 status=none)") \ $(ROOT_FS):4f68bce3-e8cd-4db1-96e7-fbcaf984b709:$$($(SCRIPTS)/format-uuid.sh "$$(head -c 32 build/rootfs.verity.roothash)") \ - $(EXT_FS):9293e1ff-cee4-4658-88be-898ec863944f + $(EXT_FS):9293e1ff-cee4-4658-88be-898ec863944f \ + --gptoffset=$(GPT_OFFSET) mv $@.tmp $@ build/spectrum.conf: build/rootfs.verity.roothash diff --git a/img/live/default.nix b/img/live/default.nix index 5461384..c529206 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -34,6 +34,7 @@ stdenvNoCC.mkDerivation { KERNEL = "${rootfs.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; ROOT_FS = rootfs; SYSTEMD_BOOT_EFI = "${systemd}/lib/systemd/boot/efi/systemd-bootx64.efi"; + GPT_OFFSET=pkgs.gptOffset or null; buildFlags = [ "build/live.img" ]; makeFlags = [ "SCRIPTS=${scripts}" ]; diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 56076d3..e8dd067 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -41,8 +41,25 @@ shift nl=$'\n' table="label: gpt" -# Keep 1MiB free at the start, and 1MiB free at the end. -gptBytes=$TWO_MiB +TEMP=`getopt -o g:: --long gptoffset:: -- "$@"` +eval set -- "$TEMP" + +while true ; do + case "$1" in + -g|--gptoffset) + gptBytes=$2 ; shift 2 ;; + --) shift ; break ;; + *) break ;; + esac +done + +if [ -z "$gptBytes" ] +then + gptBytes=${TWO_MiB} +fi + +FIRST_LBA="$(expr ${gptBytes} / 1024)" + for partition; do sizeMiB="$(sizeMiB "$(partitionPath "$partition")")" table="$table${nl}size=${sizeMiB}MiB,$(awk -f "$scriptsDir/sfdisk-field.awk" -v partition="$partition")" @@ -52,6 +69,7 @@ done rm -f "$out" truncate -s "$gptBytes" "$out" sfdisk "$out" <<EOF +first-lba: ${FIRST_LBA} $table EOF -- 2.25.1

2 1

[PATCH] EFI Loader: use correct arch suffix
by Ivan Nikolaenko 29 Sep '22

29 Sep '22

Previously there were hardcoded "x64" in EFI loaders' filenames. Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- img/combined/default.nix | 5 +++-- img/live/Makefile | 2 +- img/live/default.nix | 5 +++-- nix/eval-config.nix | 11 ++++++++++- 4 files changed, 17 insertions(+), 6 deletions(-) diff --git a/img/combined/default.nix b/img/combined/default.nix index 3989d55..2086973 100644 --- a/img/combined/default.nix +++ b/img/combined/default.nix @@ -7,6 +7,7 @@ let inherit (builtins) storeDir; inherit (pkgs.lib) removePrefix; + inherit (config) archSuffix; eosimages = import ./eosimages.nix { inherit config; }; @@ -74,8 +75,8 @@ let done mcopy -i $out ${grub}/share/grub/unicode.pf2 ::/grub/fonts - grub-mkimage -o grubx64.efi -p "(hd0,gpt1)/grub" -O ${grub.grubTarget} part_gpt fat - mcopy -i $out grubx64.efi ::/EFI/BOOT/BOOTX64.EFI + grub-mkimage -o grub${archSuffix}.efi -p "(hd0,gpt1)/grub" -O ${grub.grubTarget} part_gpt fat + mcopy -i $out grub${archSuffix}.efi ::/EFI/BOOT/BOOT${lib.toUpper archSuffix}.EFI fsck.vfat -n $out ''; diff --git a/img/live/Makefile b/img/live/Makefile index 9815c88..e63c598 100644 --- a/img/live/Makefile +++ b/img/live/Makefile @@ -36,7 +36,7 @@ build/boot.fat: $(KERNEL) $(INITRAMFS) $(SYSTEMD_BOOT_EFI) build/spectrum.conf $(MCOPY) -i $@ build/spectrum.conf ::/loader/entries $(MCOPY) -i $@ $(KERNEL) ::/spectrum/linux $(MCOPY) -i $@ $(INITRAMFS) ::/spectrum/initrd - $(MCOPY) -i $@ $(SYSTEMD_BOOT_EFI) ::/EFI/BOOT/BOOTX64.EFI + $(MCOPY) -i $@ $(SYSTEMD_BOOT_EFI) ::/EFI/BOOT/$(EFINAME) # veritysetup format produces two files, but Make only (portably) # supports one output per rule, so we combine the two outputs then diff --git a/img/live/default.nix b/img/live/default.nix index 65ad058..7be55ac 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -4,7 +4,7 @@ { config ? import ../../nix/eval-config.nix {} }: let - inherit (config) pkgs; + inherit (config) pkgs archSuffix; inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix; extfs = pkgs.pkgsStatic.callPackage ../../host/initramfs/extfs.nix { @@ -33,7 +33,8 @@ stdenvNoCC.mkDerivation { INITRAMFS = initramfs; KERNEL = "${rootfs.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; ROOT_FS = rootfs; - SYSTEMD_BOOT_EFI = "${systemd}/lib/systemd/boot/efi/systemd-bootx64.efi"; + SYSTEMD_BOOT_EFI = "${systemd}/lib/systemd/boot/efi/systemd-boot${archSuffix}.efi"; + EFINAME = "BOOT${lib.toUpper archSuffix}.EFI"; buildFlags = [ "build/live.img" ]; makeFlags = [ "SCRIPTS=${scripts}" ]; diff --git a/nix/eval-config.nix b/nix/eval-config.nix index 467f877..4234321 100644 --- a/nix/eval-config.nix +++ b/nix/eval-config.nix @@ -8,6 +8,15 @@ else {} }: -({ pkgs ? import <nixpkgs> {} }: { +({ pkgs ? import <nixpkgs> {} }: with pkgs; { inherit pkgs; + + archSuffix = + if pkgs.stdenv.isx86_64 then + "x64" + else if pkgs.stdenv.isAarch64 then + "aa64" + else + throw "Unsupported architecture"; + }) config -- 2.25.1

2 2

[PATCH] img/live: use callPackage function instead of "with pkgs;"
by Yuri Nesterov 27 Sep '22

27 Sep '22

When cross compiling Spectrum OS for aarch64 it tries to run aarch64 binaries on x64 host. To fix this issue it is possible to add pkgsBuildHost prefix to all native build inputs. However, a better option would probably be to use callPackage function. With that change it is able to automatically select correct packages. Signed-off-by: Yuri Nesterov <yuriy.nesterov(a)unikie.com> --- img/live/default.nix | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/img/live/default.nix b/img/live/default.nix index 5461384..1fa8da3 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -1,7 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie -{ config ? import ../../nix/eval-config.nix {} }: +{ config ? import ../../nix/eval-config.nix {} }: config.pkgs.callPackage ( + +{ stdenvNoCC, cryptsetup, dosfstools, jq, mtools, util-linux, stdenv +, systemd }: let inherit (config) pkgs; @@ -15,8 +19,6 @@ let initramfs = import ../../host/initramfs { inherit config rootfs; }; in -with pkgs; - stdenvNoCC.mkDerivation { name = "spectrum-live.img"; @@ -48,3 +50,4 @@ stdenvNoCC.mkDerivation { passthru = { inherit rootfs; }; } +) {} -- 2.34.1

2 1

[PATCH] scripts/make-gpt.sh: reserve 8 MiB at start of GPT
by Alyssa Ross 26 Sep '22

26 Sep '22

In some cases (e.g. a product built on Spectrum where updates to both firmware and OS are delivered by the same vendor), it makes sense to include firmware in the same image as the OS. This firmware tends to live in the start of the image, after the GPT header. (The exact location varies by board.) The 1 MiB reserved by sfdisk by default is not enough for this. The conventional value seems to be 8 MiB. On NixOS, this is configurable with the sdImage.firmwarePartitionOffset option, but this defaults to 8 MiB and I wasn't able to find any examples in searches on DuckDuckGo and GitHub of anybody using any other value, so it looks like 8 MiB is broadly acceptable. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- Yuriy, if you'd like to test this patch, here's some documentation about how to do that. :) https://spectrum-os.org/doc/testing-patches.html scripts/make-gpt.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 56076d3..55f38d6 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,12 +1,13 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... ONE_MiB=1048576 -TWO_MiB=2097152 +NINE_MiB=9437184 # Prints the number of 1MiB blocks required to store the file named # $1. We use 1MiB blocks because that's what sfdisk uses for @@ -41,8 +42,9 @@ shift nl=$'\n' table="label: gpt" -# Keep 1MiB free at the start, and 1MiB free at the end. -gptBytes=$TWO_MiB +# Keep 8MiB free at the start (to provide space for firmware if +# required), and 1MiB free at the end. +gptBytes=$NINE_MiB for partition; do sizeMiB="$(sizeMiB "$(partitionPath "$partition")")" table="$table${nl}size=${sizeMiB}MiB,$(awk -f "$scriptsDir/sfdisk-field.awk" -v partition="$partition")" @@ -52,6 +54,7 @@ done rm -f "$out" truncate -s "$gptBytes" "$out" sfdisk "$out" <<EOF +first-lba: 16384 $table EOF -- 2.37.1

2 2

[PATCH] img/live: add rootfs attribute passthru
by Yuri Nesterov 23 Sep '22

23 Sep '22

This allows to get access to rootfs and kernel from device images. Signed-off-by: Yuri Nesterov <yuriy.nesterov(a)unikie.com> --- img/live/default.nix | 2 ++ 1 file changed, 2 insertions(+) diff --git a/img/live/default.nix b/img/live/default.nix index 65ad058..5461384 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -45,4 +45,6 @@ stdenvNoCC.mkDerivation { ''; enableParallelBuilding = true; + + passthru = { inherit rootfs; }; } -- 2.34.1

5 7

[PATCH] Documentation: clarify value of spectrum-config
by Alyssa Ross 23 Sep '22

23 Sep '22

Before, it was not clear that spectrum-config should be set to a file path, rather than a directory — i.e. whether it override the root of the source tree, or the config.nix path within it. This change clarifies that spectrum-config should be set to a full configuration.nix file path, not the directory that contains it. Reported-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- Documentation/build-configuration.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/Documentation/build-configuration.adoc b/Documentation/build-configuration.adoc index db68c34..b89575f 100644 --- a/Documentation/build-configuration.adoc +++ b/Documentation/build-configuration.adoc @@ -8,8 +8,9 @@ Some aspects of a Spectrum build can be customised using a build configuration file. By default, this configuration file should be called config.nix and located in the root of the Spectrum source tree, -but this path can be overridden by setting `spectrum-config` in the -https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PATH[NIX_PATH]. +but this can be overridden by setting `spectrum-config` in the +https://nixos.org/manual/nix/stable/command-ref/env-common.html#env-NIX_PATH[NIX_PATH] +to the path of the configuration file. The configuration file should contain an attribute set. The only currently allowed attribute name is `pkgs`, which allows using a -- 2.37.1

3 2

[PATCH] fix: config support with vms
by Ville Ilvonen 20 Sep '22

20 Sep '22

with config.nix, the build of vms fails with error: undefined variable 'pkgs' .. 49| 50| kernel = pkgs.linux_latest.override { | ^ This fixes the issue for all default vms Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- vm/app/catgirl/default.nix | 1 + vm/app/lynx/default.nix | 1 + vm/sys/net/default.nix | 1 + 3 files changed, 3 insertions(+) diff --git a/vm/app/catgirl/default.nix b/vm/app/catgirl/default.nix index 61f1462..c33f512 100644 --- a/vm/app/catgirl/default.nix +++ b/vm/app/catgirl/default.nix @@ -13,6 +13,7 @@ config.pkgs.pkgsStatic.callPackage ( }: let + inherit (config) pkgs; inherit (lib) cleanSource cleanSourceWith concatMapStringsSep hasSuffix; packages = [ diff --git a/vm/app/lynx/default.nix b/vm/app/lynx/default.nix index ba715ec..f06ef1f 100644 --- a/vm/app/lynx/default.nix +++ b/vm/app/lynx/default.nix @@ -13,6 +13,7 @@ config.pkgs.pkgsStatic.callPackage ( }: let + inherit (config) pkgs; inherit (lib) cleanSource cleanSourceWith concatMapStringsSep hasSuffix; packages = [ diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index dfc7c35..b3dac9a 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -14,6 +14,7 @@ config.pkgs.pkgsStatic.callPackage ( }: let + inherit (config) pkgs; inherit (lib) cleanSource cleanSourceWith concatMapStringsSep hasSuffix; connman = connmanMinimal; -- 2.37.2

2 6

[PATCH 0/6] Introduce a shared base for application VMs
by Alyssa Ross 19 Sep '22

19 Sep '22

Here are some changes I made months ago to enable work I never finished around building VMs inside the Spectrum system with Nix. I'm coming back to it now because it makes it much easier to add more applications for testing, and with upcoming features like Wayland and a persistent filesystem we're going to want more example applications. The idea here is to reduce duplication between application VMs, both in terms of source code size and output size. After this change, creating a new VM just requires writing a very small Nix file like this: { config ? import ../../../nix/eval-config.nix {} }: import ../../vm-lib/make-vm.nix { inherit config; } { name = "appvm-lynx"; providers.net = [ "netvm" ]; run = config.pkgs.pkgsStatic.callPackage ( { writeScript, lynx }: writeScript "run-lynx" '' #!/bin/execlineb -P ${lynx}/bin/lynx https://spectrum-os.org '' ) { }; } Rather than a whole big source tree as before, most of which was duplicated with every other application VM. When a VM generated this way is started, it gets two disk images. One is the shared base image, which is part of the Spectrum base system, and the other contains only the application-specific stuff: the run script, and any store path dependencies that are not already present in the base image. This means that the amount of storage required for each new application VM is substantially reduced. Of course, this isn't the only way to generate VMs. Monolithic VMs like we had before would still work, with some small adjustments for the new disk layout. I also see this fitting well into making it possible to configure extra VMs at build time. It doesn't directly help with that, but making it so that each VM doesn't need to provide everything itself will make creating external VMs easier when it does happen. In future we might want to apply a similar mechanism to service VMs, like netvm, but since we only have one of those so far, it's not clear which parts exactly would be duplicated, so I'm leaving it for now. Other future work is considering the impacts of the shared base image on guest isolation. Can guests observe whether reads of the shared base image hit the host page cache, or even an internal disk cache? At the moment I suspect that the base image doesn't have enough specialised code in it that there would be any interesting results, but it's worth thinking about if the shared image grows new functionality, whether it would be interesting to another guest to be able to observe whether those resources have previously been loaded or not. If this _does_ turn out to be a concern, it could be mitigated by simply copying the base image to temporary storage before booting a VM, and then booting the VM from the copy. Alyssa Ross (6): host/start-vm: support multiple block devices scripts/make-gpt.sh: add support for labels vm: build GPT images host/start-vm: boot using partition label release: rename from "img" img/app: extract from appvm-{lynx,catgirl} Documentation/creating-vms.adoc | 8 +- Documentation/getting-spectrum.adoc | 2 +- host/initramfs/extfs.nix | 11 +- host/rootfs/default.nix | 11 +- host/start-vm/start-vm.rs | 43 ++++-- {vm/app/lynx => img/app}/Makefile | 57 ++++---- {vm/app/catgirl => img/app}/bin | 0 {vm/app/catgirl => img/app}/default.nix | 22 ++-- img/app/etc/fstab | 8 ++ {vm/app/catgirl => img/app}/etc/init | 0 {vm/app/catgirl => img/app}/etc/mdev.conf | 0 {vm/app/lynx => img/app}/etc/mdev/iface | 2 +- {vm/app/catgirl => img/app}/etc/passwd | 0 .../catgirl => img/app}/etc/passwd.license | 0 {vm/app/catgirl => img/app}/etc/resolv.conf | 0 .../app}/etc/s6-linux-init/scripts/rc.init | 1 + .../s6-rc/lynx => img/app/etc/s6-rc/app}/run | 3 +- .../catgirl => img/app/etc/s6-rc/app}/type | 0 .../app/etc/s6-rc/app}/type.license | 0 .../etc/s6-rc/mdevd-coldplug/dependencies | 0 .../app}/etc/s6-rc/mdevd-coldplug/type | 0 .../etc/s6-rc/mdevd-coldplug/type.license | 0 .../app}/etc/s6-rc/mdevd-coldplug/up | 0 .../app}/etc/s6-rc/mdevd/notification-fd | 0 .../etc/s6-rc/mdevd/notification-fd.license | 0 .../catgirl => img/app}/etc/s6-rc/mdevd/run | 0 .../catgirl => img/app}/etc/s6-rc/mdevd/type | 0 .../app}/etc/s6-rc/mdevd/type.license | 0 .../app}/etc/s6-rc/ok-all/contents | 0 .../catgirl => img/app}/etc/s6-rc/ok-all/type | 0 .../app}/etc/s6-rc/ok-all/type.license | 0 .../app}/etc/ssl/certs/ca-certificates.crt | 0 {vm/app/lynx => img/app}/shell.nix | 11 +- release.nix | 2 +- {img => release}/combined/default.nix | 0 {img => release}/combined/eosimages.nix | 0 {img => release}/combined/grub.cfg.in | 0 {img => release}/combined/run-vm.nix | 0 ...ble-gpt-partition-attribute-55-check.patch | 0 ...pt-disable-partition-table-CRC-check.patch | 0 .../0003-install-remove-Endless-OS-ad.patch | 0 ...4-finished-don-t-run-eos-diagnostics.patch | 0 ...omote-spectrum-not-the-Endless-forum.patch | 0 {img => release}/installer/app/default.nix | 0 .../installer/app/vendor-customer-support.ini | 0 {img => release}/installer/configuration.nix | 0 {img => release}/installer/default.nix | 0 {img => release}/installer/run-vm.nix | 0 {img => release}/installer/seat.rules | 0 {img => release}/live/Makefile | 0 {img => release}/live/default.nix | 0 {img => release}/live/shell.nix | 0 scripts/make-gpt.sh | 4 +- scripts/sfdisk-field.awk | 2 +- vm-lib/make-vm.nix | 46 +++++++ vm/app/catgirl.nix | 18 +++ vm/app/catgirl/Makefile | 123 ------------------ vm/app/catgirl/etc/fstab | 6 - vm/app/catgirl/etc/mdev/iface | 36 ----- .../catgirl/etc/s6-linux-init/scripts/rc.init | 10 -- vm/app/catgirl/etc/s6-rc/catgirl/run | 31 ----- .../data/appvm-catgirl/providers/net/netvm | 0 vm/app/catgirl/shell.nix | 17 --- vm/app/lynx.nix | 16 +++ vm/app/lynx/bin | 1 - vm/app/lynx/default.nix | 92 ------------- vm/app/lynx/etc/fstab | 6 - vm/app/lynx/etc/init | 5 - vm/app/lynx/etc/mdev.conf | 5 - vm/app/lynx/etc/passwd | 1 - vm/app/lynx/etc/passwd.license | 2 - vm/app/lynx/etc/resolv.conf | 4 - vm/app/lynx/etc/s6-rc/lynx/type | 1 - vm/app/lynx/etc/s6-rc/lynx/type.license | 2 - .../etc/s6-rc/mdevd-coldplug/dependencies | 4 - vm/app/lynx/etc/s6-rc/mdevd-coldplug/type | 1 - .../etc/s6-rc/mdevd-coldplug/type.license | 2 - vm/app/lynx/etc/s6-rc/mdevd-coldplug/up | 4 - vm/app/lynx/etc/s6-rc/mdevd/notification-fd | 1 - .../etc/s6-rc/mdevd/notification-fd.license | 2 - vm/app/lynx/etc/s6-rc/mdevd/run | 5 - vm/app/lynx/etc/s6-rc/mdevd/type | 1 - vm/app/lynx/etc/s6-rc/mdevd/type.license | 2 - vm/app/lynx/etc/s6-rc/ok-all/contents | 4 - vm/app/lynx/etc/s6-rc/ok-all/type | 1 - vm/app/lynx/etc/s6-rc/ok-all/type.license | 2 - vm/app/lynx/etc/ssl/certs/ca-certificates.crt | 1 - .../host/data/appvm-lynx/providers/net/netvm | 0 vm/sys/net/Makefile | 23 ++-- vm/sys/net/default.nix | 10 +- 90 files changed, 221 insertions(+), 451 deletions(-) rename {vm/app/lynx => img/app}/Makefile (66%) rename {vm/app/catgirl => img/app}/bin (100%) rename {vm/app/catgirl => img/app}/default.nix (77%) create mode 100644 img/app/etc/fstab rename {vm/app/catgirl => img/app}/etc/init (100%) rename {vm/app/catgirl => img/app}/etc/mdev.conf (100%) rename {vm/app/lynx => img/app}/etc/mdev/iface (98%) rename {vm/app/catgirl => img/app}/etc/passwd (100%) rename {vm/app/catgirl => img/app}/etc/passwd.license (100%) rename {vm/app/catgirl => img/app}/etc/resolv.conf (100%) rename {vm/app/lynx => img/app}/etc/s6-linux-init/scripts/rc.init (90%) rename {vm/app/lynx/etc/s6-rc/lynx => img/app/etc/s6-rc/app}/run (80%) rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type (100%) rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/dependencies (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/up (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/run (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/contents (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type.license (100%) rename {vm/app/catgirl => img/app}/etc/ssl/certs/ca-certificates.crt (100%) rename {vm/app/lynx => img/app}/shell.nix (50%) rename {img => release}/combined/default.nix (100%) rename {img => release}/combined/eosimages.nix (100%) rename {img => release}/combined/grub.cfg.in (100%) rename {img => release}/combined/run-vm.nix (100%) rename {img => release}/installer/app/0001-gpt-disable-gpt-partition-attribute-55-check.patch (100%) rename {img => release}/installer/app/0002-gpt-disable-partition-table-CRC-check.patch (100%) rename {img => release}/installer/app/0003-install-remove-Endless-OS-ad.patch (100%) rename {img => release}/installer/app/0004-finished-don-t-run-eos-diagnostics.patch (100%) rename {img => release}/installer/app/0005-finished-promote-spectrum-not-the-Endless-forum.patch (100%) rename {img => release}/installer/app/default.nix (100%) rename {img => release}/installer/app/vendor-customer-support.ini (100%) rename {img => release}/installer/configuration.nix (100%) rename {img => release}/installer/default.nix (100%) rename {img => release}/installer/run-vm.nix (100%) rename {img => release}/installer/seat.rules (100%) rename {img => release}/live/Makefile (100%) rename {img => release}/live/default.nix (100%) rename {img => release}/live/shell.nix (100%) create mode 100644 vm-lib/make-vm.nix create mode 100644 vm/app/catgirl.nix delete mode 100644 vm/app/catgirl/Makefile delete mode 100644 vm/app/catgirl/etc/fstab delete mode 100755 vm/app/catgirl/etc/mdev/iface delete mode 100755 vm/app/catgirl/etc/s6-linux-init/scripts/rc.init delete mode 100755 vm/app/catgirl/etc/s6-rc/catgirl/run delete mode 100644 vm/app/catgirl/host/data/appvm-catgirl/providers/net/netvm delete mode 100644 vm/app/catgirl/shell.nix create mode 100644 vm/app/lynx.nix delete mode 120000 vm/app/lynx/bin delete mode 100644 vm/app/lynx/default.nix delete mode 100644 vm/app/lynx/etc/fstab delete mode 100755 vm/app/lynx/etc/init delete mode 100644 vm/app/lynx/etc/mdev.conf delete mode 100644 vm/app/lynx/etc/passwd delete mode 100644 vm/app/lynx/etc/passwd.license delete mode 100644 vm/app/lynx/etc/resolv.conf delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/dependencies delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/up delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/run delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/contents delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type.license delete mode 120000 vm/app/lynx/etc/ssl/certs/ca-certificates.crt delete mode 100644 vm/app/lynx/host/data/appvm-lynx/providers/net/netvm base-commit: 2cc5bf637f944ec950f7a8162018f421e6671315 -- 2.37.1

1 6

[PATCH] Add image configuration option
by José Pekkarinen 16 Sep '22

16 Sep '22

The following patch proposes to host nix configuration files under nix folder that offers default configuration for an image, defaulting to a release image, which would be plain spectrum. A hardened default configuration will be proposed in the near future. In case of configuration collision between the default configuration and config.nix, the latter will be taken into account. Signed-off-by: José Pekkarinen <jose.pekkarinen(a)unikie.com> --- nix/eval-config.nix | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nix/eval-config.nix b/nix/eval-config.nix index 467f877..09a7ada 100644 --- a/nix/eval-config.nix +++ b/nix/eval-config.nix @@ -10,4 +10,5 @@ ({ pkgs ? import <nixpkgs> {} }: { inherit pkgs; -}) config +}) (if config ? image && builtins.pathExists ./${config.image}.nix then + import ./${config.image}.nix else {}) // config -- 2.25.1

3 12

[PATCH 1/4] Introduce a build configuration file
by Alyssa Ross 14 Sep '22

14 Sep '22

By default, a file called "config.nix" in the root of the Spectrum repository will be read if it exists. That file should contain an attribute set. Currently, only a "pkgs" key is supported, which allows specifying a custom package set that will be used throughout the Spectrum Nix files. This will allow us to provide configuartion options for people who want to build Spectrum in ways that are probably not suitable for upstreaming. For example, using the "pkgs" config option I'm introducing here, it would be possible to use an overlay to patch individual components, like so: { pkgs = import <nixpkgs> { overlays = [ (final: super: { weston = super.weston.overrideAttrs ({ patches ? [], ... }: { patches = patches ++ [ path/to/weston.patch ]; }); }) ]; }; } Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- José, Ivan, if you'd like to review these patches, there's some documentation on how to do so here: :) https://spectrum-os.org/doc/reviewing-patches.html .gitignore | 1 + Documentation/default.nix | 4 ++-- Documentation/jekyll.nix | 2 +- host/initramfs/default.nix | 7 +++---- host/initramfs/extfs.nix | 8 ++++---- host/initramfs/shell.nix | 9 +++++---- host/rootfs/default.nix | 7 +++++-- host/rootfs/shell.nix | 8 ++++---- host/start-vm/default.nix | 2 +- host/start-vm/shell.nix | 6 +++--- img/combined/default.nix | 6 +++--- img/combined/eosimages.nix | 4 ++-- img/combined/run-vm.nix | 4 ++-- img/installer/default.nix | 3 ++- img/installer/run-vm.nix | 7 ++++--- img/live/default.nix | 11 ++++++----- img/live/shell.nix | 15 +++++++++------ nix/eval-config.nix | 10 ++++++++++ release.nix | 10 +++++++--- scripts/default.nix | 4 ++-- shell.nix | 2 +- vm/app/catgirl/default.nix | 6 +++--- vm/app/catgirl/shell.nix | 6 +++--- vm/app/lynx/default.nix | 6 +++--- vm/app/lynx/shell.nix | 6 +++--- vm/sys/net/default.nix | 6 +++--- vm/sys/net/shell.nix | 6 +++--- 27 files changed, 95 insertions(+), 71 deletions(-) create mode 100644 nix/eval-config.nix diff --git a/.gitignore b/.gitignore index 8b965e2..a97f309 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> +/config.nix build/ result result-* diff --git a/Documentation/default.nix b/Documentation/default.nix index d94a8cd..9edfbe8 100644 --- a/Documentation/default.nix +++ b/Documentation/default.nix @@ -2,7 +2,7 @@ # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( +{ config ? import ../nix/eval-config.nix {} }: config.pkgs.callPackage ( { lib, stdenvNoCC, jekyll, drawio-headless }: @@ -31,5 +31,5 @@ stdenvNoCC.mkDerivation { passthru = { inherit jekyll; }; } ) { - jekyll = import ./jekyll.nix { inherit pkgs; }; + jekyll = import ./jekyll.nix { inherit config; }; } diff --git a/Documentation/jekyll.nix b/Documentation/jekyll.nix index bc804b1..6f2866f 100644 --- a/Documentation/jekyll.nix +++ b/Documentation/jekyll.nix @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( +{ config ? import ../nix/eval-config.nix {} }: config.pkgs.callPackage ( { bundlerApp }: diff --git a/host/initramfs/default.nix b/host/initramfs/default.nix index a4f7330..68deb5e 100644 --- a/host/initramfs/default.nix +++ b/host/initramfs/default.nix @@ -1,11 +1,10 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} -, rootfs ? import ../rootfs { inherit pkgs; } +{ config ? import ../../nix/eval-config.nix {} +, rootfs ? import ../rootfs { inherit config; } }: - -pkgs.callPackage ( +let inherit (config) pkgs; in pkgs.callPackage ( { lib, stdenvNoCC, makeModulesClosure, runCommand, writeReferencesToFile , pkgsStatic, busybox, cpio, cryptsetup, lvm2, microcodeAmd, microcodeIntel }: diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 9fdbd9a..63f436a 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -1,21 +1,21 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs, runCommand, tar2ext4 }: +{ config, runCommand, tar2ext4 }: let netvm = import ../../vm/sys/net { - inherit pkgs; + inherit config; # inherit (foot) terminfo; }; appvm-catgirl = import ../../vm/app/catgirl { - inherit pkgs; + inherit config; # inherit (foot) terminfo; }; appvm-lynx = import ../../vm/app/lynx { - inherit pkgs; + inherit config; # inherit (foot) terminfo; }; in diff --git a/host/initramfs/shell.nix b/host/initramfs/shell.nix index 42da6a4..cbd2c60 100644 --- a/host/initramfs/shell.nix +++ b/host/initramfs/shell.nix @@ -1,16 +1,17 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: let + inherit (config) pkgs; inherit (pkgs.lib) cleanSource cleanSourceWith; extfs = pkgs.pkgsStatic.callPackage ./extfs.nix { - inherit pkgs; + inherit config; }; - rootfs = import ../rootfs { inherit pkgs; }; - initramfs = import ./. { inherit pkgs rootfs; }; + rootfs = import ../rootfs { inherit config; }; + initramfs = import ./. { inherit config rootfs; }; in with pkgs; diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index e5f316f..44e910b 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -2,7 +2,8 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-FileCopyrightText: 2022 Unikie -{ pkgs ? import <nixpkgs> {} }: pkgs.pkgsStatic.callPackage ( +{ config ? import ../../nix/eval-config.nix {} }: let inherit (config) pkgs; in +pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod @@ -13,7 +14,9 @@ let inherit (lib) cleanSource cleanSourceWith concatMapStringsSep hasSuffix; inherit (nixosAllHardware.config.hardware) firmware; - start-vm = import ../start-vm { pkgs = pkgs.pkgsStatic; }; + start-vm = import ../start-vm { + config = config // { pkgs = pkgs.pkgsStatic; }; + }; pkgsGui = pkgs.pkgsMusl.extend (final: super: { systemd = final.libudev-zero; diff --git a/host/rootfs/shell.nix b/host/rootfs/shell.nix index 3b2310f..8c30f68 100644 --- a/host/rootfs/shell.nix +++ b/host/rootfs/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { @@ -13,6 +13,6 @@ with pkgs; jq netcat qemu_kvm reuse util-linux ]; - EXT_FS = pkgsStatic.callPackage ../initramfs/extfs.nix { inherit pkgs; }; + EXT_FS = pkgsStatic.callPackage ../initramfs/extfs.nix { inherit config; }; KERNEL = "${passthru.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; }) diff --git a/host/start-vm/default.nix b/host/start-vm/default.nix index 56be882..fcce495 100644 --- a/host/start-vm/default.nix +++ b/host/start-vm/default.nix @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( +{ config ? import ../../nix/eval-config.nix {} }: config.pkgs.callPackage ( { lib, stdenv, fetchpatch, meson, ninja, rustc }: let diff --git a/host/start-vm/shell.nix b/host/start-vm/shell.nix index c7f6365..5192b76 100644 --- a/host/start-vm/shell.nix +++ b/host/start-vm/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { nativeBuildInputs ? [], ... }: { diff --git a/img/combined/default.nix b/img/combined/default.nix index 16cd506..3989d55 100644 --- a/img/combined/default.nix +++ b/img/combined/default.nix @@ -2,17 +2,17 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-FileCopyrightText: 2021 Yureka <yuka(a)yuka.dev> -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {} }: with config.pkgs; let inherit (builtins) storeDir; inherit (pkgs.lib) removePrefix; - eosimages = import ./eosimages.nix { inherit pkgs; }; + eosimages = import ./eosimages.nix { inherit config; }; installerPartUuid = "6e23b026-9f1e-479d-8a58-a0cda382e1ce"; installer = import ../installer { - inherit pkgs; + inherit config; extraConfig = { boot.initrd.availableKernelModules = [ "squashfs" ]; diff --git a/img/combined/eosimages.nix b/img/combined/eosimages.nix index 4ec28e5..9f2ab10 100644 --- a/img/combined/eosimages.nix +++ b/img/combined/eosimages.nix @@ -1,12 +1,12 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {} }: with config.pkgs; runCommand "eosimages.img" { nativeBuildInputs = [ e2fsprogs tar2ext4 ]; imageName = "Spectrum-0.0-x86_64-generic.0.Live.img"; - image = import ../live { inherit pkgs; }; + image = import ../live { inherit config; }; } '' mkdir dir cd dir diff --git a/img/combined/run-vm.nix b/img/combined/run-vm.nix index 893bc7d..40eacc4 100644 --- a/img/combined/run-vm.nix +++ b/img/combined/run-vm.nix @@ -1,10 +1,10 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {} }: with config.pkgs; let - image = import ./. { inherit pkgs; }; + image = import ./. { inherit config; }; in writeShellScript "run-spectrum-installer-vm.sh" '' diff --git a/img/installer/default.nix b/img/installer/default.nix index ba97a53..0c57704 100644 --- a/img/installer/default.nix +++ b/img/installer/default.nix @@ -1,7 +1,8 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {}, extraConfig ? {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {}, extraConfig ? {} }: +with config.pkgs; let inherit (builtins) head match storeDir; diff --git a/img/installer/run-vm.nix b/img/installer/run-vm.nix index 4a6c849..4efbf1a 100644 --- a/img/installer/run-vm.nix +++ b/img/installer/run-vm.nix @@ -1,17 +1,18 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: let inherit (builtins) storeDir; + inherit (config) pkgs; inherit (pkgs) coreutils qemu_kvm stdenv writeShellScript; inherit (pkgs.lib) makeBinPath escapeShellArg; - eosimages = import ../combined/eosimages.nix { inherit pkgs; }; + eosimages = import ../combined/eosimages.nix { inherit config; }; installer = import ./. { - inherit pkgs; + inherit config; extraConfig = { boot.initrd.availableKernelModules = [ "9p" "9pnet_virtio" ]; diff --git a/img/live/default.nix b/img/live/default.nix index 88f0ee4..65ad058 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -1,17 +1,18 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: let + inherit (config) pkgs; inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix; extfs = pkgs.pkgsStatic.callPackage ../../host/initramfs/extfs.nix { - inherit pkgs; + inherit config; }; - rootfs = import ../../host/rootfs { inherit pkgs; }; - scripts = import ../../scripts { inherit pkgs; }; - initramfs = import ../../host/initramfs { inherit pkgs rootfs; }; + rootfs = import ../../host/rootfs { inherit config; }; + scripts = import ../../scripts { inherit config; }; + initramfs = import ../../host/initramfs { inherit config rootfs; }; in with pkgs; diff --git a/img/live/shell.nix b/img/live/shell.nix index b9f0246..dcf2059 100644 --- a/img/live/shell.nix +++ b/img/live/shell.nix @@ -1,12 +1,15 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ({ nativeBuildInputs ? [], ... }: { - nativeBuildInputs = nativeBuildInputs ++ [ qemu_kvm ]; +(import ./. { inherit config; }).overrideAttrs ( + { nativeBuildInputs ? [], ... }: + { + nativeBuildInputs = nativeBuildInputs ++ [ qemu_kvm ]; - OVMF_CODE = "${qemu_kvm}/share/qemu/edk2-${stdenv.hostPlatform.qemuArch}-code.fd"; -}) + OVMF_CODE = "${qemu_kvm}/share/qemu/edk2-${stdenv.hostPlatform.qemuArch}-code.fd"; + } +) diff --git a/nix/eval-config.nix b/nix/eval-config.nix new file mode 100644 index 0000000..9265df7 --- /dev/null +++ b/nix/eval-config.nix @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2022 Unikie + +{ config ? + if builtins.pathExists ../config.nix then import ../config.nix else {} +}: + +({ pkgs ? import <nixpkgs> {} }: { + inherit pkgs; +}) config diff --git a/release.nix b/release.nix index da7123f..91a843b 100644 --- a/release.nix +++ b/release.nix @@ -3,10 +3,14 @@ # This file is built to populate the binary cache. -{ pkgs ? import <nixpkgs> {} }: +# Set config = {} to disable implicitly reading config.nix, since +# we'll want the result to be the same as on the binary cache. If it +# turns out there is a compelling reason to read the default config +# here, we can reconsider this. +{ config ? import nix/eval-config.nix { config = {}; } }: { - doc = import ./Documentation { inherit pkgs; }; + doc = import ./Documentation { inherit config; }; - combined = import img/combined/run-vm.nix { inherit pkgs; }; + combined = import img/combined/run-vm.nix { inherit config; }; } diff --git a/scripts/default.nix b/scripts/default.nix index 7995723..2237cb5 100644 --- a/scripts/default.nix +++ b/scripts/default.nix @@ -1,10 +1,10 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../nix/eval-config.nix {} }: let - inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix; + inherit (config.pkgs.lib) cleanSource cleanSourceWith hasSuffix; in cleanSourceWith { diff --git a/shell.nix b/shell.nix index 41d0865..4dc8558 100644 --- a/shell.nix +++ b/shell.nix @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import nix/eval-config.nix {} }: with config.pkgs; mkShell { nativeBuildInputs = [ reuse rustfmt ]; diff --git a/vm/app/catgirl/default.nix b/vm/app/catgirl/default.nix index 738a603..61f1462 100644 --- a/vm/app/catgirl/default.nix +++ b/vm/app/catgirl/default.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} -, terminfo ? pkgs.foot.terminfo +{ config ? import ../../../nix/eval-config.nix {} +, terminfo ? config.pkgs.foot.terminfo }: -pkgs.pkgsStatic.callPackage ( +config.pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4 diff --git a/vm/app/catgirl/shell.nix b/vm/app/catgirl/shell.nix index 87c4f6e..852b246 100644 --- a/vm/app/catgirl/shell.nix +++ b/vm/app/catgirl/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { diff --git a/vm/app/lynx/default.nix b/vm/app/lynx/default.nix index 50fd760..ba715ec 100644 --- a/vm/app/lynx/default.nix +++ b/vm/app/lynx/default.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} -, terminfo ? pkgs.foot.terminfo +{ config ? import ../../../nix/eval-config.nix {} +, terminfo ? config.pkgs.foot.terminfo }: -pkgs.pkgsStatic.callPackage ( +config.pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4 diff --git a/vm/app/lynx/shell.nix b/vm/app/lynx/shell.nix index 87c4f6e..852b246 100644 --- a/vm/app/lynx/shell.nix +++ b/vm/app/lynx/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index 0ce72fc..dfc7c35 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} -, terminfo ? pkgs.foot.terminfo +{ config ? import ../../../nix/eval-config.nix {} +, terminfo ? config.pkgs.foot.terminfo }: -pkgs.pkgsStatic.callPackage ( +config.pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4, xorg diff --git a/vm/sys/net/shell.nix b/vm/sys/net/shell.nix index bc4de67..849920d 100644 --- a/vm/sys/net/shell.nix +++ b/vm/sys/net/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { -- 2.37.1

4 11

[PATCH 1/2] host/rootfs: use initramfs in "make run"
by Alyssa Ross 08 Sep '22

08 Sep '22

This will allow us to stop compiling e.g. the virtio-blk module into the kernel, because it will be loaded by the initramfs. This introduces some duplication between the rootfs and initramfs's Makefiles. I don't think it's worth the effort at the moment to try to reduce that, because it would come at the expense of additional complexity in the Makefiles. We can revisit this later if we want to. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- host/rootfs/Makefile | 32 ++++++++++++++++++++++++++++---- host/rootfs/shell.nix | 10 ++++++++-- 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 41cf87c..31f76d2 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -6,6 +6,9 @@ # QEMU_KVM = qemu-system-x86_64 -enable-kvm. QEMU_KVM = qemu-kvm +SCRIPTS = ../../scripts +VERITYSETUP = veritysetup + # tar2ext4 will leave half a filesystem behind if it's interrupted # half way through. build/rootfs.ext4: build/rootfs.tar @@ -116,16 +119,37 @@ clean: rm -rf build .PHONY: clean -run: build/rootfs.ext4 $(EXT_FS) +# veritysetup format produces two files, but Make only (portably) +# supports one output per rule, so we combine the two outputs then +# define two more rules to separate them again. +build/rootfs.verity: build/rootfs.ext4 + $(VERITYSETUP) format build/rootfs.ext4 build/rootfs.verity.superblock.tmp \ + | awk -F ':[[:blank:]]*' '$$1 == "Root hash" {print $$2; exit}' \ + > build/rootfs.verity.roothash.tmp + cat build/rootfs.verity.roothash.tmp build/rootfs.verity.superblock.tmp \ + > $@ + rm build/rootfs.verity.roothash.tmp build/rootfs.verity.superblock.tmp +build/rootfs.verity.roothash: build/rootfs.verity + head -n 1 build/rootfs.verity > $@ +build/rootfs.verity.superblock: build/rootfs.verity + tail -n +2 build/rootfs.verity > $@ + +build/live.img: $(SCRIPTS)/format-uuid.sh $(SCRIPTS)/make-gpt.sh build/rootfs.verity.superblock build/rootfs.verity.roothash build/rootfs.ext4 + $(SCRIPTS)/make-gpt.sh $@.tmp \ + build/rootfs.verity.superblock:2c7357ed-ebd2-46d9-aec1-23d437ec2bf5:$$($(SCRIPTS)/format-uuid.sh "$$(dd if=build/rootfs.verity.roothash bs=32 skip=1 count=1 status=none)") \ + build/rootfs.ext4:4f68bce3-e8cd-4db1-96e7-fbcaf984b709:$$($(SCRIPTS)/format-uuid.sh "$$(head -c 32 build/rootfs.verity.roothash)") + mv $@.tmp $@ + +run: build/live.img $(EXT_FS) build/rootfs.verity.roothash $(QEMU_KVM) -cpu host -m 2G \ - -machine q35,kernel=$(KERNEL),kernel-irqchip=split \ + -machine q35,kernel=$(KERNEL),kernel-irqchip=split,initrd=$(INITRAMFS) \ -display gtk,gl=on \ -qmp unix:vmm.sock,server,nowait \ -monitor vc \ -parallel none \ - -drive file=build/rootfs.ext4,if=virtio,format=raw,readonly=on \ + -drive file=build/live.img,if=virtio,format=raw,readonly=on \ -drive file=$(EXT_FS),if=virtio,format=raw,readonly=on \ - -append "console=ttyS0 root=/dev/vda ext=/dev/vdb intel_iommu=on" \ + -append "console=ttyS0 roothash=$$(< build/rootfs.verity.roothash) ext=/dev/vdb intel_iommu=on" \ -device intel-iommu,intremap=on \ -device virtio-vga-gl \ -device vhost-vsock-pci,guest-cid=3 diff --git a/host/rootfs/shell.nix b/host/rootfs/shell.nix index 3b2310f..fe9df1b 100644 --- a/host/rootfs/shell.nix +++ b/host/rootfs/shell.nix @@ -1,18 +1,24 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie { pkgs ? import <nixpkgs> {} }: +let + rootfs = import ./. { inherit pkgs; }; +in + with pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +rootfs.overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { nativeBuildInputs = nativeBuildInputs ++ [ - jq netcat qemu_kvm reuse util-linux + cryptsetup jq netcat qemu_kvm reuse util-linux ]; EXT_FS = pkgsStatic.callPackage ../initramfs/extfs.nix { inherit pkgs; }; + INITRAMFS = import ../initramfs { inherit pkgs rootfs; }; KERNEL = "${passthru.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; }) -- 2.37.1

3 9

[PATCH v2] host/rootfs: remove kernel override
by José Pekkarinen 08 Sep '22

08 Sep '22

From: Alyssa Ross <hi(a)alyssa.is> These drivers should be loaded by the initramfs if required — most Spectrum installs won't need the virtio drivers on the host, and overriding the kernel means more stuff we can't reuse from cache.nixos.org. We'll probably want to build in the driver for whatever filesystem we end up using for the root file system eventually, since it will always be required, but that should be done as part of a more systematic effort to optimise our kernel configuration. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> Signed-off-by: José Pekkarinen <jose.pekkarinen(a)unikie.com> --- [ v1 -> v2 ] Request mdev to load modules on boot host/rootfs/default.nix | 9 +-------- host/rootfs/etc/modules-autoload.d/modules.conf | 4 ++++ 2 files changed, 5 insertions(+), 8 deletions(-) create mode 100644 host/rootfs/etc/modules-autoload.d/modules.conf diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index e5f316f..a651a20 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -65,14 +65,7 @@ let imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; }); - kernel = pkgs.linux_latest.override { - structuredExtraConfig = with lib.kernel; { - VIRTIO = yes; - VIRTIO_PCI = yes; - VIRTIO_BLK = yes; - EXT4_FS = yes; - }; - }; + kernel = pkgs.linux_latest; packagesSysroot = runCommand "packages-sysroot" { nativeBuildInputs = [ xorg.lndir ]; diff --git a/host/rootfs/etc/modules-autoload.d/modules.conf b/host/rootfs/etc/modules-autoload.d/modules.conf new file mode 100644 index 0000000..dd9529a --- /dev/null +++ b/host/rootfs/etc/modules-autoload.d/modules.conf @@ -0,0 +1,4 @@ +virtio +virtio_pci +virtio_blk +ext4 -- 2.25.1

1 0

[PATCH] Documentation: add "Sending your First Patch"
by Alyssa Ross 11 Aug '22

11 Aug '22

I think having this written down in tutorial style is important, so people can follow through some step by step instructions. We already have some similar documentation on <https://spectrum-os.org/contributing.html>, but it doesn't really go into any detail, and I think this belongs in the main documentation. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- Ville, I'm CCing you in case you'd like to review, since we talked about documenting the patch workflow a bit better before. We also have this small document about reviewing patches on the list now, at <https://spectrum-os.org/doc/reviewing-patches.html>. :) Documentation/first-patch.adoc | 96 ++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 Documentation/first-patch.adoc diff --git a/Documentation/first-patch.adoc b/Documentation/first-patch.adoc new file mode 100644 index 0000000..22140e4 --- /dev/null +++ b/Documentation/first-patch.adoc @@ -0,0 +1,96 @@ += Sending Your First Patch +:page-parent: Tutorials + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +== Prerequisites + +This tutorial assumes that you already have basic +https://git-scm.com/[git] experience. + +Before beginning, you'll need to have a local git checkout of the +https://spectrum-os.org/git/[Spectrum source tree]. You'll also need +to have configured `git send-email` — a guide for this can be found at +https://git-send-email.io/. + +== Making your changes + +If you've worked on any git repository before, the process for making +your changes will probably be very familiar. + +1. Create a branch for your changes: ++ +---- +git checkout -b fix-docs # for example +---- +2. Make changes in your editor. +3. Stage and commit your changes: ++ +---- +git add Documentation/first-patch.adoc # for example +git commit -s +---- ++ +TIP: The `-s` option adds a `Signed-off-by` line to the commit. This +indicates your acceptance of the +https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's +Certificate of Origin], which is mandatory for Spectrum patches. + +== Submitting changes + +Once you're happy with how the commits on your branch look, run: + +---- +git send-email --to devel(a)spectrum-os.org --confirm=always origin/main +---- + +For each message, you will be prompted for confirmation, and then the +change will be submitted to the Spectrum development mailing list. +They will appear shortly on the +https://spectrum-os.org/lists/hyperkitty/list/devel@spectrum-os.org/[web +interface]. + +[#cover-letter] +TIP: If you're submitting multiple changes, and you'd like to provide +an overall description for the whole series, you can add the +`--compose` option, which will prompt you to compose a "cover letter" +message that will be sent before all of your patches. + +== Feedback + +Once your patch has been submitted, wait for it to be reviewed. +Feedback, if any, will be sent as email replies to your submitted +patch. You can respond to feedback in your mail client. Please use +the Reply All button to ensure that your messages are sent to the +mailing list as well as to the person who sent the feedback. + +If you need to make changes to your patch, and submit a new version, +use https://git-rebase.io/[`git rebase`] to create a new version of +your patch(es), and submit it like this: + +---- +git send-email -v2 --to devel(a)spectrum-os.org --confirm=always origin/main +---- + +The added `-v2` flag indicates that this is version two of your +patch set. If your patches require more rounds of changes, submit +subsequent rounds with `-v3`, `-v4`, etc. as appropriate. + +If you'd like to describe what has changed from the previous version +of your patches, you can do so in a xref:cover-letter[cover letter] +as described above. + +== Acceptance + +Once your patch has been accepted, it will be applied to the upstream +Spectrum git repository. You will receive an automated email +confirmation when this happens. + +== Questions + +If you have any questions about the patch submission process, you can +ask them either on the +https://spectrum-os.org/participating.html#spectrum-devel[development +mailing list], or in the +https://spectrum-os.org/participating.html#irc[IRC/Matrix channel]. -- 2.35.1

2 1

Cross-compilation support - call for test / nix-help
by Ville Ilvonen 13 Jul '22

13 Jul '22

Hi all, As Ivan (@jonne) updated earlier this week on #spectrum, we've got initial cross-compilation support for aarch64 available in Spectrum development fork. It still needs some TLC before Spectrum PRs but is a great step forward. Our current ref. board is NXP i.MX8 but rootfs build should be fairly easy to modify for RPi4 configs already in nixpkgs for anyone who cares to try. Quick how-to for non-nixos host like Ubuntu: git clone -b aarch64-imx8-crosscompile https://github.com/tiiuae/nixpkgs-spectrum.git git clone -b aarch64-imx8-crosscompile https://github.com/tiiuae/spectrum.git cd spectrum/host/rootfs nix-build -I nixpkgs=../../../nixpkgs-spectrum Know issue: - crossSystem from https://github.com/tiiuae/spectrum/commit/aad5f90cf2ec931f2af5c8d5111a9fac1… is ignored on NixOS host but not on Ubuntu - maybe someone can point us why. Thanks, -Ville

2 3

[PATCH] doc: architecture view with drawio to svg generation
by Ville Ilvonen 15 Jun '22

15 Jun '22

* Initial architecture document view a view to high level stack of Spectrum. * Stack view is generated to svg from drawio using Alyssa's drawio-headless in nixpkgs-upstream. Cherry-picked to nixpkgs-spectrum for testing. * An example to analyze the details of Spectrum dependencies interactively is also provided. Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- Documentation/architecture.adoc | 39 +++++++++++++++++++++++++++++ Documentation/default.nix | 8 +++--- Documentation/diagrams/stack.drawio | 1 + 3 files changed, 45 insertions(+), 3 deletions(-) create mode 100644 Documentation/architecture.adoc create mode 100644 Documentation/diagrams/stack.drawio diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc new file mode 100644 index 0000000..2f89e68 --- /dev/null +++ b/Documentation/architecture.adoc @@ -0,0 +1,39 @@ += Architecture +// SPDX-FileCopyrightText: 2022 Ville Ilvonen <ville.ilvonen(a)unikie.com> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +== Introduction + +Spectrum operating system stack is based on the principle of security by compartmentalization. The high level system stack is illustrated in the following diagram. + +image::diagrams/stack.svg[] + +=== Kernel space + +In the stack, kernel space security by compartmentalization is supported with linux kernel that includes kernel-based virtual machine (KVM) module enabling the kernel to work as virtual-machine manager, hypervisor. Kernel side hypervisor supports virtualization of hardware resources - computational cores, memory and devices - securely. Userspace virtual machine guests are managed with cloud-hypervisor. Linux with KVM also supports portability to several hardware architectures. Currently Spectrum is supported only on x86_64 but ARM64 is under works. In addition, hardened kernel is to be enabled. + +=== Host user space + +This section provides high level overview of host user space tools and libraries. + +User space stack is build on musl standard C library with added safety on resource exhaustion and security hardening on memory allocation. + +https://skarnet.org/software/s6-rc/overview.html[s6-rc] service manager is used for services. kmod, util-linux and busybox are provided for essential system administration. + +https://github.com/cloud-hypervisor/cloud-hypervisor[cloud-hypervisor] is a host tooling for virtual machine management, written in Rust with a strong focus on security. + +Wayland refers to whole display stack providing communication with compositor (weston) for desktop services, including libraries and drivers for direct rendering and event devices. Clients are implemented as application virtual machines (see next section). Minimal host provides only Wayland terminal client, foot. Wayland, a simpler and more secure, protocol for compositor could provide support for legacy X applications as well but as of now none are provided. https://wayland.freedesktop.org/architecture.html[Wayland architecture] is well documented here. + +=== Application and system virtual machines + +Security by compartmentalization in Spectrum is implemented with virtual machines. Virtual machines currently launch using terminal and support only wayland-console. Wayland graphics support for appvms is under work. Please refer to https://spectrum-os.org/doc/running-vms.html[running VMs] for more information. + +Reference set of virtual machines includes system machine, netvm, and application VMs, appvm-catgirl and appvm-elinks. Please refer to https://spectrum-os.org/doc/creating-vms.html[creating VMs] for more information. + +== Details of Spectrum dependency tree + +High level overview of Spectrum stack is limited view to the system. For detailed, interactive view to dependencies please use `nix-tree` under the spectrum repository: + +`nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree` + +https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[See video of Spectrum live image interactive analysis with nix-tree] \ No newline at end of file diff --git a/Documentation/default.nix b/Documentation/default.nix index 02b3c31..8b969d4 100644 --- a/Documentation/default.nix +++ b/Documentation/default.nix @@ -3,7 +3,7 @@ { pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( -{ lib, runCommand, jekyll }: +{ lib, runCommand, jekyll, drawio-headless }: runCommand "spectrum-docs" { src = with lib; cleanSourceWith { @@ -14,11 +14,13 @@ runCommand "spectrum-docs" { !(hasSuffix ".nix" name); }; - nativeBuildInputs = [ jekyll ]; + nativeBuildInputs = [ jekyll drawio-headless ]; passthru = { inherit jekyll; }; -} '' +} + '' jekyll build --disable-disk-cache -b /doc -s $src -d $out + drawio --recursive $out/diagrams/ --export -f svg $out/assets/images/ '' ) { jekyll = import ./jekyll.nix { inherit pkgs; }; diff --git a/Documentation/diagrams/stack.drawio b/Documentation/diagrams/stack.drawio new file mode 100644 index 0000000..23feae7 --- /dev/null +++ b/Documentation/diagrams/stack.drawio @@ -0,0 +1 @@ +<mxfile host="Electron" modified="2022-05-24T12:19:30.186Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.7.3 Chrome/91.0.4472.164 Electron/13.6.1 Safari/537.36" etag="8D9FBMK4C5lWSHGzQpql" version="15.7.3" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7VrbcuI4EP0aqiYPTPkOPBIyJJtkdneSnUnmaUtggVWRLUeSueTrV7JljC8ZHArjhZoXbLdal+5z1FJLdMyRv7qmIPS+EhfijqG5q4551TEMXdcM8ZCStZJohp1I5hS5SpYJHtEbTBWVNEIuZDlFTgjmKMwLpyQI4JTnZIBSssyrzQjO9xqCOSwJHqcAl6VPyOVeIu0bvUx+A9HcS3vWnUFS4oNUWVnCPOCS5ZbI/NIxR5QQnrz5qxHE0nupX57+WD/h+xfn+vYbewXfL+/++fNHN2ls/JEqGxMoDPjeTU+e7/r61e34/vvPt8Xz8hX2//qmqmgLgCPlr47hYNHJ5YyIvoS/wTQpcF4jaejlA5kQTrJv6Ri+xnklWbnLYi4MhYJhhKvtGs5cPm8AdZfCJqHwadV3LtKehQlJ54macv6mD4OSKHChtEoTxUsPcfgYJsNcChILmcd9LL70Te2drlMuXkDK4WqLOMqV15D4kNO1UFGlG36reeFY6nuZkcx2lMzbIlgqA4rX803TGXbiRcH3ASj1FqG8R0G0io0Xthva3Y+vJw2m3W8bTKNFMKeYRG7XW4eQLhAj9KSgNPt5KC2tDKWhHRNKswTlHaRBvMyy2D1FPwpLed5ZjFPyAkcECzDMq4AEUGKJMC6IAEbzQHxOhVOhkF9KvyGxFg5VgY9cV3ZTiU4ePwn1GPgIS0cqciVStcZLFJJvEtG4HY9zsagbtuSWLXwmf6QC+zwnZI4hCBH7PCV+XDBlsep4lvQhXrNeGiNHYZoP7BI39KppbjbFDavEjRvC5PSOmMDvN0FaJojVa5sgdovrgB8xMT4H+BL7YMLkoyu0R0XhuwtEIhYkDt4fhYI3Hsem4Y5hmlbMOGVMXixV499adjAOAlfsNX8x7GSI+WE372CMJhRIkpzQ+lraKhltb5V6FVOk4DkYuEOZSmaBcMtTtaPawaNYMVa6gHnxQD4OIHRzWXAZvm14KtBJZRRiwNEinztXQaZ6+JugeJak7HhnH522wGIfqkrbuWqhHX2woyEO6BzyUkMxgTZW1+LUM354ewrGV7fL4XDQ5Xr47+T+IGnxR4LgzhDmdOl0R7xqIHyUwkJdQtYOH5ZTDh+DhqJHJdLl3Xm7SC/BGovl6hywtmukYrp+TLDL2+12wZ5EbD0hq3MA29F2T2y9d0ywD7F1PiTYM3lQHO9x48YmNC34JFIx/+IsSDDYPeP7x+SA8z/jwItPziK0m3Vmu3VMpAcHQHpfVCOOcBcnR+Clye2jAPkAX5SLTgpxy6qBeFN5XyXiepsXHkwkILy78E8Kw9Lmu2pDdtQ1Wm/zniOAPANwa8ayNVv4pz5fi1hvMvxtrKsyLd1uDOyqXOtsTmoKJzD7A3ikkxo7Rw9zMMi3UP+kZkdDDZ/U6IfI6faNIIgyhioiCAjD048gllYjglStFg1GkDavQ6DY4b2wc0Xb7u1Gu/LevEG0y9nbA5xBYaa8E9UWiPII4LLffTD1UABZydG/702bYk/h3tTUayYHe9ybis/sL3fJKpL9c9H88h8=</diagram></mxfile> \ No newline at end of file -- 2.33.3

3 8

[PATCH nixpkgs] stdenv: update aarch64 musl bootstrap tools
by Alyssa Ross 13 Jun '22

13 Jun '22

This will allow removing some workarounds (661dfd83478 and the preceding commits) that I had to do to get aarch64 musl to bootstrap, and hopefully will also fix some of the bootstrapping failures we've been seeing since then on newer aarch64 hardware. This approach unfortunately requires users to have the Spectrum binary cache set up, but it's the best we can do for now. When it's stabilized, we could use builtins.fetchClosure, which would avoid that requirement. But hopefully by that point we can have the bootstrap tools updated upstream anyway. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- .../linux/bootstrap-files/aarch64-musl.nix | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix b/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix index 252783cce47..64c950529e9 100644 --- a/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix +++ b/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix @@ -1,11 +1,9 @@ +let + # This path is available on the Spectrum binary cache. + # See <https://spectrum-os.org/doc/binary-cache.html>. + build = builtins.storePath /nix/store/bq9z1m3dvcsk522j9li43gm61caj5ybr-stdenv-bootstrap-tools-aarch64-unknown-linux-musl; +in { - busybox = import <nix/fetchurl.nix> { - url = "https://wdtz.org/files/wjzsj9cmdkc70f78yh072483x8656nci-stdenv-bootstrap-to…"; - sha256 = "01s6bwq84wyrjh3rdsgxni9gkzp7ss8rghg0cmp8zd87l79y8y4g"; - executable = true; - }; - bootstrapTools = import <nix/fetchurl.nix> { - url = "https://wdtz.org/files/wjzsj9cmdkc70f78yh072483x8656nci-stdenv-bootstrap-to…"; - sha256 = "0pbqrw9z4ifkijpfpx15l2dzi00rq8c5zg9ghimz5qgr5dx7f7cl"; - }; + busybox = "${build}/on-server/busybox"; + bootstrapTools = "${build}/on-server/bootstrap-tools.tar.xz"; } -- 2.35.1

1 0

[PATCH] Allow static building of dtc
by Tero Tervala 25 May '22

25 May '22

Signed-off-by: Tero Tervala <tero.tervala(a)unikie.com> --- ...01-Use-libfdt.a-if-building-staticly.patch | 54 +++++++++++++++++++ pkgs/development/compilers/dtc/default.nix | 4 ++ 2 files changed, 58 insertions(+) create mode 100644 pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch diff --git a/pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch b/pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch new file mode 100644 index 00000000000..4f0d6193030 --- /dev/null +++ b/pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch @@ -0,0 +1,54 @@ +From 59258caa6e02b5a7d5afc0bfde777b250ef74bbb Mon Sep 17 00:00:00 2001 +From: Tero Tervala <tero.tervala(a)unikie.com> +Date: Wed, 18 May 2022 11:20:40 +0300 +Subject: [PATCH] Use libfdt.a if building staticly + +Signed-off-by: Tero Tervala <tero.tervala(a)unikie.com> +--- + Makefile | 14 ++++++++++---- + 1 file changed, 10 insertions(+), 4 deletions(-) + +diff --git a/Makefile b/Makefile +index ea8c659..9474b63 100644 +--- a/Makefile ++++ b/Makefile +@@ -151,7 +151,7 @@ BIN += fdtoverlay + + SCRIPTS = dtdiff + +-all: $(BIN) libfdt ++all: libfdt $(BIN) + + # We need both Python and swig to build/install pylibfdt. + # This builds the given make ${target} if those deps are found. +@@ -198,6 +198,12 @@ LIBFDT_lib = $(LIBFDT_dir)/$(LIBFDT_LIB) + LIBFDT_include = $(addprefix $(LIBFDT_dir)/,$(LIBFDT_INCLUDES)) + LIBFDT_version = $(addprefix $(LIBFDT_dir)/,$(LIBFDT_VERSION)) + ++ifneq (,$(findstring -static,$(CFLAGS))) ++ LIBFDT_dep = $(LIBFDT_archive) ++else ++ LIBFDT_dep = $(LIBFDT_lib) ++endif ++ + include $(LIBFDT_dir)/Makefile.libfdt + + .PHONY: libfdt +@@ -261,11 +267,11 @@ convert-dtsv0: $(CONVERT_OBJS) + + fdtdump: $(FDTDUMP_OBJS) + +-fdtget: $(FDTGET_OBJS) $(LIBFDT_lib) ++fdtget: $(FDTGET_OBJS) $(LIBFDT_dep) + +-fdtput: $(FDTPUT_OBJS) $(LIBFDT_lib) ++fdtput: $(FDTPUT_OBJS) $(LIBFDT_dep) + +-fdtoverlay: $(FDTOVERLAY_OBJS) $(LIBFDT_lib) ++fdtoverlay: $(FDTOVERLAY_OBJS) $(LIBFDT_dep) + + dist: + git archive --format=tar --prefix=dtc-$(dtc_version)/ HEAD \ +-- +2.25.1 + diff --git a/pkgs/development/compilers/dtc/default.nix b/pkgs/development/compilers/dtc/default.nix index 04dca4772a9..33369fb1fcc 100644 --- a/pkgs/development/compilers/dtc/default.nix +++ b/pkgs/development/compilers/dtc/default.nix @@ -12,6 +12,10 @@ stdenv.mkDerivation rec { sha256 = "sha256-gx9LG3U9etWhPxm7Ox7rOu9X5272qGeHqZtOe68zFs4="; }; + patches = [ + ./0001-Use-libfdt.a-if-building-staticly.patch + ]; + buildInputs = [ libyaml ]; nativeBuildInputs = [ flex bison pkg-config which ] ++ lib.optionals pythonSupport [ python swig ]; -- 2.33.3

2 3

Re: [PATCH] doc: architecture view with drawio to svg generation
by Ville Ilvonen 24 May '22

24 May '22

This one with a large gif can be ignored. I send one without and link to video in diode.zone. On Tue, May 24, 2022 at 3:53 PM Ville Ilvonen <ville.ilvonen(a)unikie.com> wrote: > > * Initial architecture document view a view to high level > stack of Spectrum. > * Stack view is generated to svg from drawio using Alyssa's > drawio-headless in nixpkgs-upstream. Cherry-picked to > nixpkgs-spectrum for testing. > * An example to analyze the details of Spectrum dependencies > interactively is also provided. > > Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> > --- > Documentation/architecture.adoc | 39 ++++++++++++++++++++++++++++ > Documentation/default.nix | 8 +++--- > Documentation/diagrams/stack.drawio | 1 + > Documentation/spectrum-nix-tree.gif | Bin 0 -> 6007123 bytes > 4 files changed, 45 insertions(+), 3 deletions(-) > create mode 100644 Documentation/architecture.adoc > create mode 100644 Documentation/diagrams/stack.drawio > create mode 100644 Documentation/spectrum-nix-tree.gif > > diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc > new file mode 100644 > index 0000000..40003a2 > --- /dev/null > +++ b/Documentation/architecture.adoc > @@ -0,0 +1,39 @@ > += Architecture > +// SPDX-FileCopyrightText: 2022 Ville Ilvonen <ville.ilvonen(a)unikie.com> > +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 > + > +== Introduction > + > +Spectrum operating system stack is based on the principle of security by compartmentalization. The high level system stack is illustrated in the following diagram. > + > +image::diagrams/stack.svg[] > + > +=== Kernel space > + > +In the stack, kernel space security by compartmentalization is supported with linux kernel that includes kernel-based virtual machine (KVM) module enabling the kernel to work as virtual-machine manager, hypervisor. Kernel side hypervisor supports virtualization of hardware resources - computational cores, memory and devices - securely. Userspace virtual machine guests are managed with cloud-hypervisor. Linux with KVM also supports portability to several hardware architectures. Currently Spectrum is supported only on x86_64 but ARM64 is under works. In addition, hardened kernel is to be enabled. > + > +=== Host user space > + > +This section provides high level overview of host user space tools and libraries. > + > +User space stack is build on musl standard C library with added safety on resource exhaustion and security hardening on memory allocation. > + > +https://skarnet.org/software/s6-rc/overview.html[s6-rc] service manager is used for services. kmod, util-linux and busybox are provided for essential system administration. > + > +https://github.com/cloud-hypervisor/cloud-hypervisor[cloud-hypervisor] is a host tooling for virtual machine management, written in Rust with a strong focus on security. > + > +Wayland refers to whole display stack providing communication with compositor (weston) for desktop services, including libraries and drivers for direct rendering and event devices. Clients are implemented as application virtual machines (see next section). Minimal host provides only Wayland terminal client, foot. Wayland, a simpler and more secure, protocol for compositor could provide support for legacy X applications as well but as of now none are provided. https://wayland.freedesktop.org/architecture.html[Wayland architecture] is well documented here. > + > +=== Application and system virtual machines > + > +Security by compartmentalization in Spectrum is implemented with virtual machines. Virtual machines currently launch using terminal and support only wayland-console. Wayland graphics support for appvms is under work. Please refer to https://spectrum-os.org/doc/running-vms.html[running VMs] for more information. > + > +Reference set of virtual machines includes system machine, netvm, and application VMs, appvm-catgirl and appvm-elinks. Please refer to https://spectrum-os.org/doc/creating-vms.html[creating VMs] for more information. > + > +== Details of Spectrum dependency tree > + > +High level overview of Spectrum stack is limited view to the system. For detailed, interactive view to dependencies please use `nix-tree` under the spectrum repository: > + > +`nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree` > + > +image::spectrum-nix-tree.gif["Example of Spectrum live image interactive analysis"] > \ No newline at end of file > diff --git a/Documentation/default.nix b/Documentation/default.nix > index 02b3c31..8b969d4 100644 > --- a/Documentation/default.nix > +++ b/Documentation/default.nix > @@ -3,7 +3,7 @@ > > { pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( > > -{ lib, runCommand, jekyll }: > +{ lib, runCommand, jekyll, drawio-headless }: > > runCommand "spectrum-docs" { > src = with lib; cleanSourceWith { > @@ -14,11 +14,13 @@ runCommand "spectrum-docs" { > !(hasSuffix ".nix" name); > }; > > - nativeBuildInputs = [ jekyll ]; > + nativeBuildInputs = [ jekyll drawio-headless ]; > > passthru = { inherit jekyll; }; > -} '' > +} > + '' > jekyll build --disable-disk-cache -b /doc -s $src -d $out > + drawio --recursive $out/diagrams/ --export -f svg $out/assets/images/ > '' > ) { > jekyll = import ./jekyll.nix { inherit pkgs; }; > diff --git a/Documentation/diagrams/stack.drawio b/Documentation/diagrams/stack.drawio > new file mode 100644 > index 0000000..23feae7 > --- /dev/null > +++ b/Documentation/diagrams/stack.drawio > @@ -0,0 +1 @@ > +<mxfile host="Electron" modified="2022-05-24T12:19:30.186Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.7.3 Chrome/91.0.4472.164 Electron/13.6.1 Safari/537.36" etag="8D9FBMK4C5lWSHGzQpql" version="15.7.3" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7VrbcuI4EP0aqiYPTPkOPBIyJJtkdneSnUnmaUtggVWRLUeSueTrV7JljC8ZHArjhZoXbLdal+5z1FJLdMyRv7qmIPS+EhfijqG5q4551TEMXdcM8ZCStZJohp1I5hS5SpYJHtEbTBWVNEIuZDlFTgjmKMwLpyQI4JTnZIBSssyrzQjO9xqCOSwJHqcAl6VPyOVeIu0bvUx+A9HcS3vWnUFS4oNUWVnCPOCS5ZbI/NIxR5QQnrz5qxHE0nupX57+WD/h+xfn+vYbewXfL+/++fNHN2ls/JEqGxMoDPjeTU+e7/r61e34/vvPt8Xz8hX2//qmqmgLgCPlr47hYNHJ5YyIvoS/wTQpcF4jaejlA5kQTrJv6Ri+xnklWbnLYi4MhYJhhKvtGs5cPm8AdZfCJqHwadV3LtKehQlJ54macv6mD4OSKHChtEoTxUsPcfgYJsNcChILmcd9LL70Te2drlMuXkDK4WqLOMqV15D4kNO1UFGlG36reeFY6nuZkcx2lMzbIlgqA4rX803TGXbiRcH3ASj1FqG8R0G0io0Xthva3Y+vJw2m3W8bTKNFMKeYRG7XW4eQLhAj9KSgNPt5KC2tDKWhHRNKswTlHaRBvMyy2D1FPwpLed5ZjFPyAkcECzDMq4AEUGKJMC6IAEbzQHxOhVOhkF9KvyGxFg5VgY9cV3ZTiU4ePwn1GPgIS0cqciVStcZLFJJvEtG4HY9zsagbtuSWLXwmf6QC+zwnZI4hCBH7PCV+XDBlsep4lvQhXrNeGiNHYZoP7BI39KppbjbFDavEjRvC5PSOmMDvN0FaJojVa5sgdovrgB8xMT4H+BL7YMLkoyu0R0XhuwtEIhYkDt4fhYI3Hsem4Y5hmlbMOGVMXixV499adjAOAlfsNX8x7GSI+WE372CMJhRIkpzQ+lraKhltb5V6FVOk4DkYuEOZSmaBcMtTtaPawaNYMVa6gHnxQD4OIHRzWXAZvm14KtBJZRRiwNEinztXQaZ6+JugeJak7HhnH522wGIfqkrbuWqhHX2woyEO6BzyUkMxgTZW1+LUM354ewrGV7fL4XDQ5Xr47+T+IGnxR4LgzhDmdOl0R7xqIHyUwkJdQtYOH5ZTDh+DhqJHJdLl3Xm7SC/BGovl6hywtmukYrp+TLDL2+12wZ5EbD0hq3MA29F2T2y9d0ywD7F1PiTYM3lQHO9x48YmNC34JFIx/+IsSDDYPeP7x+SA8z/jwItPziK0m3Vmu3VMpAcHQHpfVCOOcBcnR+Clye2jAPkAX5SLTgpxy6qBeFN5XyXiepsXHkwkILy78E8Kw9Lmu2pDdtQ1Wm/zniOAPANwa8ayNVv4pz5fi1hvMvxtrKsyLd1uDOyqXOtsTmoKJzD7A3ikkxo7Rw9zMMi3UP+kZkdDDZ/U6IfI6faNIIgyhioiCAjD048gllYjglStFg1GkDavQ6DY4b2wc0Xb7u1Gu/LevEG0y9nbA5xBYaa8E9UWiPII4LLffTD1UABZydG/702bYk/h3tTUayYHe9ybis/sL3fJKpL9c9H88h8=</diagram></mxfile> > \ No newline at end of file > diff --git a/Documentation/spectrum-nix-tree.gif b/Documentation/spectrum-nix-tree.gif > new file mode 100644 > index 0000000000000000000000000000000000000000..bb367556353d034e3317b04e3dae5340b290cfff > GIT binary patch > literal 6007123 > zcmW(+byU>P+x^g~AkqykxO7TLr;9YWbW6LWf~0iUf(S@=EFrO!l%#~D2unB8CC$RV > z-}m?5oH_H%z2`i4&YZdD&MP%l2}x^u{BrCo79jutKmY&%02lzk0RRaAC;&kJZx#ds > zARqt+0^lG32?9_c0R6wEAP4|~05AvuhX6<jfPw(%|J8$F00aiWU;rEjAYlLs2B819 > zDF_Zg-~bE`z~KNA4xr!w`hQ3uBmhAIFeCs+0!So)LIUXj!Gll$1O>oQ02~D%Q2+`B > zp#R4Sga#mJ0EPzOXaI=@P-p=C|3sl6015%1FaQb%phy6U0-$IB3V@&>5EKG}!az_s > z2#N$jQ6MN91O*^a5CjT=Kw%Il90El`peP6w4S@nMC<q3Hz@RV~6b^$TVNetdiiSY} > zI1~hjLf}vs914d+k#Hyq4n@PE01^s9LLo>f3<-rJp-3bYg@mG!Pyhu5p`Z{H6o!Jr > zQBWibib6rrC@6r2g3wS18VW;0;b<rl4Mm}$XfzZ65Fh{n0T3_%0S6FB0D%G!XaE6# > z5Fii&0z$w*2sj9V1R+o$1R8_@AP5iy0f8W35Cj~8Ktd2G2m%d305AjyhJe5jFc<<3 > zLm*)Y6byleApkf61V=#N2pAjzha-@11PYEo!w~=y0YV}mNCXUtfFlt|Bm#v*ppggw > zg#e)t5EKH2Lcmc7Bnp8-A<!rUfJT7O2nZShLnGj51QLxvp%G{_0st@|00RLqFaQGw > zFh~G{0x)O*1As6f5C#Imz(5!{2!jM+P#_E%gaIHJ5Cj8(U|<jo9D+eYFenHH4Z#2~ > z3<!pSz%Vcv1`fj@VHgw)gN9)MI0gjAK;Re{90P}AkZ=qNjzPmQ01^X2VjxHi42gjw > zF-Rl^g~Xtd7yyL<p)e2>28P1GQ5Yl&gF<1@C=7tcfY2BS8UsUP;AjjIjX|L?Xf%cm > zL!zbfMqf!mM@5)l5C`i&L9yN7;^F};0PlZe;Qx;aK!^c2Ku<EPtWrX8AF)f9*IM^P > zk+Sh;?+6yE#8U9sw3pYt9Y|tSaAn*TDppHjS1**VsJ9zV=e4fM-W4v@%oKjN-d@q* > zFqS7Ba>TeNQm$L57{@MyKsrs8X%z71?1>_v2s#qI>3R4&#c!nDo=fyi2SyE!;|R?W > z@9Kpn_m$xU^^Y}6-+hkvMn^t=Tj>nO5I@%Nsa@-dCZiDrM|fa|z{zCAAnu-<xM@`O > zk+iKMTjRM(Va$BrMR$Lf>K9p5e}Az*Tl2Ozm#<CiaG~k_#&UI=`0+|-7!C`6yTs|n > zz~^U{HSLoB{*D!42GCGXuRZ(cNq?Br+6<0Uy5cZo+RXoa{%r}B$$ew)xVt==8?n^J > z+y~rUZ2ax|#?pm(0M4GTe!_ha&kw=F&sYs1X5X<2BNqm*g;L5&SUaY>OvTDzKJ2VT > zu||T|qd{qo>oMTc^7UBWrk(XT!2$3_yy%SMMuNn4`9`Aj<<3TuJU-VZT$$Qw6W$e_ > zVH>R;irS3Q_M^3n(tpt3N;a~u*h(|=-rY*K%Dh&WG8I)emcqD~+h@XRcI~s=JGmUP > zKa4p!<oGOCIOGN#?K<QIKX5tbhd*|9EQn@CI2Ok9?>QF1Wx1V-Q?;C(N-``EPNmtd > zdroEfpSYdNixd5vqTC9J%MkDChf6CNdoX*}h!4Zg@2Xp-5c`PMv+wUJJFxHeE4wMA > zU{(F>{s&c^Vmwt11B#VZ$X~MWT#&O`SuTwz@5<`t#ZNr0Et|{Vt6Hb#3(0LSMQ_xr > zj(T`X+TTpQE9p3TT_N!0&ScK56ODiH?FT>xbMGd^R0?*Juq?XwkiI&o?IYKQdGyoD > zRSESoST1@DFh(8J4YDS|JcqzRRl-9&d5fOIygv`>e+iDm-j7OjRf&vBtt`GDlO{N9 > zn2;w66zr6Hu8Evdr8yM+tu4Uk^;^eK#%ubGrAy;@6QnK!*D=#%4`q|Cc{yiamT)=m > z+&p@@;QI5?)uQLD=GBt---N4Wzbnk>)ygM=$JeW&G|Ls=DL|}vskfY8zilK)kxF)b > zc5soL$#BbQoyoSWx%r#pcI3O0pXB;|r#wCP<!)78&Fy~mw<EuU+Hu#mgO=G`sl)b_ > zn!DqU<4@mZ>^QCCs*2dPug@aI$I<`BUXk6OPZ?_8U(7fp-Cxds9KXL>jv{-w&T2}L > zXx~||NxD7wcKqq?q)PyEdVx>Yeth{i3G>i%71V{f#bL!fy!ru9y9eWIuwv7If8aAu > zgb;_b;;=xvOE|bb(Pro2Z(P+3d%1_P{<1=|qXfJ-R?R&W3$6_yoz(Rw5&ZFNqz>R- > z`l*vhv0rSDKbH40Q)@*y;uMx*rU_(zA>Uc-DHVK9?NbIbC3~jCt#1_5wsM)&#TtdL > zQ#EF&1o=+HnTIb@cT1=WMLvl)(|ATVe%&vor<-6O{)~PW{7O8{BhjO{i22{(E9u+G > zL|DKEGX^{;i(Q`>0or6DlZ2?!3nwSOv1O$p8hWjFmh$-*h+U{vO;c<tB{u-{Tu$<} > zZlqpHjs^#pHr<dxsNR>-a1L%mE_K5+&$PPY68`r?>Sni7$&jZeQHfVA-gv(qj@j4w > zgNTMeAhrnU_)@Ymu2E;bf0;ACIK`SPMqNGsWi69B6owns*p6}=yd!4Fo3P{ayc~_j > z65;GVQPGZk(h&L-R3Q6r2%}AO`!o-;=pcVbG!aTXosTP1uK09#BIG4y0hzhu-VxnP > ze=d>tOt?D=>&kt)75dq%cx4c!*Hg)v$YQWe#cTcHsW0EpiwgF@3%2wUzZehW^xGVi > z8HguyUr(1w-&g2_y`BzQN0vjS_O#N8-;_T7QvnHd)-U_6eLH#-4KLa`d3;OaU=UXd > z*RxRnxu;j3*;wVcH6cUTI3w3crRDa;-t715*@pSX8lOkcjV3`ekdTWUBGl~DV`x_# > z)4xz%P2M*&>d=0%v^u>6UYvUcj$sq0>gNr-ee)&8qw6FM5;%ulqC6(tUoLIR?j!73 > zMofkjXX<KP_+Soq+SM3#4NY*vPW$Fyt=Qw4JO;1dmcP&A43MFegDU)VDRGNyo`j_z > z6I?CIc$W5L-D*60k5J2qB^j^u@2me&>Q!HsQ<ybsV(s@x*f&e;te~=e=e7jnR;@~@ > zV;UJ1;5x6gyyh0FtP0o?cA;A}rD#dRKH3T)$hNYlZ|Np65Db5gUL~fv?jdY)j}-c~ > z%5hVa(RqDxYp`HCRDH~{M<Ub(^C0^(GfU0VWE+16k$l{L&64A_RSC+jBb3VMmBv3y > z64%&#OfvTqQhJj7#cY#R+;>p*lV{3{s#iIxM=faRxAuT6frU`r@Z-<xX<wyRLBNj= > zua`Aa3m?p+yFX&u-TA*|Z)i9))mK!zG`-5jbeqdoH!{`eeDmA)_#5OXspWskpVPTt > zs*LGn8ZiG*u+ml_Y5ILEqZ6q%URiqHq?j|n5fShkY3RU2(^^I?icnMFzY-cReW8l_ > zltf)?JpOy6ZuEtV-QB#!zuVdGmu2<|3;cSu>jUW=v@xfZ7V^!7q}7mhxoKV-ZaRiJ > z%+g^ur2X*ml41wTKvkK*z~RTbLZ*zkKOQb|oxb=CsJ2hd?HihImY*Nbf@jWx4ks6* > zcExP&9{HsGi$lg+JPLe$Z|J_TX|~z3JK^kkoh?zlt6|!{PAlMKdGTwHT3g+BFHj^8 > zGnpu(gdTIjST)X>>EdV9Gj;|hR0?QyRwp#Xpr*gKjERi>%UOw8-`_HrxWF|?j&q(4 > zdMP>fNX1fM{-bBp<*a9c=)SbcT)dIx+nMH?i<<ZM`xazUsGt0o^`Gw#?6sxlXKOB- > z8txBWlcW~^@?U-beShRVF137BbJe-hL1oMOq4W1htPM>kRY~w^24s>S!eY`FqW#-X > zO*hUDUSNI&j>zY6Dbb^pRcmC$L+<`f(q?IX6u@^B*)sM`o+o#1pX$`(mF2D8^m=bd > z&Cm6Om@R|$k@I55Fewd&TQOy+fdb1KzKo7V5S{orgYXfa%6*bvs`T|?ZG=Zrf=Z|V > z%JC|u6La0G{cxJqh2D|ty8S)=aM6YtF*p1~ba8l5RwQv~_RFv1WVzs?Mqst-xuiuu > z_oFgTSg4unx_iCN2btMm*}uW$5=y%&%6(}LE|ETN-XWPE%}V4Q8Lu4oRf$Ug%hYCr > zp+*bq&z?>cj*MVu#(LjTPTwLcvUi2%Ov0gzp3b`}_;j?Mnc^;a0}dgazTC~Olb4~D > z)J}{v&fGskO}3q0IXdod`A2!13ABW;mpkKM2b<A`SkUT`wr~>C^8MZp_0@BJZ}Wa* > z-8oIzH<1%uyZzzR#>Li(W2oO!z$mKO!0p7|EkMHU6K!;$L^Md`En1cMSdn;V+c)yc > zdKR<obSda{GeevRiP_&KtNBX?TlL52;74b;{k$Z}{Y$n&9eb1?-Syo0VbzJ<faJ`E > zq(mP7(4EB3Te(>N@$po^jkkZ5ky*T9Y%!s07vKs<ft)Xce$K>I>-elR`K2|w8tD1) > zXT;?ZCUDW%Y;4D65JZPxM=gV+7ikk!yln73dnPv6-FX>dRE=<+xgQN$62Em%E3!0* > z5GI**@)q_|r*)@z?(sp<!<-gwDFMgc73;Gn5%D48b|mBWAqz3^bI~E&-HKHnba46c > zST7@tDI=M22wuEOMm*%)G=X3B!=cy6iG0=`<&|>RPqf$Xu%fD*`jd?}iu0Ky@gQa^ > z=5iG-GUypDmO@+MQ1pY?J|l&tBJks&&$HHu-_*$x-YGUlPUDfB<gZc!BH^MCxQQ{` > z)L2R0AwcmeMfu92HPlY6!fePZGK(TW)DW!sG0~L9%c(FfQQs?BKS^iXR<|kLZ!X>U > zCSAYD_RV&B2$!9$YDN@MMvQSrKt)Cvk=;`WtZ>k$_u4nvC$JSo%x3WQ%b<tgw%?_@ > zYZu(NWR)axmOma9cNJ`g@QUg6@hvib%X>)zk90Mla2x=|6%ED0eMqk0%IUuGUwsq~ > zS7QXXGY2k%7}m26p2bf}WKTiNE_Lur;Gle8GZZ-fn!_jhCL?|>CzLKX60@5VXPmp~ > zo4Y=jv9g=7Taj+iWV35zc}Ad8S1Mp(ul_d8<q5SrSF>N2t-T}X^M~m0gn`s|6p`jb > zUx+jFN%z3|gAUXv54~~=zO)pM3I`5fvnsd+Q}h>3Bd0`y0u_|U=ym89u^es(l2Aj( > z3@-;I$EYs`pQ&y*vm6TFDH<ONhBDC=xKk90iMf=Je^DeZ4#)R+d7EWF==kV+G0A+f > z8xCB@=l#R1bMYl7V9W&}0EeoEKi^9Wf#%`9O=n_{0aJRNhPtg@CoE-@!oHWfawpz5 > zn7m~V$~HEuhr}6`x{j)rrBs9ke{{5SG#j|F4o(j&+YYK24BNLSTfyLG4K=wG4*G(3 > zVjeEa792D}@UyOdg!PJs&VI~h`&m{_?bO*4_*2Q}!YXrmJHY>SV6KFQKt+NJahV&U > z6zNylwD;0+*u&V{Zx1Z!qb7oPWh&4MUz}y*_7S-<D)AD|AN@|`<5{BhvRcEWnwU6- > zdd>so7fAZt5&WjW{N?jvaz|yNw}vs!h>xCvC|x%RPgl=kGiNhv5l*sUwxg4<c<0DF > z@aNC7y65zsoOI65U;Fe^*HjA<FTDElr8My5B1;5PP5!Q$;+@zO!a=`WR^R!#O8WD` > zFP@Hbbxsv^S%s0lXK7c3L8neOa}uu%XFu`ZeG<H@7iMS>m1+>fm^MiGH@wVhkk0z3 > zJ!dmnt}1bv$`)GuaHF7XYOFtPIu-3zSQ3;`9t(e)N%S%CVm;3Ul+c;)WrT;+rf^mA > zH%`PsT34E!OA3lKH8?Lcz}lKz|9jWnHF+{Le;^66jDl(-!!I0E`ApMst`hxigErEG > zOvHmp%j)(TiGF<vzl+G1l&FI<)TLB@IS}<sEis6@bWt-3gL$?p-4$(2r3FT|7T=N8 > z$Sc1_a3e<CAE`f2{gd|l%%D=ruwK#l-B#=Cm#yC!THF1>iEUabS8cFC&Z#}aGHibL > z>ymJls*C!tV_Nq-Z&OE;%7)C!KmHxlSsg8V+Gk3F6<co8m{J&tO8We}j;*+k1^>=H > z2I<Xm$^8r$>b~}SWL57SSASdE^+Mb2$QO`5xYl0=lhuXY-i6C6akT(uc*ku#FUFar > zi{)eL8lJu{*}XlNdmx!ht<k*`)6H0wOSRa&O4LJh(9NLHLx=9><n5sk=;2`O;X?QD > zGxk2??d7TJ0c-S%OZUDslRU1-?eoi{Ov~e4lmNKRlBNuMOn<zV{-M#{r?uFpgYKjK > z_#-wnvZVe8>}ixbdzaSl&fUBIZIXc<h5=jYfmwee&4>Y;_D;KB1NH|4?$RB~pZ%CU > zb=CU&F-l!V0pK^h{krI%pBM*&r3XXJ2A6;Tj7<4qh*^|=bRcVMXk#?n=_NJv-gGE6 > zU??qnD5HHSb8#peJ(SBhoG(3GXf|9NFkG5FT(03gW&i1Nc6gIu<iop`!<nv-`k#^6 > zBjN2M-xo*P(IcIVT<=PMfJ*wkG!mU*M!mL{?-qZ#9Q<-49d+j&okstfVH};69-T8A > z)!lGTf2-lW*nVa!d7%I1oF`|#Ibau7zQ^ctfF9eAANe6Y-mNiyX*PZxFy6K}h9(_9 > zVjRaXM)(yv+>=h=F-`38YR5Ag@wN9#n)M1EOgwrtDIq;c7C1?rGpPjYr5~MSV49@l > zn`AMce8xA$95}_AFv-C*#alfE)|{%x9pX8h+QS(4{iqzj$^I?T@%!b{Z|VErsfzVq > zr1+Z{#Rq<k&c~0w&Y9LonAYkT)#;el=KG_&G!4}hx=?<1pZe?m{h0jyA6cduE14M^ > z^BLO&y+>8U&K<+=mS$l0Gp<at?lQBU=CdCHXT5V~eL80SmSzL)XFXqy*vp_CG*OY} > zsOUgc>?5P&&w=|JC?cjgxXj#v(p-}HTt?trCg0rG(YY*_xm>3Ce3|(i&G|g@`SQSd > z1mAqc=zOKid=1k=t;|BT=E67gg_gjD?|cibqYG^=3!O}hKV%j=G#9(f7Y73uhxrzV > zMi)n17RQ*DCS;aIHJ8TCmr#LA^L$Hlqe}}Im!)N<<yD#GCC%j(^W~ku<vqUT-O=TJ > zm*pd-m6J#FSxkzN44l_FE4Ljh=%tm1`xStB6<c-{*J2g_(<))^Dsks3>GG=C`g4z+ > zRSMZPN{cn>PiwTfYxJFKjLU1x4{NN<>+DCM0o?TG{wvRN*SR~_d6(DuAJzq#H-u$3 > zL@hSN64}XX+M)Funn-1Rmp^ZqHsxhE<t#Q012+|2H`Nk1mB%)famLQxD}X<3ys_9a > z__Sq|yJgb3WwyL!@vtSsu8pahc=CAL{?oQ&?zVI1Hf(v@^<mrn@z&QR1^vf=y+8f+ > z$^Gls`4{KG!}j{`C+3}B_1{CacETR-L}4s;qFs05J9iRmcHp@?Da$*_TDzYgcG4|& > zGhBDGJ9l$ycJp(03zv5bwDyV~cFPiZS<cMxsyXfW1><``GR*t37W>Fg`}d*ywYmH4 > zo%@}xg5Mt>bnqW^jcwF_I%twT7|1>7m)##eIv7nn=<YoD{rIr&)8X{P!BoxRuf)Td > z+{4M`!_}k11?Hna7Dt=>M=PC2JF-VdV@G?-M<<_-PA!hlKOHY79&hI!qdSiua*u8w > zpWGj9;&yG~uWT+Z%&nUJ4Ol*Tj5#4^Ii-+0rL;Vy8`~p1+#6nC113(HF{i97XY6uk > zAj>n(pfhmZ8F$wi@5&iJ=1h>~pD;%5pQz<Ov7mnvdH-H^{gYn#CyV(f&vLFPcdl%C > zt{QX>$vc1Db*`~;u7x?*VY$%OUZ+$3JATgQP`X#qc~QE2QGRq$@^E3pa%nq$@z(9q > zj_lI0>(Y7U(&6~h33KT=e(K3`<vo7sm2~xy?8<NDDgblkOLpzgavd&r9hr3X!R<7* > z>)Nu1&7o<p7=N6ejNQRN5DRm3mV0w4dy>(0gIT^wmpjfDxXshP#kRc7WVx-7yRA&R > z%?-L`KD(~M+~&*OA++yGE$?cA?%Lh%I=k+=$nL%ipj$1`)!OJY{<|Mt=)t7BpK|C4 > zH}sg?eaAR@5`&(yydV8`Ka0`6AFn}Q>0RuHt*?$>ZwEan>pkprJsf;{wzu+d!h#XR > zexMy6|D1GjmWR1LW<T!20H@lk@}YQ-n3Xeh))XTjG0BG0>aHutQt?^zWhAo!2~0Ad > zaOw0mAt@Z1xyoPlIFu7{4)5z7c(ygZ3I=hg>D88ue31!!z@>k)qf@97cO;tmW>>FN > zGe`D?4;ezCn65;tKNGrdP-Cud(0KJNr@GG0?x7)_HDoNP>+$y&gw6S8p&tn5siK<? > zO*=jA#_mnUPnNoUNsceFXlre{<B6oiNa;>(h7)+?IvFMZ*^cB0`OlDM{&ScrP)`!e > zp}%nMs3vRk{`KT?d#(`kF0ZrtrTG4A2gfn~QC5@tLZ1i2Y(n;x+gA5iIhMyAJ_lQq > zhT6@E=64>uiw*uOx$SM&$D6%|a>wn`ekX^cZ4MX<S^qg4yXCT*GrN>V+)6(@L3u0! > zP_29#sj{E~9*wPF1%~^qOo8ZmacvKFH^oSr!Qj+M74nv0M}g!Os80DQ&SJ$E?>Dv{ > zDm13yrz&)Bi#=2s-u0cTGQHpQc*Ww6d-jSg4CD!U7N>j$;rMLpsrEc8Tu;p~w^(@C > z0QFKxov(TG3><piOD31)-czqGGOjG5881Oh0*J4Ki)cyyEpE_~I_?wEmbuz&(3ZpC > zis~p3a3FP*$W%lzx{^3L$VuED3P@ix>-}GD<aD3tZ(Ob|3Z(`yMxH9Nk!<ab^GqeL > z_G^O|RGkLunqD1-#t9d1pbq_w28K@&-*hd5@WhO)qd1z3-X^Jt%_%&pd@a+OyL4e- > zZvCan#JH^FkBRBGKVl}X&qo?fovXtP*c`J;xTZ5NM~37yS0coh`9A73S@<9Ki(3X> > zZ8fjRE&YKY2)8&|tWqRKDXe3q58bOCsX*i2ggq~9v8md*5PzE>zTNURNluG;HGa6U > z#rCuICqgd4-{x1P-b%`n_Fo;CW1dyqDz9p*25GKWX;MUsgcvl@e)uaM!AWmhl5Rh* > zSrn+^H=&o}{uJZ*wx~%2ORvgXKVJ+(M|%*`!x;_BePC1iTt_2pn%_j?$52>HRdZg@ > z;xusX8n@i^Hy2bR4Th;ds&^oLP0;LLSi3UAz-Rhk?-;AjcYixV%Ct>xL;mbPv(-iJ > ziu&9`<v9D?U4~5NViGElbMae~rOHquq~)`>xC;UBaS=;CKd0<;@qTo5moX}AC4@E1 > zSvD%^sqfbA=F6+nvNC2PR*>TswSP)(hx0vdeZ<W|nIl0dk^OU}81dPgE#{#4n9ea_ > zu_w&_Ut7}u>1O<(k^6Wy`fBAL>Tjnv*vdhVl^|PLl{sZAg2cqp6z8g*p?60GR!Q+l > z5#mk4rM!djs_C5}Fcj~Z8I5AyjZ<9A?~h=;?qNCnjZFcPCk`q<VSz;}VyFCw4e7Qe > zxfmp&@=OP~P>OD@lRaRq@?ens9%qjdK#o}><^ahP6+;F!J-1824~0~%N&v+md3VsF > z4RMiA0S^VM3QN6iv^;C!6Ff!yp@dS|)wgpR1IX4O2(eobt}W}5mx#>0tFGHT9fdO% > zpoE{0{K{W{jP?&?tuB-}wd(gVsP(yq!T9IAH*YA-*zf?Oy3cOxI1ii1h(j!SEdA(} > zvEU!Ee1fr5#g4_3$l4dnSn|Y0yd@M`yQObthzP4dnb=L_UE*vZX-&KllO_toHQg`| > zB8<YKLQB3_s183{0uRGNrz9NK!{`1N6wSlQ%B^5fPH8KTJ-4o^!XfWWk|iHUX;c6g > zX3eJ((k<M=^(5n8eebW=vZ`>j7E4J{Omz}qI?sMCVYs-#eCQ6D$6dz{SKV=~Sq%oG > z*NE|Ii&&$0z$Q(>I^SHd;@(I)MautS8&#{y5$|{U<%bs@<P5=fEiB;)(EYJcm%?k% > zW+4@zOUB=#R$&)U^yeWBR~WTHEnoQ)o0f`59G8{&z)D4?bT#qkQVi2TB|X={lc&zA > zPbzBi=Y`&4cfE9Sscy=*$2G7`dSvuPcTgW8=h{uQ7BW>E)%cXFrMAJk=QPeMwh)7x > z8wnVoY>C~tmkrEAo_l8>YZf^=AJJC^|D58AosPG)v-C6;x1$D<Yr`g6ZaI_hZHS8p > z3n-i?sZC)aa(XBI7ADEPpotS2GxA-5*KBzA*J6NP`A-tenrS|p)l*ltezy>mEEn3l > z`BRA6#~2iGH+MngA|H!I#{#uK#L6Esu_9C6MnpE8E@w(U$cHd`eLcljhV3~bu!<0D > zbFZaRb6?|2nMk7H7k6!D<(q?5Dgp3k0uBLd?j@DoT<~Uo(!+wlV)E66c^wg-NA7g2 > zgXAKKg&hL-5tLxqT9Xwjj*GN5(t3{~BU~-%pnf+fLVD9@GLAAE$iz*>%n}=-Q%S3V > zdFp&pfD1>H_^1VjT?!vm@ziwe(3A&gp=3DN3M7;94wM6{iA1E;sUI@8Y_>M-#Wg?c > z>P2$`13Ix_Z91*U;`ds`g7~J8d`e9(u8HAW2tWJGWMWek#qdO|KrRJVv5sAK%4*Up > zPnWoA!b9HXgI|1Co?4Jvf}#+eBv7igicRP55|Kw`oZc@}ytFdQaaMnsoINHaWOaHW > zDPH>se8xP4!$M6otY^r&_PUF3N9<lVmIydyZqgB>VSiE}tX!Z+A-F3yJW+%5-jrgs > zw_E|gbTSC8!fHlX9{Shk$h+*0Q%}&bq$PMFtg|ib%x+iCWX%G@&JYb7t;t##l!SA= > zfnrZ~L}3@24){>G#u7^`_7njvrtb1jLze(qDo?UdvYNRUywQ{oH!j}s>_k<f>GH)o > zBxGq=M7q(TCN#1WLG7V9Sra`Y?UzXje`hyXX?#mu*U7?#NZ%JIs`pVFDI(%(_t^Tr > zx75^lReqG&FMn1r+nUHX)`|yY{n)2o`d}c=wuVUV$l(hfNSsimAoPo9%rc4Y0@ITN > zDLc-Td#6`oxWT07&s4UDfwBf{`o?6(u&lv4Eb%`ZWq`b*@f|_1(|0cHFoFPJn;=*> > z{|jU8;c^iaGvtRVh|O~Ex7<&?(IU>(b&jY#mq?n&ip8j4HHYOMvm29%w|7>4EbPdE > zkpYW1qu!0Hfp-Bc&)W@tAGkJDd+3{!wOUDG31s)sf__CPt0%Lf287zX-4+Q+Ktzkr > zBd}kyk<&cbd{u)7Q)Jx5Rb$mW(980RuFFl)XI)Na0zA)_{0fk@+<4(JubvQc)3*`f > zwpCZk*R{!$|0CUs4aGmeI}zdLk_RwT-{?(B!NR0m;3yJ?!mq!xC~#>WwUf}HBXF;| > zAKsPoV%J$lllxyk7wHlB-GawGQGg3+rX&?#a8}4v?>=KA{m@G0HA$9ER3F;V!;_s( > zt`N-Sj>p0)VgP=D3l?QPAuw%;@UD&GG<{*M2*NDye$~PW&0Wg`0BrvgDZXeKP&onX > zM7Se6o-mjLtsZ)m!l8~WT4+sar4QuVfP_VrL>VQd3q)k3Ipsh&BG0&FNJUglJG9P0 > z!bQEfwGk_COU36(v|^tdsRvtau)oYIu`3id5PhMl9;#xKifhy;yiccB^;sYJ`3<Rx > z@p+k9Q3?3zfPod?#~8SOMsV;g!J9h*OL!<B3fl*fN@PV)?;F7n2_<xIYRedKjxURd > zE#t&tCsYb0Bm|g3DfJ{+zA!|8C1%irW3vYTZ2C`#R;usr2=p?r{rk%OuOnW*k@yI2 > z3BkfPW##&Z6M9T9f$35FRvQs3Qbwp-hWXD8Xo6#Ntt!TnmPP!E`}Hi^A5<RfjF%)$ > z=%q(1l|N|5i{px*#9J+BV(s;eg`4^eg%iiUQwk0D!Q+p9?x&mqMm-M{8S?lQ7qBRl > z>+>=*_NDAuQFQ8Xg&BFg?^ER4)PY`Uys$Qeahr}v=6GCgczm8;2B1pRF|Gt?8nyR| > z@E|jD#{55rP61}?Oe60=5AdhC!#oorVSp9h?HWQ4j5B@K*IUJ0tugpFL_pFy%J_X| > zTlk2|2$Q%KAK+dcF8T{y5w&AYFg%h|>C9>G_e&$?S6871YpupFMV>$H8ugS-vOS$# > zt85t)N(+clCS+Y3@Y)Y7>ko8c7W?XSQ^NUb#X0ivsOIV2Z}EhK<vl(Nr}?EF_{CKE > z#e}qQXNJXqLj@00r66^99EAJ`QgW=3ZlI0eHCLF|v^uU8L5pMUV{8GA^3fn4t*>E1 > z3~dBlgyP{70yMX^jJR0``vqJ^5j5Wk?yUt(GvX-XNM5ijFnm_4PYy*>{_t<<?%4TJ > zq8!IOQY7@`<s4}I35PCj-B(%qFi?xgi_8)|BV~iZ23<!yJzFJdG#8%|caY9xQF5C^ > z1)dgRun8=J^--QIDWKsmBB=nd(nefli=c_HO|9_we4grMOv+Q{W>}S~9pJH|!iC8J > zJz5+9=v7nvB~rpHjqboaS3)$>tJiU>NKA&BO0W<)3QKsX7{EmgZe^q@@Wg&Gy}-lP > zTWt{c&{sBjVRWm)*Ag|N*P3@LYO|<s5U(W1@G_B>_A#o-bfW3q8+OyxV2=+tGBA?l > z&*0FP;`Zn_`SEam$oG)C)=!RcAN=bH57crQULvuwV;D0ozrQkX{L@4`7-Fo-4`B<P > zZwi`t)4dM(DUo3F>Q<3w$4sl??Wn0I6HBGCn*xQL``RO|QMyZICu^yNJVo%cs0ell > zEHgMzxHXNph{%V_^f1aF9A=yFk`)4e+0U*kH0GTJIH)kfZ(B@hp*HtX{s$7tdWNrk > zvok!uN1l>bgwbJ0!<mTNQ6%W^@mogZJw{1?P)U#45;SuxbS3MZ2_SY+!sOfr<@BxJ > z^G><)4E#Q`$8rNZU$eV&a_a~T;Gu)Rc`TjPZ|gpHopwt&;i*M`_AZZDn#2Bh!X{yy > zEU9K9o*e8g5AXqf!mPY%jx43`!u~4!t~;a+o`&xjC$BnvlfE&`41C&ZD@)cPEmbkG > zlpI=Tg@<&BSXDD_>Y!+@7Af(1ss}ffz~Y-npy)}y0~xsZR3uEETtr6V5BZ3uA4ogO > z%%6dhdeOn@?h%5aW)L-ICU%?<$lS&2_}@$d%P{s99rG_?w9o0IZ#)-w*U6C2qN}^* > zu%FE(QyF7O4vbb*qVqzYJ-D-T{1EE#VFb1sSX&n69C7pzX1(zG5LOeWvs&IB(Z1`i > z@&RL6Kz0aiv&0_T{X%V1Fk?%w@WX2Od<69!k9l)OL9(>sE!Xl_7)koem3{v-=`(m% > z@USD*D%LWw*lv;JWTS;Jy-5Gl#-h*G-=7kyQk!K@nxd8J`>o(H`mcy6jl+uNq|%H$ > zpMH5On%qi2tW)SoTxMp@5MeYB^raQ?GAkn`I*S*_0ZQo2ZuL|st8I4;K4JYnIov?U > z`n@7yLjl8gu5~KeXpk1z*`FcuYekgrF!$mb&%k<K?XW}$ws$^2G`1?en^PPUx<|4h > zqWTu2G@1EmEWJXHHKSD!{e_Ukk0fxowzA}HoO|5QiY4C5jmT&r{F3}ATHB%E=#w*n > zV)DTZ)=D&>%k!`SNwZBtt<M!i=)K*r!phVbc&hdARKQ;(zg0xfJ;J~WuTFF0%^0or > z<x_*2N;0h}zujU51SK(T6kRE0r;{~(XR7zYV0tguJvQ1dJGjeuAZLFoHlH#&do$O_ > z-h;lC?vjvW*)HTYnA4un+}=96Hg&+)p1SjEcv~vhST=X=%Q(EbRlMa7_6}=MhY#L= > zy)FLwl>YVYeEj5#1JM+zPm{x^%fHKq4#5w9SD1F-)$)&0Mx{F)b|(Ht>o`7hbsUM_ > z!30q6#Bw^uai;$s%<lqk@|r}?`RpVOkfyLXC2Tvw?5W*PSGYi*E8cCVNA5<%?0zlX > z#p(VN*}@s29{ZMhJCk`bx6|J7fpCD%-Xf-=w0K?D!zs_+IWO^bGU0A%+FtBjOiW^H > zNwjlP?jF|IUX?xJH)^NHfidq!lG2=<B$oG(pLWX(_M4c6yM^{#o!8AH_p_R|pQ)DR > zCvInrIX%-NbT(+lV%y`X+V`m6Z^e)4@wV=uc8J)2mp``coVFdIWD~&r(lhd9iPmD= > zHZ1IuL;S<R*khORQpZVtmnpFXslxckqn(6hhu}2F3E9KR(!;r?LqGnV`9z1}Ra^{L > z{`fx?629vhltBV%eUxrA?iw@0*1$f|l+*gm-p(h-5alHDkvM-V*Qk7wq~ZNKr>uYL > z@&Bp|=&^N<waQOt_7^)3Bk*bF>{&lsFdX4CBq_GuFzf8AK3et8x@{tTI4ba;2)`0_ > zJ6}G)%c%q(=sX15Hru;XBn)@Do)FU1V^||hAD4Lu$_?3D6V9F(T(>Y&X6X)|{Ox>K > zw|4Z7!hX;~B9tDME|}dye){7-XA}FS8u)d{4^alPEKO7>#0u{@_Dh!WCBE9z%GwAp > zJQNHmRPPpLS@B>Ww|JU<!Z_P{sOt%GD}SbaMtuLyf7OG9ERN90ljd%p4`PiW-)AFb > zAzXZYMw|D&R(>z$krh$dKMK43mp}hyBb*u2H|`&-=~yQIrX3HUa1G#p`YYVp9X%*m > zjuiwLT33(N3IvkokLqvSTVl@d(>%jiBbMqT)F4`Q<YAw#@w6m#9@<j;GjKVPQ^MF6 > zgiaB<0vC(QMWJ9kT~;ssByuhJB2yGDXVrTF`imfN1P&jeUjD^X?Pw~lB25V|!yjG` > zMFrvTa4MXaNB$oe<-GLDy^sbkRWIvaS?ozazakR2dN0s|Xd%!uin8F3a5fy4v?1_A > z8{WJYH(f2gc@k|P|LoPuRYH6?FFc%(4TozYGys{J)QfzKB+y$uEwJXtpq^#~l@pxa > zMd?n2Lfroa>0I*-Dws@NKM^NnPRG^zQQngro*?Z*wtfnZ_V8y7gFrm_;Nz}R*D)gI > z{N>)f^yg;FA0s)!V7x^d@~t_WR~AbjMPG^)bm0}|u_y19=I4P6)4#>^ex%CV4ROLN > z?}|{rq8HAPFl_noc>GVbwxMH}T1I>6;ZV3=@Qv+Z6X>IV2sq57%P&G|a%3C+6;bit > z`#4z4W%0rPf*SpUAYfWHU}=DLL<@bvi5?IL=nFy*v7m>=(PlSfT08HV8S$_%1X?;r > zdPYKOR*D~u>VhUhy-x@<ci!63L|dl2HSYxUPGD<ZKX0xjn97^N_@P35yX^GHae`Jq > zB2Q}dYJH}g1G^~*CkYC@QP=~m5qjkp8MGPNNu(^4@5|kqL(*&GKK2B;NAU_)_(z{* > zN5&~8k*l5H)t*K1S{2O_T>i^byKul`o(ipXjF?}+_LOf`0|+gvjV^xc9j}0b9v+r{ > zPaUsxS&~0jOoh~~tHjgsS!_ZkH<aP@Qugsidc{Kt&kYeUwW+OFX#!kgwtwBsDkC^w > zF~y@^$CV0gQYQW~($=qzcyXU#Q_`?m3e_B<>ywCGQr4@a6brm1lYiyk)Yrryb(!8_ > zsg`1fYkJ1F*y8zjs`}j(|8l$E)!E_xmB8wcP=crYu<H|LOmj4ii0i?%(B@#ubNw0^ > zUEzAwemnp4^uJt#l1%i{!qo2>$LPZ*$KjiCW-cyMkjtEMGO58gjA&z+&v71Tw##s# > z@`c}19`B-uWI7qdpW}!l6E*gGffW5n(q^j&33_DOTo!EQJyk_+N6hY$i<-8sU!{if > zeq-J>!+Pns?SO9tUsjU~Wwke4=CH0lMlRW?dBT~b^=dr$TjQ%?FcH6PoaRM*y;(?Y > zj18M3)%V_r(C_Bn?6HaWP1c~B4g+0h@+F(kG7LRitJJ8H=5;V1h0l}3^XRp&p_Hek > zPw8|WpTsy=*j?HfSEDGljJ|Qe<xRPqTRGCr{cboiFxLE+#Tj;)zMPp(h*o=Z+mw_g > z6+_KFB86{~xMq4?=<{h#lMF(tY{nrvT_v)Iw<DZgNc}bNCM#l%Hf|<t2Am03jLjp5 > zzL@0G<E_ARk@>Qz%@gRFLaAJSv!0~#!c`H!^yQpLX3X0USK}2yrKu@0CKDpbpo`l1 > z3H^>(;^ev?+jA32Y#dTU^`8n=hAR*I205EdTvL7vQ0^5wL_DR+sxVuMRDWK%-07vQ > zSQMXppd9wlJH$=a*uZ5^S>lysY;zNrW&+bywr|2?2oUTc;z3h)lSrWj`<|$i&VA0W > z_SE$&(^4L=C#Pz~lc5>#so<RBD#;BjM!n&!!8Ep9se9cv&Bng3q8xn8<5t>~r+7K| > zS4g+(OVe2#d0jjC)bx`-t8%ZCg{S43DT&soAw}`i18Wu^EGIGXkY~y*F;!3ftu~_n > z1lq9NzW*&ut%lDdJm8b(@m(vR*=&~z?EK5R*U4)-C3>N8vf{tU3Qdn9D8^r#D5phk > zSMxs0;pj_2qeEw4#HpN~o^jf;p?wU+u5drDKBTlP(NDK>{KM^jO|NX*L1Xi$yBg$% > z#moKHg`>M81i^~baUGeP^vMv-x3;q$aqaehBN+bsQ)HOR{lCGtU4iFIN|=tz|5B#1 > z{}w;XbzE<iB|Y5yX^@je?=|OjpijH<zP-W}!za(1F@te6Ia9qvxvU7;M->ExFd-T( > zWFe?{Cqb>)3hb|;$Vz3>FR@*M=grEkKN!PU>K*<q=sJ|j1d=32Ntj+Q;i%u0p{N&q > zrceqevF~Aj-Hx(O_HR^TtMOI;X3&I01f0Cc+<B~PAEAu@M9B5OD^G^9KE{1%pH&6< > ziqav1`>yp{+<6-dZHJ>Q|EDJ(pU%xF?kVt_O|Ekl>#T_(FG;j7dfDB=2_+Gcl?NZn > zIZ{p{CRKf?q-I7~pnZcE*>=<un!7KYyQhAfa1N>I)u*JwGB`vkhBQ6v)o~Ya-?gu^ > zN4Bd;3AIIVB5R4J!xQ9k8WR2hG-4P+3*3Ysv5U&{-7)K^Qm!jU70|72J}}2lfhhi} > zMR}ed0k;t|b!44Aw<FVet!PKd#i_%eN(7x5b!-&0kAL~^Yf$a8S84Rwf5;s)-o93N > zzA+IF%BE=LlV8{W!<8WPlq~{m6vcS5xltwGPpn(Lkq2pr0LfGJHx1Q)_UUDR;j1Tl > zv;&Gj#{8{oapwyk!;hU;+ww_#(IV?T!1MJjge!HLy-IpVZTBXdubA>Bgma)ktb<GI > zy)&)G9<ogK-bvdxmL@eny4+oe+v-5<O<R1juUj{>PLWCNuSFcbCihoeV<~BkwrU8A > zdu}5RuIbv8#wvTNr=01MEj@OaO)ggzI@5(2gAd*bc9U3BGczWJcd4Mvfa(!b5*qau > zX-Kulqbjqf`-Y?a{k5bKB(sbk4I=Ji-bMO7v0<td=%|mci|IJ9z9FU_@pgSJ^x{xM > z|Ejn&vM#>Hu9|k1X5ps$va!CA&&ADjVQH_ax%Hlp&Ud7C^){{LM>UJ5DUaEPuy|{O > zOO0pTzS*MURqJo_Bd@k*lUX(|PR1M853jH-(&eF?OF1Jx9SbI9k<H(S{1G1(vX(Z6 > zn>%N=j_$r^E&p3@_R}IBVZXx@tWou>$Cnih)fJt=6=~^yyd9^50j=rI+0}aJI$L1v > zrxVdc_WD$h*tjXL5vWu2VnYACvz2C?4RE0@6>g-^G}c(B``t3&X<VD|zG`D=J%b;8 > zsP3mLZLWoX({%D;BeD1V?N8oTpRaR<2AD=@NJsfkwNBx*u*@yK_j8q0ckgVT#G$Uv > zslI)|dv7brLqiqzmD70)60JaD{9;7!SB$xk*{<jIM2~Tq=R=%ra^o6tnB+uk;QNf( > zct_!Xl9LH(b@?(MO(wd1KAd(4lvYStUfIlcW?p_M6JgrZj=P;Imc1xfNP*m@W&SA_ > z^Fmnjy)*ooIn%IxUM*g|f9opFfA?YtaW=K-dGU?C-{9Lf&9Z6BxaO|oKkB|ohSK(3 > z^D{M(=5S3>Sd7LU-@H9W)IUqpqV(ZtaQx`V`95M^)%C?d*1mB2MxZ{=NZazIDk@a- > z7r8JlL=Yx>YQ|D@ob@;$CH>E%9yyJ*Q0+cyHAkz+<9I`V_ceu&c-N{r;L}#rVfEwT > z=-wRK6Um<b67tM*Cw<dZ|AzK0vZCgZxBTz5(@eb<w?k*(W*Lr8RoQc}H-EK(l1(63 > zfSUWTz>zLK2WZW-39d*^Ay3SRuq9Q4$Or3q#^DC+&}7R8_csIGH2?ivA^OiEGxC}a > zf>}#)8v9X9t$(gh4<vQ|7(q$l{^7zz1`EnCGj%g`n6g-SSmmsOZDEjKr(ba3pxvfC > zEEtiX{|8|N{b+(IUHp^(zVn||XzO1kf!iEz#yT^p!0GSgV=B$97r82dW=f3fUPO|} > zspQoGrfPqw4c~K`Iy?&c>$oFyA*HS3?Z)d<dZI<=28^Q;C1~1BctsHlVl;>wyqep# > z7ml+z7fMO&l4lT(F&%J-(^_PeP+-|xd$OD0x`-3FYuI-^OYrgP)mxHn{cGlB2SS>Y > z&u;EmT%gP$wans*Y_)Ux8e>x&+0~jxdfZTx8r^&xRjWqLd|aD+T_IKigH?Q!e0=*= > z0*idYv{k|x6XH--B1GC``vh3Id#>Phh#Av-g*lIzIRzZ6(p{l6H57Melp19<uh2)% > zDG#v97Ybsc9`VOfqFDQGw5DX%=C=Cyz*w92VU$RvDJHgg6>@)E`E|9vCpA;bscfv~ > z*4RMsGDq&%O-6EEK5-!{)3BC_M#xh2AAy$8)sis*cLnC#L_ToHJ7gF`N*|RfK(MvM > zw8sR%ahf$1A}B3!c^4?rIw~gxH@nGd=pJfZ_9hivHetx3M;5330?AwsCA^o;eCU}$ > zIEY{QbzmH)z4CXwj8wc|+c(Stm+KkV?1K0kR>2W^8dwxolcy!GvDgzi1;cr8^AvfD > zRPU9zGyvj8MXXz#YXxN<<vbaPyc^j3TIPxAA`V^r+h?tFiP>)v;m^i|!h5k;Uw&6o > z0bxQY;n*gF&rDz&a_$P(RFLa?RUQb?MOgF}+>cw^LkkXZa{O7qie<6XOKJPol1pBu > zw#U=V(vC}BCmx#)2vK@aqLg^D&il&g2ZcFbAD_{NN&kj&YKRPH$g3qKMH6!+*G-kg > z9)U$m#L1=^n^v9<ol5*Bkhw{zv7!~4Hq>hL1r@eam93y!@@5KeN3#Tan`>G-Ov$<= > z!x~&3jmKriLh9Ggl8D6vlGk<*=L=w})aeH4VNG!XeK$BJ-Xs!!eVh<^3a~r{OhFmm > zV}3TyOq{#srH*|*(Ja9sOFzm7!rGj}LA~JIoa>=|2;&iD>NVfe;gYA^fy+%{3V~9s > z$Jk7390~%CivEl?s*qAjWqFx5rOGfiUTlB@7oZ;QXHBvHO}ed*45RWRu5{<5@l=#5 > zF?^eA?>x-pb~{hW8cdpKj!R{wtbye+U%IYL&i5zP4`iO|^o5_;1nOMsgKzGWSmxWr > zC?t$a5I&;C+{dVfMFa1?sJ8tV@rXiMo?==N0ttZ){#9)07V7N5Jy*S`v!;Lm6fF)g > zAQ<L1Gt#FDh?bj7_x{&XlL35kM=bPB=p|JmpOLK!pupD>8WCCk$vY7{Alo6w!Gg6| > zQ@KxQm@s%RvbTb~1?o7cjQi#0j|^D;+lhXZK_6HVmj-uGBy&)|1=CAOYT!#)Vur=y > zqye1vJ6fvACn6#O&Fmp*xCCaFFtfD(6r23=nRfl~qsL=dnFiPfCZN$ZqE08m0Vk5- > zima&$#^nm4d8gb@lZfio+?m~MeD3_m+_|?E1v3S?%+80lTSkQ{<^$^jExoh@>#y9g > zu#sUri`xP}I4WBdu|FVi^S4o4-P9rB)}d(?DcIOM;hVZYSE0CE0nSst94c=tbLC5A > zP~4TP&IqgRl&_{OT+W5;h<v}S=UV1dOzWOv*>{0`GUO2k>`XDA76iZ{0-Jj>;7aPA > z6(dDuTxGVklJa>w8<tPu#mj4;hp>A7X;)XfGU_tz)OV&!m{8Nq-3O~a?w-P#5B4gH > zN|b}<S=<!HyeZGjm-y?!J&6{+oV>mE#(v8<RE%TP$}E}&kdR-`gJ*hT1;h6uH08B` > zC|LJ>$J;e96Xg!BXfhUVtF0PJiK~iN+rW`h*_y7Z>c@et3f8L3>02OWpie9tdsDUd > zC_>MS(<xQqd?=dfpuD#=bfBUb8&H^_JrsZKFcP}shTE+o`c4iX!owPpK#`l;G#c2q > z;dYP^E?Q(a0G>j;!Vb2b6w8s(_#u4BZ9n;k$Nk4g>5Uo1MLzV({Ji4aORi!CqT!AV > z{qGAskNGy{QR`y-o3a@;?QrVBd`|FxQLh}_K2@j?=G8KjIa%b03?$^N5yvvcmE)*j > zgQJ`1#pvm7&e?7ZE?5DZd|meu=y0CDxz0|7wD>Wxr6D#<d75>m@im<dzj8S@MXKZG > zx(b6?4#?_=x}6W(Y{`HwXQLyY;tjiM`_Z|JG6i7XL@cmtpggHy-I}WS$Rp&L$+rWG > zB?`s9f3g-3m>b5&O+~|ve4^CZ8<!c8L)mxSG`9U`&h^2B?~VO3RAnw$a(6>a0G9!U > z^GXo!^B+f*QT!IBwau^ATN|;3@tDaJ$`h-|9K&cyEjrPnY!1%MEjlMBtsJQPk1xyw > z`+v;M7t^K$?>w}WeKKs-6!No;pi3vwb-A`LFsaPmGY(Y!`p6#KK0zbOnW64f?APH0 > zJx+BGP6ZS^1<rLeB=4xhJZL`at|bc5xCv2bdC(V~*4CWT<ef6SGGy<VkdIl=Yt(AA > ze<cMjWLk-+Xkt0*WE($a^phm3{a#1g;BiT5Tz{d(M&dbIf%}Vctwx7|P06%%`mxC> > zv)bbNk8So4r!vwc&qPf@uDp6JF8uZO^4vzxgSk*@=>ilTzM|)$!VEKa*-#giQ0Bp0 > z-yw9Zb9zl|OF!0=pG73}tFW*@1Bd1j=$@NVScFrF^cfDmXqR>E<5}<6{w<=6?T|9j > zs0O?^5lMgJ*+JvXEtQunCdXSR1pa9Sl9&~Wt2xpny29#)gD-PMYo6Uy+5KBzvo8Ai > zSq2VnR1{|4KY2%8Fj_0kB5DB^Rcvl(u3LWG@JkQD<y_~aS>eI(@dItI2m8;uSMSeX > zJrjnQYO5t7**}Y_e?zLreR%!zgL;ux-V9kIlP3#9y|3f~tsR4y!&!^3E?7~yR>o9I > zT(}-8tn*5Q?ii`_lkJ+cKA9<tL!j|f$#}i7iAp)cvkTQWCYWT@3oWh&qx6e6YiCxJ > z3-V9#DR?~WJ738EePrX4p`<P<SJY_1D)f_!{&3gSe81oFt&#LXf8xoR{#Ry;1ut{7 > z$k#{Sf_aSC6b<W{FYKf!jSa=@vtmS~Fv6C!e`=r1JYyhk=CZf)@pkxZYL2!ZQ)9C! > zw+3MSuOv8l*i~Ix#j?>VyJ#Vkr+-@Ly<VSR8a)32iwmy_NT=&`7Ik>RJaTz>;2fCR > z_t}^kC#XTlp~-7y)XuWm+l`2GO=NCaq;T$^6pqNi>M~I!!BH~D45d({$0+p<$8+yQ > zTtFU|wbmk;1%SEo1@7hFE57e*jBG6TPTkD852c&Odz>Wxz#(S&B@%yJ(1U_M!X-3m > z``fa$4G-<x!1Osq+v`AzrwsMhT^)a-5H1YB4}Elxp9ld1wnA7eiW=@d9dOh}d3qjN > zlL=+<O%0ij#MNU}i&GCN7yTUDgwdod(lFMkQ#JK~h~NAl07pQ$ze$@kl&dsM&$N?A > zxlHTy6-#-QYx$FNxs_9SmtXmm-)WXBH52Qh?5G{-eA`jCUrcy}POvO+uN@Oxgj@ZB > zDMUdXaY5DWAp)>e6vDZm>ySMJI{Affalc?k+_^?giXXDMnZwU_QR@8!mO|J$a#!<i > zD*Ehv3#GS&J+uKAG~G}~1c^GPR*Hc?bc(d{aiJfTr=pGUsjo=QtNm81S0eXr7UpN} > zjw?tDM3jP4bc&3>`4p%+s~?+;+`1Bgovyd#B=#&UBApAyHZFQ+C>QQ_Z+pUIH@AQL > zWGIF#W*^p_q4m{xX|Tc;xOm)vhPvnfcLR~16es{-lKL%7fdpjeL71Zn!Mlh_LB4B2 > z1KC5~1ppgDWYplhzAxXEnBDzNffxdS6Ukz}C%}V07UF1u$1+3;lLp=apa!?KWs>Yj > zNkA!>cSI@x1eMCZy*m(dY-+&dcBs3H4?NphUw!CqdXQg%G!q+U=f<sP&`j^!{?Hbv > zdk`o=yUQg+XsJ1Byg80zNw@=l(mQ^VhRSne(#{cIYAC0yr-;d@gi{~gx?ztBOL++3 > zI~MdpxWd}Mh|;NTTq^}<j_3rC`WA?xhz?+CxIxmIS=4ZV$d&AWC!9SfK**$h|8b@* > zxWW|Rz22jW|D{GPGDKrO)?+jOr2=X@xOcvqc)RC^K4MI6DI@@)7oaCf%M=vn6m+K7 > z3EmVyfDR=|3I%{G#A-e0cRL{67N|ZFc%~HGhoJjlr=IAcD8Vm`t8cQ(7W8@hQQbtm > zd_<H!q2KFCWNTSCCbCE!9kU7{=H(WIZAU1*CP+Z|t6keQW>EvlFDO8ske^omzJCiI > zAd*TuX{GCP0_<nOq)Y)oXz|B|)2}U3{E^Z0Yf2P$zqm;G_v>FpixwG<@<;Ka6jJ{- > zPC@dJOO%ZYYHf67lA?lEwv=$ZXwg;xH>48OlS1wn%8jj9auN8eoh^)0G|rq!@t3P4 > z{*+#%xP_Fhr~hs#@wd_cR)mwRG)dTEH0neptu~3Zw~<qXC=xi8t#~WRm`!9jC9ucz > zpT~*}$M~}~<)d7zN_;`Zc&}BGT5{=b{betSy|P*D64cAr??x&bZy{Cu>z)G3GXY%O > zYtNQDTdZME{*@$&Ma!uT=hnTOcW>Xnfd>~poOp5L$B`#jzMOe;=gDzDmp+|(b?eu$ > zXV<=+d-sZ|6;Bvt`=2cZ?B)XTcZ-9@d$xf5yGJ4EEhPS2Nnot$A7QALT_NMX`I3U~ > zAA7b)LV_fuh@zr0kYME=kSuCN0;nde2*5IGtH?sA6r)gzRutP{yqOYQPy&fOlw!n= > zqyRt)%SuC{6?@44a8M}>6|fOU9aqB7w2BgxLO};hV+Sf*5}9a%an`dBsxCBPC5c-U > z;{eL??z`nYsD44ho>I2BWrC+PA?2R-pt4UsE)-OdMfuu`YQ!w-8}Ph}g7i><vrbF# > zNc<$Qrxl4LQFG7oO!~)-2v4)<9xhHKGR_qxwMw^@4lGo@Ei@riJuiJEW6q19az%k8 > zTDgae4sZF$9tk9|C`2CHi9!im#O#Q~inss(0RSebC#n!#nZUA0TTuXj8_BB(!wEu7 > zR=pkNQ=lCwcJ0-oJ161*h+(HN_M#+g3Gud>EGw$0mx@|8fdo*SC@Ptxx>sMN@Wp7~ > zrT)#A-+=x99oS%i4;C2Vg%xhNV1^%d_%DJdPB`L=D~32@iS4y`V~ji2IAo6d{n+G@ > zK`t5PlT~h+;+0``+2fe&m04t)Nq!mUly&ZT=ABvg`DUSW{+Vc=jZS)KrHfwL>87DZ > z`e~`74%*`Aeo=yz(Xgs)qI(i(haplnvS%~@ez64s{9-Cape;rrO^YbF*rYOsww_F{ > zjAbfAiU6c|jL#1Lx@YdX_x`7`fp_{#>kxD6iKU9B$ors;lt_u9#QDv9Be3hqDkZuS > z<M$-ZOF@XCRk%W6FD@cLLR*mVE?Y0F(%rlr(WRJRC9%;q`?Qt`19+!jLefa@y?pwm > zltw52Qh;8W`i1EiQr_OAC)A<$JAuGQn}T(@MhnvAtc=R4^m`KAqLl<Nr{#0bmpW^H > zbN_OM{)y3lOZbT!w~;Lb<`?f9-osp>go*KmAzQdr#n_jGEwsZ^0PEir{%5ZM8YC$O > zESwa?Rx^yi#uo7^;elpDuZu*-J1unK3t<>T8P1S~HM9r|ZJ0wH?vRH)^x+Si6P<fZ > zVH509fVpk~1j~qLE=Z^d6Yn7<Ro%l4wUPoE?!iPZ$w!N~2*OM<bRgT1f^hkX6#ELJ > zsR>yqQogFl5R1s0GMs5^@>zhnesL{Ifhh=Wl-567L8-sohZH?ZKoA0efRxOHN9g(g > zM;VYrAAdygKL6kX5NeUS({#~^W<#P9OEZcwY~fXtG*NmOsgycth(b~LRTfW_#B4dT > z4Yirj9Fst{RPspzwrE2n{BgxwETA0^($N?ziIr8dYbka}W?4$%tONug07$4s7!`*k > zHcbQwWSNj<FeJ!|JhLL2^5Ywi=nKo#CPd(}WK#aYD#Og`obut7d?-{;#+(X_>0Bp; > zDgv%a!Bd`p@mjr9^N)9x&=xO|mkMtPM2Sw6q7}91MYV%NjBb>p9rfr(C5p9Dgsm3# > zAe%u{=c170O?b6S8~hUJytl<oYvq~3Q&z{7#8Gb|Rq+t~=Hi%QPDBc<u<7Lg=+cUz > z%ET`d8EHF8N``t)#1>HqodS%KH8T8UGX}gKKc(VTPE0PSzK{!G*rq!1>GX91`Nt+4 > z=u)$VWJbSm(L@@;4mT))C6!|<%QztrFn#MI$zy8+*~Sv5um)XnvC8$bhu8$NgcDhP > z-)I86k+V)je}dU;SvhMItF&Sw@2kfzIH5YMD8O}p@hT}0a*MHGWeQt>i~<s~Jf;>l > zBV5Jby{14Srp(qLZk<R1T04=Vl&7Fk@ltLx7|_Cu&9{H4>TkcJn#r{0HQim!YzUWK > zkdBwU<u&hlpQBOqu9v;-b#Hl76DimxKmu|6g%%=^1Tq-w7f))FdVWFwfpIj^9~Qbt > z8~EcFt9-8&_v!DJl9(aLwP-<7aO8s!kbuT6ctqiakcP<;&{q9rzV(%hJvK1lf7G%~ > zQONHUhWd~BPB1DN`l<x+N24Q3Dq8kQz!aQ<6#=)&!CVN{I9XvP2*>9hztF_I_A8$< > z0HGe{snvu3YpZC<=N3`Wu$BcXo-6|r2|soPFz<27N3xTdW<+pbKH`cp0~wfrl!Ejm > zP?(^88C$j(LjV#006v3J%d;|98G?`iCAdPqUZ(LIC*Z`<z!t~yX$+K?1yFpJ27gNQ > zm0~VAtUPP^j}|e5L^9-*X>!3>)9R#$fS_S$m0D6eu860<^{apXhzkh;kbsijD98uK > zmptxjC_^9WFB}4mW=x~nv#mC@XjiS-(ylhOnVs#UO&i+R?zXm_&24ded)(l*wz)Yb > z-fwT4-0ep9y2I^mcE7va^R{=q=`C-3-}~JB#<suV4Y1VYTVXuqjln~PaD{`cEC^5c > z!&FvqhHuv4|5`ZTigWRVL!9F$Yk1^(!|ab6M&l#@?8cE(@q_zg3#Gsp*GOIRkozm* > ziUTV}mQ6wvewQ`~eechK9(1AaP`yJp`q7b|^yNfn!iH)3aULigY?4pY!kId)L)4By > zZyf+3CN#E)6ZNX|m^V)^%hf3~;isd0LKq@-aLm5T=EVK~fo@m(+_hZyw_}Ta#cX>j > z<BqD{1fH1^0g<M0*sfzQG}$Xu`r{!VdCBLd=##hn<uT9R%FitGCx1A}L5}f*ABQg< > z|6$EdFCEk`XZ5QGJ+*~nGuKCjg{$E%33*Pqb#<Qiz4!e_H~)L#51;rg1P|GFgZ8qB > zUGTKlO&FPv{N~G@@V;UD-lL*V<cBDX-QRMg&bZ|;f*<=z89(y9&r0=+@r}qQfBWG- > zcIO|9{$;Q6+r=*S>w}&8TfPX|#V%5!PbmcO?doam>QC<ePx=h-9}-WY7z$oq<3Q#` > zg!b(M`wavCO#=gN12gagJun12a0Kg(14|GDNl@SaP;lS??gT~91yeBJWUvKQ@C9oy > z25E2xP4EU`um^eY23aum7Utz%59Lr$^k8N;0#D;)F9(|sXJBq3qVV&KP~!L|3YidP > zW^eS6unD=a+EmW-EN%?D&{n2!=7i8U!tm?*B?u=%=enzy_96)&O5KX@4)HJ#tt0sK > zun+z44|zkq2GIYo@9?UxIQ|3g?oK%lkvIHk5dm-PAQ1q21N!do@SIKo@d@kXulzU) > z6LTXGnWGUqaTM7v6e*9H7K-sGunrv%5MePEWl=8pa29Rx7Jo1J2#&nAu-%M<BS?|; > zo)8U5PYkK>7?YzHd(q?^?hMm#{w9t&O3oSolQ5!eW4l}f_hh5z91j=0@f(M)7Qt~G > z$#Lj{?;a#T(BjB8Mk_-mK+gv8`LZto?SV3Mjq2tpH!P9Xv@ibpQPcKOQpjK(=dYd$ > z5E@T$0cS<fDn&y^YXR3!UAAu?GZ7*GBOMc<h~g3b7?A%Eup;}96A?1~4umr5NF+P) > zo1zRPHxecHZxdnC9(*AQPEylGV-oQ170J)oB(U-9W!ao?2X*iWe{d*m&?s?GD1(wH > zk<tg7vM85QDS%Qbsgfz9GAXUHDv$Ciosuh|QY*1?EBCDydrK-1WO)1vIFP5D6vzl= > z!WQOaG!O*4hGQy{Kmn#8L|{V0$N*&jYQf9&%U`%42+ZnW)(|U%A{d|X;6%eLw_vKW > zunfno;rzo2X9WlpfNZEyU}l95H(@L20ssIY0D=GkPV15Q3UXd<GPFQJG?OC6Ktb|y > z3fDschJq9T0FT%QoEQQDVr>c*iDsyRBhXRMH1ktDQ)29)H5H%~K$8wfGtmyvOKvka > z3+FLuX9@t|H`n4prU2t|E)KnG8^h}r%kezX6Y0b;J=wE8JBrv0gFWDB0WkzA;Lak< > z2qveGo+L3hrb_+}MLq`7o)W+nwt}?yj#o(Q77$cE5@<-CE<#i36{w_z(ySpnk~~0> > zMi7zEz-jN6Y3*80p^B;$p^3Er>_H0Fk+fXJzDfZ?%nn1gponJ0sBEqx6rw*#Ws)Y~ > zT3|AqifB>{PcEnkdM4!~bi%0aA$1O9Mv2r(XGPB}VnC;^O?re!eY9hM^i66d*a{2r > zRB`c+%_r^9J<aq?J!(DCbWPc`hOp)z5=1l+Kzr~Cp6p7)0ARYNU<@MRd%TVS6NJGI > zBhY+vGZ2IhrNFFiK?(>p*7kx?lORto;!ybVtP&Ho;))EErxk{xG9w3V{AE!aqcHEq > zbDT#a@J15?z<wY?3^qcOQZ)eNi8KOr3ld}0{=uyNGFH!m;u@2v@KjI1VpWC9R1ak% > zE|n8(t1RF$K`!mIj1^V?5A}OAr%oG@gB0^3;EHQp0U<D=Paq=+%&LG?hZJn9DU54@ > z4kH4pObO0L!w`TR{z5bg2zrR;9x;Lw#=tB{i(K1CCe)|2=7b`;m3*>-UE%UwlOSH_ > zbyMxNV5L&KSR)(9a}IUSJliy5HJ0c$wqre3hFB;-XXU~O#7<r*(v0O*un$M#=!9BC > z6m8)ngtS5~1R(#xl`3Rbr8HV*qZRT4i6Vr3&?Af3AvUp-?<SyVm4FK(g_P`JNaC-N > z9MqOJAZSbJkwT>uibX!EwpCQtk`MwxB!ZOKM<UFXeXRCkX>Ewc!;eOUMmWGmT*a2Y > z<QLpPN5aHsm1WESpyGW>feaGBDkNl2?jgwjfegCTW#!{DZ1Vo#v{GJj0ZTywVhk#P > zjQixQasP<Z%Bhm_D9N}$UZXZ!G+_+5fDC+eLR<+<IG~7ZVFL&<DpbW-Cghvsqj*Zg > zq{d(-bCPqhf}DU1A}ORaO80bA_a7qRSAKy5_~UigvB}tsl^&2Mk*yUiFe|ZFEVDN( > zxwk99(tEcTe6@FcyZ0+s(0s{Pe9QMK+1D%E_j}QIecg9{;rHFJh6@7YtxmEm{Y45& > zp<(|cY<^)AG%UZ)^{JEsCh~<MV)iP^qb>8LsrZ*82slo*AVb_h0W54{4&r+xLkS|_ > zAz%wK6~H0?P%8@dq7}R%YjR5xG_1lfM{|H8Bv@xMTn8ekVyD(hx0*+I4#ln_072r$ > zuXJk*UYJ49QhEB96qo{S)MhV?6?!t*ge#(gD~vVTMj$qTiJQk1SOO;+G<=k}FOGOd > zCT3KFLw^$BI613P$_IJ|rh!;P3T_xq5p^ptE^Gc$0_tI>`a*OWVmkpR#8l?700xa` > z!xywbAofRE<hWatt0~|ZU}oblrNABhSc0_^2nHD=f+>U?k9y#cHRLdbeC}gWIhC<P > zO;x#-T^TrpNY4DB9cE`P=)+{+$UblFKV>XNUc^HxXf9mjE|Q6jqQaGEBx#ipD<aJw > z0!3E;wxCwBsE+cejkxdEz@##4fsYRaYE#K~-N2T_Y$67v3lzjgro}yEX?6#x7TiG! > zUN;JAA#h-c%gU41<g@8|m5auMk|s$3=wXtA#MLHAE>y*rfa!`#BcKZxNxn)(8gxzO > z(=c%7LqX9nzR5{5qzw2YMAF8IF!~m>z(pGDQ)DF~h!m?dz$8H0B5?Li;;1or*_da7 > zSG)-&KtfZd4=Y@x!!ClO1Mg|Y_8wZgB3^n@Vw#*5En*XD701+iyYZE!da8?qm8trw > zuNrGsLK7y4fqr$j=9Q&3BXV#;F|49=@T!Kv^{k^%lPCJ)Ol8#8Lo4QIFNhdy_5$Mn > z?piO}Ad$*USZuU&xM!sfiwjKCDRO6rg~v<kp>;w>2}WTQ$Y2XlIHz7JuPJk)DMFL; > z#uhfCd7ekJNI?>sidnOm<>E#G<i~l)s;BJrS3m9{rQ#mC^RrZ9h4HE;+gLU40gn?d > zb>><!@cJ?kqspQS&gS)`?g512lddNQCK}YGX814a5$0Z1wWupT(FHcfx*~9ux<~_r > znBpF+im5dO8@=l_$n%u3y1m`oyx%*%-<yTR=@easKc+6s>MWX9mJEjKo-`yU*iXz} > zH<>)iWKY8lsB9wE@to+QQ&@DIWtq&31(JA76d$D@)=WOul0T6)R9fLooMg%WrbL<% > zSIFu^JthU1ZlNS@Hwsw6LY~4*Qal0gfuj?$FMj0~1dPD`fm-lm0vd%b?7<ysIhFFm > zm#nBSfjn8D04n_3L6hmm`QsojBB>IQM+W5n?12mX&cz`OJwEh3qBmP4t(=TSRfblJ > zCY;cGH;)tpDt_6LVj9=<Pnl06sXlbY21JV}@XX7jm{dj2h*m#&?9JoDjbIwq>>)!Q > zeArG=@toQxaV~u0SA8-4(lz~kEuGWTchl*&(?k8?rc%`Fchu$g)K%ToS^d*JJ=I$s > z2eldj%8Wtqs52{wG^XHxDJM1)KokB{D%?RuB;iJw0w<7wT@`0ls3M5}Nle4ShPpkX > zZ`@VC&WS0C3MwXdHB<ziPooXo6`t)yWxzcFCLvw;2Qkb}Z-QWgQmT$x3V#(N0$8DT > zx;aohvoJZqG-snz#hRBVLkf@}*L6I(ZKqu{%zf&Lh_`)U9?M)G_aFKg2^9V(7Q`)d > zE#MX8fEI`jW39}Fr$JPc<}4)9Bmvzg!ZA9AfMbmghbJ{T;(dZ33X16>lOPI;9Sc)_ > zeRe9C^lTGEQ`R_sDad3r=n&9A9!K+iG*a^~Mv6fm0wYxZ<5<{U=w}P+9j_1|!&qfB > zsAA*7lawnKyx3d5&3>w_`s~#nO@#=?zGYuc1^_1`C*6(!Q>g&|>s}u<7ZuSi(<CzQ > z>5kAY8XxoN!tu%O3|;WrF3t~s(+FQ8&98Cq?(gJL3JWn3pN^N)4xrZ35&IMHV~+bg > z(GV}O@j<b6HEqv#$*uw$6(KR9UZ3i0HxnuUAGFq@t+Cj|)S;j{_t*aSQ#tK{fB4H0 > zYiM#KC<v6FV;Mtl`E$dw13)wdEye<7-HEYZN-sE={})9@8fR$xBPUx;P7TFf8MS{m > zz+c)i`;={uVq?Q%$wT;wfBwBu{^|ezWs!y8(nucdHvq!kzk&V+{u5}>V8Mh04c>F8 > z5aK|D3?B}(_zxq)j0ZP*YzUI$M2Y_vb_|(Nqd|%vRhk_C3A1BCmLF@zq<L|rO_4ie > zCVc79B+#J|Z34|{6XQ{yCMn|N`LLi#l{Zg%q_|Y6PNYnWmh8C|Dp-_Zh04?kfT6&4 > zw%E36YfGTL1PK_X{pAtwM!gLACiH6%@Ib-;2pi6ecyHpqi~lwb{CIHW!jlgxzRVc2 > zW6h8`OZH3|wB^jCH<#{w`g7{gt4EuDjT*LU*{o^1j*a_n<=vurxAtutIPTfQYZt#w > zJh^e@$d@~B4qZ5N>CdT0?~VO;_Tbx(+jTzHCgG32mRg@2o;Ba~JK0ZHJ=(qc_TlB5 > zwoiX-^Yquj_NPr=dgsBHUv&g_2ikq8;iubR_QVDM*MsdG^h*+TC6tzc8g9s8haP?i > zVu&J+NMea5o`_<KDz2DbhAh4aV~jG+NMnsQ-iTw4JGm$$QDlK=-cD3~^;1njnt0@l > zN$SXCiBdU9Bv@t{)#QsjcH|aZZ^=d1cU67~W|(4*NoJX5hIi$eYOcv<n{K}8W}0f^ > z2jh74!HH*{RoV$)PfMc6XK`ckXwNSXx&z^Zg%+Ab5?<;FX{3@)O6jC>R*Gq+nr_Nz > zikH@eQK(l5`PGnqhKJEpDUnp9imFm-s+d!P>QSs4Mb%YEv(DPms6R%TRjDLR<yEUZ > zts2sh$_7;{Nm|8JP+Cs?!&a7Tad{zSSN`Y!9fAgG2QGKvc02BS<$fz}y5y>BF1zTi > z3vavezAJCN^xn&@aszEU;JNwY+poL-4jk~j1bch1zXjWyu)qrgj4)z%Ar#+&0^${D > zPq$%gv0}fp62(I6iEK*(q-fmnYW$^)vVM!v39-TnK8!NQ2dd0+&I;+}56^?y5(PPu > z1?t`=w)8w7!~PNEb7A%bom|iMY(X((+bSAiT!vQYD2APW4R+XKkL@+tW}l6A+Gl>M > z;yt!Vkw6lpNcDEyCtFqI76|}Bid-MnEw@h}x#FmggK6@r+KO|y&@FHyL5f9nBSE!6 > ztxNz!;2#yl$pl?9v`GR25jEE)1C;^)fGvR`zR(r`Bmlq`^+dT*&y)0>(JxadkOV6U > zZ6QGtl`kdG?n$)rYU!t^zPddKxxV)l6Bb0u7PDTtmMw2(iS46{e-D26;(y5Y_~xIF > z{^I3leDOcFY|@H9w)EaJ{QlgcKr1!<Qi>FXPC<sm*zXof(4Tk;1G1ipOlRq9An4+U > zKUVaN6cC)kD0=3K1hB(U2CRu2IFSUFd@Ok4iCfR4fPg~P?iXCRpzgNNo>FK*WK$5s > z{gfa<{zaisT!7(6ZehV2c5p8y`~?bCxR)08Yzi|R%oHTJ5keWt7Kp;yqPT_@2WoMP > zT;x>wx(LQFiZM(sYQ+}dMvCD7A!HM9lfr}s0=EQgp@VP>fb>{Z6fS6DAq(Mx>7bR7 > z39zFe^V>-u|7aDHOd$aZ;6m3bWQ#>2p&-x$0P+qZ0g5C+b^l;v0{l0aO~UPSqr4+m > z&h!+Dh-xFL^p#p%nUGe7<SEO#6{#9>4?XfRBW2LUKW@Q+Q54H7_aK~FBtVJ?JcW?K > ztA{H#Fv)*x0SK_W<Ue%z4>x>nm4ReOsT5ETf84_jQ8?Wm-B!-pWdxiVaf1{{A`n}s > z@tM(_W;GFlO*ceyAVBj+Q~rTHYH2A#2kDa68fLJEAyjD*YpBc~>M(~!6ru^0XhkW? > zP>jY*T@BLcLN|KRiHbD;q$JJgNKIN&kfL;@DsAaUUn(3I*+L0zAVn1FAq7(CupYli > zX3x0LM0eDpAnhPSp|+p}K^5kSK$$}PiuDU6*n$*B6&+`E*M|L3;ikXXLdbq`k53wv > zJyH<bQeEf@DfEmJTqT8C4JFnG6^0Wt`~{-!WYGiW)ujrh*<*AE#lqa96-pTDFP6Xx > zO-R6KDf@*hLPjJljDmo^Fi(ZXun97Ru4-k)*c8SfKPi9~Gq*@WLGVJrp%%1IVb$p- > ze5V)!9?ei8Bu^4Bt68g3rn5^0Z6rtQG$kO>ARV2egdj8_MrCVXV{EQ-pUb2#LU+2< > zt!{^0<O&7YqZN_=@P`!0a1RN1b0C|%MH!I5iqS>H6=Xm{Mz)ZI;7Q?~2`R+{RQZqo > z2?STMRDdnc_zxi9jUfdoMJ5jdU-`cC9Sfj`IJG&L0F+`WQZV3PyPM0C-9rK6t&l)| > z0f17dvJA&doOR7);F<iR6$;?neieX(TG)dOpCgOgCb<xht&=8A$)6N9>E47K?;>0P > z00IEu#+|g{zy-<W9$kUC(@H@C0GPt;)OimoJXrz){)jz%3}l83c_2kTa*_j6Tkl~> > zdjsuW*-i}SILmoQbFQ<U=Zsw?@Nb4&AjK^LkN_maEdf_9LmLfy4=qTchY3I$d4TrD > z!MGy|k<N?%0t{=95CUdBPpkAsULl0ae(bBa(5+)~I+%MzpdO|5#V-SZ06{m3r}s!< > zMcQ!aKixS->zSXYe;LH6BAU>LaV%tiSy=fc!PP%p_LS$MLcEmKi7sq~sTng<PQ!Nt > z2?Xr@km_hwZb1oiO!G!s{cO>3jJLY0wr=r4=_tl>iq}fi)*7YeUF&<_U6d}r{|#`p > zN$Vd_wnZBv@sFHtc)YaBNPEQ@On&ZDB@c0h1bj!4?0WM_4`Im!E=cgtsW>6og)Or- > zd}UWe^A|usNU9S4)PZb*6xAyT{s!5QTl@kEs+#e{1CAq$`|h8=oMbMAS(ui`GD<i0 > z=RZjQ(c%OE8VNDi$(QZni$W>=D-~`s2SgD{3&T^pRzXM`%6X3kT$$xB*{?(r$jXCg > zH|s1%NI|-u5f1#ZLLID-KWH-WkyMM#*D7eX@WQXX!c_5nWxV4(+IYx6{-k`Jd`cIu > zQRY$}^P7)+=Pl1t&R^d1qX&KEC6D^fXGnf}CPh;F{0mO3dLU0FMJ*7h?(g1jk*fMd > z$xJTT1Wb=ezes?s2T}^{8gXue2{gs#0(|mJfohJ6K+nR86KPlaZEnPv?5~XI<Lkxt > zxUZVaOfUM-lOCmx75-ls+ts(aV%0r8HvW*%kG~|avI&u`c_@oC@0M^K_E`nd3J9?O > zc6qS`hBXj(H9zqaRb)U^J2VjY=M<_lKdR*s0Eia?cxJi-S|5XegYj75hIoB}HT8By > z`1WQ%=YlVYHvR^KGf0Dwk`@Li1qLSxofJ9$FbWZX1X3VO#WZC}@I(VaI1x}`>9m1V > zwl{&cPOk$5v9=KRFabDb5NVVIM#T}gaD`Rqgs+nX<8cp;lK?c8PCF+~dr@I2;W-d7 > z3K9?mLO2-vWDsv7Ukq1+FtTB<^EyV?3Kk+qv-1!(HHZe`g?0!%nS%hFG!dX=7EQ)Z > zg;8ThbV<2nWJnf$x8eqpU=jxyH>y>JQV;<~m|VZ~g%2@U7Lo+6uoRLQPL%ln7ZtW& > zWu}Q7kxO7FZ%bz_Z02UfwTHndjFKXQ!)T1h7+rV;9tFZ7Bcm@0u|NURC)0>C`+<yr > z!i*wfS(HIEO0^MFa5B+|9ysF|HnTq#!wNg-5mIDD_QoimBM`-Sjq^y4EFzEfh>!Uw > zB7tWl&H^kDF)Po~5wG%3K=E=#LUd9R7RXW)%LI^OVN3dGEW1Kr5E&)-^(4RYBninY > z!z4Jmf=97rN)^c!0XZxo*>E5EkM0vP8QBp5HF(<-7l!v8>34dcmwKg_lj?_l>vxkn > ziIYJIltU>m9VL`JxsyMMlt-zQO=*))>61{|lT!(d3Db=?(=wzXA27rJKA6#!_p&ln > z$&^x=c`xG|TX~k=*gksIAuyAbUzwH6#c9ZeTohs<`6iKhsh1&gk9+Btf7ybDWF{NQ > zCb|ZgZPJk_VwhuLm=u{Lh7*|CBa>S~lYx1dmx-BpnVFjjn0?tMaLJip0-ERNCZm}r > z?3gu*a&I*TP@(CXhv}KGDVsAWkF}JN3JH?GGLc?Ek*CBgv)MMh>5(LPn>`^*D~X%Q > zX)FwxB+Igx74a2Gf)&HWkJ;0Uzj%UU8I{{9mPF~4+?kYQsh!=ql-_A4i5H&WIiBfB > zp5NJ?;%T1h8K3VdF%QKtPM0+KNsVtwE^f&t@L8S#>YmG36kN&wKnjAD;ux0a_>3Cj > z7taZn1(KbCk%I1sMV1Mi8G4Mdsi7UZYaCh;npK=Dl9(PkCOp<~F4+^@<RLFHKbyFm > zS0OvOLMyr%7Pb-|E;?rF1V=RilZ5Asl?jX{N~C)TqD6|N({-eQp+t)FHl{h7U{wZ6 > zaBO%%W1r+t9K!`c5GZd00D)tr79s`Awt)%YLZl#mQ}af;0|3;NgJ98TsYV7$@NTWp > zrU3u|E_4ruc5Hu>1OY%zMph_qV<CuCJ3!=aNTjAt)TW;_TmL`?Q;?FV*^Vm+kFA-c > znHq3Qs;QmIMK}1HHQI$_;7c0Po5(qxxrv+tDUhjpEC$j44xr?zAi+O0b`m-^W2KrA > zGbLOe`GAE{3IovwN^lS&s!sP;5K)#B8R<vwMPCDU5v}k}B2x;sY7nlVj>K23cGwff > zS_UU$4^n_gQ^EmzBu|lsD~I%6$94~Kx^zE!okA*?@+qJII-u)`uk*>C`<k!yO0OVg > zl=|AQ_bRXj>#y{Qu>5MU1beWjp&K9=1)ro^_MioVmV-zGGI0}vJ=8P$g<xXJfba!7 > z2rvp^>Znp91r}R%kO5Xd_DK+=1mUE8U?s9raC>+m0qtN<DzpWH27l7zvsp@~Vt8#0 > zJFxbOv@cVFgFy<LfI&@!Y=Ovsx1u`|_!p&MJuCMAtb36J5x^F3BM^E6JLG^dRNz9T > zV142t1vwNLOAD+`^b*Fljgnytl0c^3m$iBESx;kDT@`2mK(>WpwjKLb2=rDpNuhKp > zA$GZ3m^!PED`%Vvxsw|{plXFk&<iuxHB<FoNmWb;U_eNxNBMPM8Q}^Vwgm|H54eU& > zzO$$4L<(I)2J~eMSLQ6UyQ>p&bcED3Tp+3Pvvx_4Q_e$X?=&^^#RbOeRUENoCYGtT > zV28JWUb$p)Q&0w12PzZ53S%HV2qC@c#RV`r5elFQW55ONC1w`J1)}hGTVSXjAy26a > zy(V#9Q6M)I@e6jqtzHImacf||mJoUeUmg+v1+bNi;<N?27)m(tqmwz4bJ4hxOSuy) > za1?C8&gCNB23I`gG*e|-?S?cM(?je)Xd7V*oPYp7<w8gJ3;Cl!Nx)NS)DBM9Gh4tr > zyaPi$m9uz}r6u?aGY6~+FvB9#T6|GjGHVYhgg=<3ZXq|p7~{4y+eTU1KQ}bBd!YbP > zzz(;Nyw~SdSXvXM^)_DH#L`B90DuHvs)2)}1o#QXRq6^7<Ty|(w;JQN{G<c{FgKPp > z1xOGH31C2a5ngx0rbrL~NuYt+w~G@&Z;7Hc@2Hs<EXnC3!INysn*yr9G!CbQ1heA{ > z^R)#JXN7u5W{rZ#A|nY>5X#Y{PyPV^WfOuwnG;_;R~T*u1X4iCA>md?)?qLlUTa4R > zeXIp*x^;vEPNxL~&NPX2M|))Fyc&ut#bk$E0Ip;ZGG3Tm5@5UMCP>IE5Y$|(<%FUe > zCQoX$bpsIvPjwG^M|(fV6&7a}P4_I)TZQwxIS$cF4L3EqdK4TmNp3|Gc(=}f)C&7_ > zcQffV(-M;e1;JLMutvMENt>`o`>zuH&<u;w)`*@IJ<$<e(I4H>5G~Rf4btiPu)&tZ > zJXL_cr)fd-7hHe<?BEJYAZk;v3K*nrX(b6;u%`JF8xtge@G(KhR|ZbCwM?sS*ryPs > zRSW%KC<vHZQZs_cmZjM^(iYAC(cu}zrgeP?*aj971X35%4)S43H8&k-4;-k}8@Oyd > z1#Oa%Q=S!q{}2JW+<#pC7kA8!t}xYgOrbRK3sk^-eUU-g2W&2sQwtQ=y@fPdEEzk* > z9#^Bdk9?s+ddZi4K9!8wn{9)oG65Ktx`A{-wg64yq;j?J5TbidT!3~TCqF$~5G|w= > zulsi^R;&gzKjvW%6vR)abrBMf&Z<}nxb5347o8cw1sKNKWCq$ySaKA63wGe!8G%gg > z+;-aBg)n9kt-Pm>Vuc`QOjtIF0mcyNOl0gU&GyidRv3NJZCY|MN2qvCz3p@_hJ`ja > z0cLd&T%fzj3n~eWgmxJJRq8v0DyQDPy^gQci<H^W7|PiXZZ?_?;S;_l8O%fxy9KKA > z3+)g$$mBnGF{O6%Q#m}yAHzEnfVLQf0MsPB2yvxyQ(*Iz0Om~|OU8<Q2h$>B<7cQp > zVAaG9;z$YLv`!P`jD^(k1;9w^w?Tl$>ShlUK&LkQ5(MPNRQeLb>Znc=JQ5qm%jPuK > zhH6W-;&HRG<tRfUqc=&=<b`5jg*q5%zUG6W1Zf)>$z~93F68*R7vP9A;>s7*E#}mO > zLufwOEx0I-teTKK5)w|~jZP+v?&y*JC7B#?Dk&-%Ssn{H+Z*6b4-w`u>XHKaL%pQu > ztzsy|^c5Icc6=iL>LY@qiiwhELM+-?k@%&N4{7U6hw5B0ElU>|38^f<P{1gfkm@uQ > znZ7CpSr&WIks_3kr7#698IKA+P=(jvRaw$mt?eEi(j?vOSk3Jh%`E|Y)!82I+g|S9 > zZtm$W?(6<8C(Q~8cr$3}m6Ui%?H~a|)S%?(KAWVUjWsXQ*zQzmpz6L?{5kIjs*Kdg > z9!7LD3BT|A`67lQtAJ5l6}q^}70Hs$@txiAA5SSRvd)3kqC4>*MHt^G60SR1C4Pe> > zeyO6wxh9agA&-fiJUXHt(!gyduZ|1yKYyA)FZ6hl!3(dJ5<m1wpWrLe$X2wP8?W?H > z-z1PO^;J**K4mAIDmg31>FdM^q0kEUw#kqja*#Q%oUwi)v+gEp|LSQ!ocBCSyZ$1s > zYMVI9qQhCDU*GG%sL<8^zy_r)126dK?(Ktr?uXCrgb(iaqIrm4_=@lDi~snJFZq?f > zoy8@g01xp5%K1p2pY;AMP<l~UiF%|D`TPDa0*|1dulfH@nhToxW*PBEua=RGf|G5j > zP;d3SKO<Dn`@cUZ!xZyt^30u5{OB`sDbn`ovG>1%qsLFl)b7B)*!aM2{k?zv*)Jpf > z@-c74RQ@^pqPn72VKOFSjWZ)6ECg~j5>>EM=wE&}W77Tr@c!uEmYp9WJk#)V5&s(k > z5WhD6+4}dZ;6H>36)r?*km19F4HG(q7}27^U-lA^s#wt87JC#G0#F#zo~@H9QD*#= > zBuPSh|44SES<~iCoH=#w<k{2bPoP1C4kcRD=uxCeiOy8o)ag^GQKe3$TGi@RtO!Z^ > zV`%H;zYU|f{u}!*Cd0C2@6jx4Ps-VZX<@!4yKoCZm^jxm+$15vma?s+Bv8PyqhP|L > z7H(1ybnUM)n#xKY`|I8o#U%ye%?h#d<;)MuZj$nqCa2Oc698D907%SVgOjk5%TO=? > z38A-?1{~M-Ed`nomU)dZ<iLbmSOeJ7F5%vVYo(Am`-_u=m(^vLDLYt6tJ)7YC4ejc > z8@z9XTy1UHgwX5R<F>{sUkx%LDf=a5E{xgEmj71%L&>ZJ5?F{NktTX5L5Ub*&?1E% > zlyE@`9jtJn2`$tR!wVzqP{9p73=u>QM{KY}6h%ytL={_P@x>Eej1fi%Y0PlN8FQ>r > z#~Wqb@y8x*{E){F5gZc8BZDN;L?oG1vPm4F^zlh5leE&wE2FFuOD?s{a>_4->=Mi| > z#q_dFrW)d^l_>oAWeVEjGRPk{)6-GK2fbp;AU$<@ur9LdY{->1f8heKUtCFmoqz6W > z<;6r7HS5ua1WUq~%1W%0PSI}RLW?>n5-C(6F^X$4^xh;zPzpzNNYG7LJ@L=~h+v%& > zqbbDt)6IL@;N&G*4e~V!Sqs`^3^%6$^%f=KTu@O)?RoT<QlxP6P*f*U0E&KM1E4w# > zsnB(|Y$L3eR5v&&6roMPV-~zOk!99dJYB&K(>IrGmcm=a^~Wnp7xecRavPdr)dY=f > zNG1PHaxSDK3>>L5iYvDGVvIA^cw>yy<oIKdLl${tl1oO)V=W5g;-z~i&_s(Mq;T#Y > z37q?f(1xUt0EjIDhDhfidNymY0Dwq<<y)rM**<&xD%vH4reF;Sp9M48u!rv9Lb!X% > zkfPDZ{JF;!NgCy1ld;N0NNdHQb8iAt5~!!5dlZ;^>?j)Df)(W?U<IN7t+^_L<%72X > z;=ANr8zZJDD2=HV2`<C$@W0VY=oSt_`)F!PC08zjzw5pm=YP2P`kykis7SEENGVNh > zyAdM&ADwv)_@2zwR%mR~IvN{^wCfTqrh66`J#UD#Zf~uzB5Tx@)3e^T9(%u{kLtCz > zMu^RtCSY`TSAbV&th4_CZuIDrKbX8mV;?*Hg<Ro66n~T`g{#e7>s<SPxWAlOhIp($ > zwg?b_2BZ;*CB$PUV_*Xv_&^9o5Go6lU<ECBK@4WlWK4Ne6zb8zWO?uxWGIvZYT>3* > z6zWrMas@&7<PHH9WQD&7&_QgW1Ug})6|^B(Z~k#Z6-nWH)Ivi4w8&r_0r{sO_eel1 > zYLNsNYJ?QA;bF5hA%VN7?H5a6#TD;zkS&mcS)BS|J^s@WC7h62g_=SV*kLFYZlQll > zaA6zY$b~{tAq(|L8mP`jFi>^zO;EUFo4PWQTxClNhIx}4306WirjS<&@DCI#Vv7K{ > zf)jFEWJ~ISLKT84Q7$Y<BD<o*K>?5^iHZWa3?egx+(HOB3}3NgX+;T4A&MJ$6AvE( > zoH2|dnB*~)6v)vKDj>@ii?rqClz^!S5h`0$^apVI!NPS}D2!vQCJtGcfMzCb43cQ& > ziHvD5WlBLT5@FwgG~+Oo#M2}#>D_)bcu#!hQ=j`(3I+N9`A>ibRG|3WO8eNs6*mB{ > zW#iz=FMv>wx*hZ%Zh(TD%zy+|Fb^SHNJ42OfV?swp%rU^i@mr49j>@yaAR?YwE&e6 > zDNyPb6QG}6l7JlvL=SR}L!=Y{K$qf)^d2M>P5Y2gi=lo+FlLF}KRAk=)I<~^Zioa| > zHZTtNh_4HpXay9BM*)VK4+zy<06n;Ji`SH53-_1+@HQbvu$~ooAL-~#qldfDC?uva > zxy4np3cK&QL6wk8!g-upS1IHTa?%pcA|sko;Qb<@88ypuGFMr;dNh5msORn!Qp=c( > zMNcPnnLk`Hfw9`jPIWt~J;-1hDfpBwmaCg!qF{^v;=uGgxB~zJ05FBQF$6lUn8HaF > z;uU|uWla*e8x<sTTJgP<csOg?C-320)1o$bc5(#=kWiWd>2<dp=<QE!mJlVl0xnvq > zta1L~9|JawKnXO6$F_7P{E{iZX3}qe`TO4`?Ki;w9Wa2UWZ?S}_`n5b@PZwjU<8M$ > z!L+ikgdc2S2u~Qp8!nTEJAB~}XBfl+9<hQ&d}0oh7{wvp51*9K3v4D!SN}l38`J0} > zHZ^ZBQaItP65_Q4#B?lfPy(PVqlGANVP$9Gtc8S#Mta#u2@y~ToZLjNUnl@xObl*< > ze$lK+hUCV#xKyShbYmK_aTRW|2Le8bM<h`Hk}Pjpvt<xq4{jQSk~dst&1%+5H{IgS > zMJ?s3JS2ru;Hu7T{-6nQY{-Hr0eNOl$PK9lD)zKQjX569j(c2ao3L^+qMC>_BSZ#J > zHRLi%=vg^y^kp70F(gx{^9mvIWnV}EB<+0VF$*%VR<si$uJER)q$w&}MB%(2VRb7a > z04{|vOFm?lk6E_R>}NxyUr?xr*NjX%Mgm)Is+Jk4?aMHEvIJ@q8)&=T{cd=R%%Ab5 > zcfIX>@1iip9SSh(&5+Qj39Qo=3gtIf+E7n>kmBDEflI<oV=uUXCaaCQnFK<+5S{{D > z6gJ^XDK?M<!-%dSspUdu9BPkpv8^HhFDZor;+%j095s3FYXc<27I3+WtqVY?#qb(M > zfwtX(;K4<!J?v2!GHl_sH_wdFxBJpxT-Ln^5ptu%6-$EC10e^mEGZ@ca#U&56$-dT > z8>%{;x|y|C2v_)6FsHeNOvVnTSuaEE@NfK?PwLU(^unG~@|D|ytzC|;a1~$=U&I6m > z5z08*8d_N09X+@TxqDu0ApwqLtMB*7NM&0du>i!p7Hiuc@*W~T$<PdI1!>$#OFkCJ > zue|Q|%u9R;c)$X=6W_b%eeZqWk$L}K_`@guKRKxDY9=;>-I{{6VuLJV7m5?3r=hSH > zatl(7zJEzL3z<)t!+qJJjczah>_ZlLA33{r_6>0dx1}9vGT@l|G_?3HT(|y$xZ>IN > zKON`ISLR!j0_c;pWZ@xOASP_-p^%sW9@2<e(}IwQtq@`dB6B_lbQ4=Z0Ns%xP(c;U > zGZ+KBv*+`N3dFz&;G;CE2+q1Rf540pj6OG+zJdt0>X@O5zz9>j!Tf_XD65PX`k~k} > z9|dr;LD@goaT6uri0Z??Jd?FzVx&Z>zR6n<DR34XgtT9{g(&!*N+Af&GOR2cG>h;G > zd})PS07AoJh@{b#`!kgfETwhosxBjx*-I0Xkf(Hm7`mgq;_E{{{KE?hK0q8qLM+6K > zDJa$pvXDXumx+QT*n;5y8n^_|rK-A_D2RXrsF~;R4ZLwg3GlVyK!BsMhkFpZAZvw% > z8;%08B;A4w*U7~8n7X<E4TEcwQcQv<n8fpP9p*SMw`mAc$i@7?tfp!&iu)lXNP;PF > z1)KYY3BW4zi7YY@j)EvGXM{$yaub0l5Hh$M;xHoxC>v*7r)-o(wa~(K)2*@atAbd# > zyHmzw(>t+PyG{J8!)t{|_y<Ygh=GI0e6&PNlmIR)#a!G&)gprcPyhf(N6&Jtgj~qz > zlB%=lssu={i2J(a0!CpZx+uuS4d6n<`=o+MEp<c)^75@W?1(Zjy?;PRg-ik@z%vB^ > z0^d3sgNv9XP_O6z__|{x#^5@*Ak#_Js*6|PpML;3@gg6Z#0^kX9{%9N%7U-klP?5v > zrxjDN6f?1`j4&4KO04ustprQ1{7SM6%d#9xvrJ30Y_YBMu(NbawS>#HluNoSvAML% > zy1dG`#7n-^%e(Z;3fl=%+mIU>Og}NOsVL09#EB>A3B%k=!)(kWxwRB~OsSxijv%!# > zvP{7=6U+>?4>5?Psu6|ph<Iw4GlG}|VlP8%P1k(Qs}MxkoK4!S&7F`<2iicTXouEV > z4$gQf%V`ST(aoM9PTS;|kBABUlb{H!2myMpsk}Yq%sqBHPV2l*?0g8^%uep?PC(pF > z?+j1zY`4$<w8M-L5S5rGhJ?!TY)|*>J@9-_`JB&vicg(b&ItO&n`n!NSU>;lPXJ|z > z{}j*wbxr;3$qZ_^ydXVUgpT!t4hXGB1eMUBP|%4{P^l~++@mk3^vk?#%f19r4-HYV > z{Lm65(YHiV$P`f)Jy8lv(GiW&7KPCnT~QaU(Z#$`9i`D7wNV-^&CkpU&g4tV?9s6t > zQW-T;6Xj7R_0g*cQWA;GD4kLtyAvw?k(!WF!c>rRJIy?#C*7k@F&)!@vQIKSQ#2(R > zGbKdhOjCTrPo98NjgeEEcuw03km+1a_H<J|?bDH2Q$Gz<L6r(XolF`T)cJ(cG#yg& > zOwTX>O~;rhR79OrN)-x8tyE0SRGdIWV=R{qB!gI#uB_n+wIGU84bHfeP?SQCifB_! > zwTS~g(D0}US8WS`@eB&xM0;4ATt(FbjW|`6P~H4hdni9b`-QS0P&*Y64b4-l#L*>n > z(j0BlX_eM!tyXNkR&7<$Y~@zDEKO_mR&h;IYTZ_H4cBw+)^s&jXdTxgxroZ3G|DuT > zR6~&o@iN4m)fgEypP&dJa#tk<*m4zE1ycwzkTh_i$RvorBSQiOC<PZevLtv|;EWIj > z5QA>vgl}0GNW=&#005(rf@;%2Ef@ugAOi&;790fG00;tPK?pKXzF<iL0H|4-{GfaP > zXrlx`8?{-(Z|bs-=(3Lu*^w=SDVPx;MK^~ri<Z#SOMO+Sjat@}TB@zuoA4NY9m=7! > z1*6Q`yvVztFfOh@6;JgEjglRUGe)abidG;4;1Mmz^M^-q%BHEDtSKM2uv;~BIkA}n > zk~~|pX^)S9AK}WuwI~I*9gBN#gRi-`?MYmm09CMoShomOhUhN(8M}crM!MaCvRMns > z(axv5(+v&NwLM+b-P6=v-B;zHDHwxXq$7{us33D8N0AMLnI9B-i;SRMDYzzs;HUuT > zEt7Svh~cq7D~%x2z6H6A<2|7!EVraNUfZ=e!y%@_+F4bj-B`Q@qCo(op;OlX71A91 > z1u7tg5Bi@R5{#ZD794Y_Uy?L48jM*x8UXOMH_?^8yR|7u!~9d(f>5ipxWWzeS1Ita > zO%SDD%9VO}!`QH2{EfmKBj5mfC-i*OdV1RN-C+6DU=IFZ@gxc2@h*s3x*u8<tx47| > zM2O4?n^Ks7NaTp2s>C*ET!9pu!MTNdC<Op2gJeA;%W<t(JR_C+Kl34jNgxX#Zi(|d > zi!ErS!8IFexeT3>09OU!<oT?(1vqvXgGZbh=E6EMijJmox}hYV;1~riAOn)5I+@}C > z$>=M~WmTFXp5rRK!I6R_TrbcXRq<koQV>X1MTl46fCNUkf5<q@kqle^m;lRd3ANo1 > z3}sgIoG&gN*mWgXb#2#7?qp8><bd_$P;S?$W!F(YWlRR;R7PcVRpnM@<xRd(*VPq9 > zlL+hsCsi4>Ly4QY$rM`%#w*j6gF(W8ksJHdGT)Gdl*JBFDupJB5H{%+E<hG)LFS2I > zqA5BEO~W5I0}eHEK-ux8EOq5rUS)HJ%<@%$Ewn#20*Pf3LsyvskboXBbPzg<g16u` > z+KFJ0UEU4Mvr^+hX~8pP@j)KE!5nmt0$CM#HJ>R!fQUISm>MFf3E5A3rFJkk3{l#Z > z&_mP3Vvy!j4-RRPuFZmKnRWUFBoI8>(Y3^-8AdtVE=Yn1pqlRgaA}2V1K=6Exv_;I > zi~v#OK7=3zzlEiaa2+6kCC-Ziq=I3JLu#cyIXl_6$pZpg;OT~n$(Vq+k_L)_Q``ku > zvvwp!@6DxJ8lPG8UMc9`!sEolBbPQ#<o(GER8*Uknt<86yU7r(!u#ruI4S25&@GS* > zt-Tp80G|89xXLNwy55UlVCo9i57Kqk>g;OEp3jlKY|Z{oL<+%K;~~zHu~FLsB%lX3 > zP=Fk=vPgMfO=u1@`m|Yz8{)LTY(t%%p+->&#*jEb<+_CcP@@c#N#GW0AcUsPX3WqA > z!AVNrIXjnyfs0q67Wh@eOEDvGhAAl2m3!F2j$jt!9+ZRs9u!_HUnIhiVK$XH(?2GR > zLNL>-?B<nqULxYI>^pQ+_1weEK5qRMzWm;A|K2^OxQ7j>#4Sh!ru(4{0G?Y&sJRgc > zn9T&S;Q~oux_=>UHz9#g-HVXwhwT9yN=fdlAqATWADGCgRyYeXuo+Y=2&t|M`Kc<c > z$w%C&Do;sP0+HzdR>6N`GVviH_1NkK;6jZ^Me}hE%^l?S>9WPMlbDJn4xpd0$t|>A > zn+cfP<Dxq*c$Ar<#=mNd1laM*fg70M0_VUS)02ztO1*@dj3#fCCl?!6c#XXnt-M%P > z+oNm+(qvOU=W>2!KHqac5A;4CbU?>ea3yp-PjpfL{&Phq=R$w<Mu+7>H<EnGMNM!a > z-h4z-NZXdg$s@AAm(_?z$@DPVKUYA2ID5p9==7tBzy86gs)LAjz>WxrbwimygqZc? > zJ&)^l2)f!?&fs3o0Bk^$zp%eA=X61b^hG~*B4wA>h_i`|7)ij3xIuzHn+PRHpJ2PG > zcJPH(QGzohkoBs<gxG|B4#Hc*4}}<yAUGQKA*`MK1+GwccCU_(x(G{hh*^)bXuqUQ > zI3k2dNpN!rayObFBp4-p)`juXrB%(3&T)r#&HjFPi65u{7xODlRar#qoIS7Uuuwvy > z3CA^`z@GTdD3`heTbl^^oX|}=t-=E((ES98i-*%v|CoaHTUHIlQ%kOFlJEIGjCh|9 > zdJS^jC{xr21G8V4^^8UeK7pc(2MVLV?5OXIsn40;jFKWXiXoj6w^;SaHjeW|&6H5L > z3>JE`=kK#md)_<gxcUi}caPZm;z$l#fQ!8WtqAdbO0|~?Sw-@nc+f}&d<Hc}k5?{M > zy?c`nRy)4X+M{Huq|So~Oh!+3M5laYcl1b~e9X7}%>Vk!m-NoB{LLSI(C2*9C;e1D > z^eV+KE!DxMX#M-zdem3;(=YuiEea9C6R2OML%sbAiG9z!<Ot#D`)>HXKYofwe&u(! > zk$!ro5ZmQ%H|Kwf=_l2t7~Sa9`Ksjk=<j}l|MGtD&nF71dL$wLp)mjO2O0Fg38P>C > z-A{@}#lx}pX!ip8^uPbAuz&pD{|S02fbjQkpg@8H4<bzX@7}?J3J=<QDAA!qh7t{8 > z{8#az#fu0Xdi2;aB*~H{Pohkz5@ke%BUi$l_;8~~kPQ#MRA{p%PL3yYE}ZFeU`vfR > z2Lk<gG^op^2qOYW7_pr#s;#QpifAuEf`n0jJp`*zY(laK%^pNMQ0+gqZo|SI8@KG- > zvvt$nU7NS<-nVl9(gkc+FkZrX4f93pH!$PDjSD|M969mi#f~pS#w=MgWzLo}gZ|og > zv|`egO?yTSdh_betv|Dl?OHbM*|uxr|5i;~_if&}O9MY0JoWI`UiU)u_^W7JDGCCt > z+6kC#!lZ+9{<4*X9NF$Cy(4xk79?3;JI(q7>oz?XG2~0uHX7D>fKlnwZY6j5yhQ9P > z(f*>ubM{fk7g+wJXVG}zWkeT)1rB!KS=w=Dpi(4lVoO@sF$Z8>79m)mcjFzkTmq!1 > z*I9b@Xwec{qOE05R#?f{&Oi42l0;VsMHQ2eKLQz~kV6t#q>)D=nWU0SGTCHGJvtes > zlv7e!rIlA=nWdIYLdla-K-nWjL=|yiACLkuWX}{u3Dl-kVvZ$AOK8e>k#quaq5u+G > zX!VaP69ACFC?T~XL7=20w21=J|9zPdMGJAE38N-W+8{?9n0eMF_Dp)`kB{n<&@Ehy > zQ|UllOn?MPg#vNnT2dsCL@Ufy=SrXe0I<bD&dEw^M_X+&P#HiJ^@}D&ZE>p>^^_z9 > z83)<oTo8ON6m0@PY*CQ4NFaKnQ@2RVMN8Xu%ci&gY!R%VNocs%B<-TBtc?SWLajnw > zdTSL{S82u7aa#f$u)qToT(H3hCy6h@3p3oX!w*9ou}S#eGSIwW`NFI_Arkb<J+%-O > ztuGh1H3flu(Wsoo8u|jQ66R?nRy(yY+m<`Fy!?-b*?9yqynkuiRw+^R^9vWj{^Cj! > z>@3&HMM)#Tv^`F%*NPNL|G*rVgclAp>dCT6w47WXj%{*5ut}IUgDY+Q)-PtO>;@-E > zob5`~O_%K6ElRY)=r5&cLEaQ&_yexfVRMxptT^giR+;Hd5x8=IJ48y~dlP7R;R=wF > z-WE>ZqCnr`S^f{W1b$w40;5BA3l;WhM|I<8J^ps!>P^AQPF><y<BT%)o?{aKN?g40 > z$0MJ-@<J-Syz|dPAHDPfN9E8Ih9Uq28TV|#iWUGUz|KMv%&YW5QUDM@pg<IKkI4za > zn#30KN_9{av37r>1OX&~ME?STOHVDj5unu9zkzJvR;^%&Ku9441Vlhx5+Q>oge3`K > zNlIstFx}&b(i4Mt|E(ZJv4;d&p#)kzMFHg-9=f~-zQc7PgKc18_qGrK(H&$92|xfD > zA|i#uO+`#K1OWLySSR-lDu)WpAuEzq#CsIcPf-{M1%;(64;rKuQ-GBN=?2B}N#Rur > zkRSc-fvJmh<R0cj)6oioCJEF-Ag&1GMci;K5W-O)Zn$7mY$1Up)Wa29P!;!rkN^P8 > zY9JiQpd09h6eHeZ0^<7;8Te?FTa+P#@`;pDfP$t<y75#Mq!U73u}FVNp$skALaPwj > zl2iV}4XQ%QEh6C$T191q|DcM#sM4<;@daPM5e_klQ%vI&XE(|0%`%5+%x5C=n97tU > zGpXs!XhL(E|JkhOHm|A8)(k|Lz8PmT$!Se<b`zc0T<162$xU{?)1BUoCq3tBPj$+3 > zoKX6k6bi>T$RUAgb4!m9YLOmSM4=$AP{LT?))r1k?q-l_*)Jd}5O;uVAPYTAJ4k_} > zEl6%%b>fQ@CJ8n1+2dKUxrgYqV+(<71yP&OU4U4j2?=BfG&Q~H7G4ULEsSDsSsRNI > z4El>!B=l_oHCj@+kgZm*3vy9l8Y!BZg?iY|A6q~X7ZOUrw=IpSXA6tjyd#Bj#B2&v > z{Y6#NqX45K^**a}=|yf4gf@@?kWxUA5@>PIrrypKMlD&lUW3rHDZ!>V)zIF0v7!Ww > zRWEFf|3DNbzzOsGA_17aMFLRRH6@H;6J!`gXZ2FjwNQa@(IT8j_QtWb+yklxg{^GK > zBL&?_raQ5fm2GkBnp+G`3JH}DSWerP)cRw!XA2YK^dd~)(I_j+5gzfTXWi>!H@n(h > zN%OYr-S2`oyvM7aC>OHTg-jrcw;*Kr{GrAvGQ<@U7!yJ^APNL6qzp)4#Ync0gg**5 > zze%VC1A|g1TFEi51v&6YfAK@4q}RO=@h^bG$dE$uHYie%0uZFYES3&9rNgsHL_l1N > zl)@LGH&TFBD4c{1FC=^#g>QsSH@thqkQGVb6ne+eHC2T!j`vuvKS(i!TW~;CnjpnJ > z|0dvL(;?wTE|vft7jj00{18p){jqx$K#zqq@*o%nqKjed3I*7r4IA7mOX=ZWS48<9 > zWsp^YyLM%g*g{n$9^;AgYF<hGX_g$hMdw@*xq)l}(5Ye+ORW|KTL1xmqch{Jq(DE3 > z{E5CARR9n?>Mi+p`cb8j08B4(=1$XU)O}KjE4c7tB(TnYljcCBYjzM>-K&1Rv<1Iz > z<S!nHH`u}+Hn9WN-C`p<*~(tFO7>|r8AbJNG}Vq<;W}JDDFHwZ)kqs)^<a-!Edf%< > zsLmL=p)EMezvEH@e(|!`wTRACX;A`X*OEKW^)o<5-HK=1Qq;#no3U79)S0af|GDU7 > zr?X7;U*8U%9$R#lvjPE20;r;dXqC{?(q0kWQq<5)A>`eVn|8<Fnx43-M+uHJb+$<n > zxUK3tJ*MdAK)Nk%a=)#x$~o*>{%xWbOW|`7@GN;;p>TVkn+vTd?L+n!trju8xcVs> > zftKFyfkaCp!2O!(2DjV@T)MShSC2iw?h#u|0Ri<W1G~5OZmOR=0W~Xb*#E)1<^1k< > z(@g>ky>)D613vJANA}+bZ}`I_etP89SroJ|g_l~nR8a0RJJRC@C@grsN8EsY(s(03 > zhw70jjq*SWMW$}$S;D42;y=E93(kjxzNm@<D@vTlw#>4F|Jd;(69RkO|J#EU*GfPD > z)R)mmv9$7jO2j}YP^w=L8OK1Vht+E4d;qd9$ZH`M6iG3LEsz1oH70WJkNCc)w0_bQ > zFjMqp{EJ2wgcE9Ig+wB9<}_ylmD_9YEyDarISbl7te;7k4^jZtTJT;__(K~^k?4&S > zm-QY4qR|3yN<H`jv4mVA{lg+5;NjGt=D9@bA;c}L*Fne^qBz8=^~61hpr1$p{cVIp > z*g~SY*8~(+L_~oGav<*E);;71k9iQjOvS#W%vJzXIMowAC1E)+;W;&7Iz1sg-BT3i > z6BRC@6JjA0W}y@+;fq|M7J?xcTHzFKVHK9)7mDE-l3^H{AseD${~M;E8-B(<aRM1o > z#K!%cgYk)WB$er`PTOpPuBZnoWK=saon6=hDYT0rGR`dwm_J}vVinF5Fdb+0jajjR > zSS+2>q*T8pjojgkPbop?{DajURRXxfKac{cU4$Vr%PP8vjr~kn1Rb;`)f6DyR<K95 > zRMhQELFeQlx_wl>sTDgc)X=?6E*>Ie{6Zj7MsYdK$UT<gsMd?6$1OxbJ%r*!;LP8= > z+$QKDw9y<PwuLFS1xRg8*c3`yEZr+gz{==MhPA~G*;-CT%N0>YJkDZ62o;({9V%VS > z*p-6ZDaSvl$0oSKKx{%7HKS22<YP?|aqSS}G~>=ZWWq5`|4nRN+%d;#K@Q>kjoPWl > zVR>P9K$kC7#f{|15!zkix#Ua2<nk0=Owwdc+9Z}No~;~`0wh$EsTU#TSrSZviXnw8 > z#R|MQSge%ZhP{{vJ_N?SSVaCq8z2GSX-J+$!4IumDOM%BJkqxfgaZMA1n69K9N<69 > zS=U5C6bPjUc9FeY#9MwCM8FBx#A5;sC5^Ek3vrSUCIKr%(oedVvGAR^ECd;_&sNaU > z`T<biJx%<WMV6tLdO1>_;pH4H#P5|sl_`aM_1^_Ki_ZAZ57E&a0hm)dVyqMh8QjWI > zhUL}R%hJV4A~D1-EE-ZkRjl+->2U?F2?a})*zSSE{|}WI7XSeQ1i%Z1#67ryEs2?q > zU`tEnCboQ-0<@-}MUtRRjgI6>b+%>_sD&JriT&staY70JAOI23A5x5GuA$$q^_me@ > z*G<xAecES}%;bIQ=YH~MN$k@UTq5@bi@d4KbPNe~SV#0gNKBB;(=<$c{0M~l$byoD > zgu=xyhRt_4sASm&ki11JSW<QMPILj!-Z^GV_GgN!=!!~2ezNF`!YEAU5_)xx8u?~| > zo=b9gmbsK4Nk~UWNQW+Y#MKR{Q(VN-wI(7d#Eu$;sR-!<W(1g=1d-Mc7Ud{`T8fn- > zgy*?Qlp3iArO87)DUbjKj&A9fvYwWrDNM~z|45h$P2{LdgkGBFXi@yiQTRe#;!>UN > z5)t~6uK^QSh|?H$VWAT0p{k*wCMu%Jp`tb_qsC#R!Xen8sHDE(rLLi+X6mG3s-tSE > zr*i71b}FQXDya6xJ|UeFELJS4#A2)sa?HhoJ_xK@A!lq8N(|Ghn#8PzhlUzxZux45 > zf|HzPXn`_@te(VBS*m!@>V_U`NO;F{%?L^c58usajACoHzNCw4Yqxr<ut5o(*u=OB > zg_C$ns9@=*L<y(}MY=+ayOyh?#H(F43B1AtyPoTBy6IA|YnIOIka$X$&gT$bg}(gK > zwSH^DDr~TE>%uzh!^-4=LhF)HD8*(;|HT3f#tQ3%-i5}(gvNSokm#zAq+?20D6S5x > zN8BBBv8;(EN5evF%*yP;!0gQ0?9C305dN#b0%>y0DZQ#hzLEr$a!t=>M82lrzVhr% > zbjr>$1(a5Y(mpNF+KJPm#7mHpyf%fG)CtlKMNAkio#Lz2GHuyfX}>Oo=N+xndeWM{ > ztgiWKRsia8h-%%Ms@;<6s4nW>J}TdSYTn|l-<B$2z=+`LZQmm9-x_Y<Chp)a?%hHz > z;70Ca>{DXA2Zs_@SVWey2Fd1*?8!3Y#O|u+#wuO3Tgd*ZgL>RE{)dIMYUr}Y>LTkj > ziO6IWYc=uevz9A$kZ!NiZs{5;|L+p+GoIwMs-%t}EY2$L@)AtVGVk+3uab~glkray > zoyCoA6_%!}&{l1~=7>`gsJ&KgkB%>pkijO%oA{Ch6a|X)riE`B4EqX7Nz_?MkgJ-$ > zsnk*}Ljb8SaLd;2EB8dBzOJv^no5|K#eOwKLHNQXT!hZTr`*aXutjeLTQHS4?*(h{ > z2A_m5ZChI)T|m;5h2+)EnNEGA2N@{bXn;%Zc8wIYj9J#*$KFE;O9ID43jicRS=vP{ > zM1c+?N{YCevfl24l!>iv?2)`pricai%rNX$$rAewUy#Ao5GhzxF(9pltpEW400Bt) > z3Hl^~N`*xY>&O&@NVK#{|L0upJZ6sC!B(4=j2GkOKGH?H#K_gS3SVqy6kPGg{P17i > zhYtt94%by#Arw}%g%jHw9go|6l)>Ze&QstWwMrL`SStl{@FsKek7RHsgYqWND0>bH > zlfItywnQ6T+|&e1=!jpW0Wh4p1DvRBlm5djYZ{vRYe-<`KU9&8H5~mUZ9GzqRN~AD > zeeId9?JdMmKdkMXdhOZzETq^6x$-ETat*r@Ex%6lJ+x;?lmh6*YdA+)OSnM;%17*! > zo>15WQBsgVZ~>x_(0V>?M7VMdvX4?s90vr8H*+SYg%KN-!j0V4T97jubq%il!g{Ln > zKfvGvdh?+8kyPHq|1a<}q+FXf%ky#Q5-;&u5&l{*J+9=E^hld@;~uW!qBP^GbW1z# > zN<Z*PyL3vw^h~pKP2)68>-0_U^yI>HX%wy$?+UH}0E(2%7TnJxXH`Sw2-sPRC*Fe| > zD>c0ZijK6Hj!3~v?ad*@N@-Ez4o%%s>SpNL!Y15~MS0^v5u+Wt>Dz$u2=dLqZAN<d > zoYjn+CM19)8VxU!0y5?fUjGxK`49T~LkW?E4@(<hd*FkV)j&u=@d1E-+;v`GVnLl8 > zV5J8qT+vfX!5`%&R@HUBMKx9<<p-Zd-CP@O@LYNT4iax9VGtTGx<W{;1sAXu>jsEZ > zFU~(qLHZy7|97xQ)kVPOp2fY5g>9pD8{6AkpW|&~25^l*m~_kA1j=v!LU5nlaIXc* > z;f1vNPKjdjiGFe@leYzrcX^{X^M<SGV2VvR*x|Sj5@ZGFxxyFQh*l_oXn|iomu(Hx > zkuBI7gF%Ztqnm+g9(m-Tn88|DUPMG7K`qEviLDoZaRL1`nfKLV*=iXpjDZvw=0HqA > zDUeWbeu)az9DGdytkp1lbwRMK;)kWm0)ZY<-%^2tAG%BtDQrP}%lL^Z1d`oX`AKGc > z6PTYR-$D>&jzBmF%IngF^N!Kpk(XivRx#Gm50Z6WoQ_nU!AI($Nf~724wu*jaCnA? > z%8>0B|I$UImZ2a=<afO&SVzzv8=)`K4T^O{60->GKiFVe*nyy(Ihl9(7BEXBl{uk9 > zZL0;MkN?A*LrR?=pn@czCVNl>A1tM(cczE(d24#7Lu};=wYyFCz5N{4q0hLr1-o>1 > zJ(Sia{1N`N#VXd_Ld7s;ly%2po3a)c3g%oocD9Zr!R^G75=6mtql>qNNC2#r19>8` > zQ&m!xLO=CmGe!Wgi^(sT96yx|O&J}II2Ah(<hCCaWI5u#$z2;vfrAY<P~oA{;7I($ > zim-=z$niR<D^0br_O&h4+o`*?`-9RPRp4mBC}6w6O;+87dNcmf7|~o?+<S%coW=Cb > z|I@iu6tqJ{)umjZo4CDJDPj#{IAXL!0ISC$zxOs01i;0GL^tjPR3YWVKMhYldwL|m > zT5&<fQwDG$!2OWSt~WBu|C?#m-OA=2cvDvpdV11}veGmC1ux!Y@~LFL-jn?U5Wx3x > z3{sbc2@GRkJ+FlW0f426*`l1qhJhdHl_J?E?T|}S?BRr-AjEmuLsy}f^8Er5%*jnO > znMvWjElfZ=_=7tzxw+iK0*KH1dC>q)-Y?8q6u?@EIUkpu+1x$2e7S>KAyq5b=3Y*7 > zMKI>tj}Ddt5FtsxBsFLIEWn6Ov;EDN)%=4bjDnQU&_RH`7F;=FKB+_WAKHrq|D1I# > z)&ZH}FT~yN1REiWH)pg<*{Ll9r*K_hOZX7$9F0cdJ-DsOmPZv%tkG|_eOs^(6jh!N > z0(wHU8WPk4vnPjS{=@WhCG@x6&p-v-?%JgvMB)ClOy@LD>%ULiH2(v{zkk0378FR( > z;K76n7a~mP&>+Nu5*a>RIPfAxj14n(<oMAd$cP?876h=d<VcYdNvcfAa%DxAGhx1z > zS(D~XohWni#QAflMxiZ%`V30+XiTF^hbnDqlPS`uPNhEWX|*WUq$VFy8Rc)JlyU@r > zO=0Elm%Rk`MkZ+YD~YXox9+W7`HPg@f3}i<B|GKoFIRf``eiAB)jza=|G{1yNlJ<Y > zQUq@)@rSmqy|Sb*>rLqI78lOHGPg;R07(KM2@)?vtKjUvQc)JKLThX7FD?k!xl&@Q > zHd?jzZYiiG9Kl>AV>FTBl-sOuy+XY<R)|<}S=(t5XjXFeS1BoI|B;b0Ve(H~N^X(T > z;&02SUkX}*oo#(!sqP2k<Ntin71}h4C_wTGLdzZK0?X_md#wA#GO+CFj-d7GbBrt{ > ze3@%Fg`{BTAjSxipcRmc%dQ~4N>mI$0VlfWGG7)OEwf+xBh52|(4s;Vw4@+~C9(Jm > z?4Jl)BQ1)0Bs$5Vv~01*mUf<u@*mAkOXwuWwA_-*F1`E`%rM0q|C7uz%{&v$G}T;_ > z%{JY96V5od%u>!e?YtAuJoVg@&p!S96HtNZ1nZVABJqbUgOa%Lo(UxJ=bkO3_~#zR > z?kOXQDix^bpiH$SV@1AFgsn<SCHTsgEi%1lFH#dNXi>24aRZ531On1EN^e2J9(N?L > z^&ZI}Z3PKPX$7l{C|DE-tj1g^_8&|sWzil5Ue%G(2{ieq6$zC6XM(T(*%i_*G=Y?h > zNgMs@o?R*Bg0@JNtrQAcx?MF~N^Jq+qGkmu<IrpWi8b3%^ewj0bG!Z479?zOhdwQR > zy{C;x{*i)=gt_&WQcwkY);cmC6`0v+_sWzlf$<YH<t^H&|8k3M_kxy;d8<^du|_$& > zhgh)qa>c)r1-fU`f{;jP(So=znCB#X*|Jd(Q(TlS{M_hE!f_P<;xbA>j&{{!qgB>F > zuwLQd+<RQ{6xJ30S(;K}ZJGKaIt5}&%2uMZMemdoNC3+~1s|O7!VNzh@x&Eh$nM1* > ze;o42C7+yfI2$KT0wg53R<eUImLh>o$ZBVb)1;UiuBUM%`Jj6fNcx~sP>VoGR}@G> > z3RcD=OLomU$5kLQlz5#CEr0oOAX28d=NDU~Fx~AoKtG+!HioC(AW@BP<@POF**$mZ > zG~;9oSrI!vHno_C@3c+);v~6S0(h^i^mDu@SRgic|ETkU!oQg<t$3aHCUrbqkwOAm > z@(o_zvxUiUF9`%3$oG<<042B$bNbUvM!4`g0Gx((1L<HAKA4+QNT7WVF--!nHxMO6 > zpfPEA9t&*+mk%OfH6cvhK!!t>i{Yv)07wn&j8}^Hbn1oxfT4|0BgE;vFoyq-9t-Ct > z5bIIVcvBbyd4A+W<PBsCB_yJaIKw!DXa-7Bg3=n(0vEOks&a9Z;~eQ&M?3bZad*_? > z9{JcuKgRJS)?ro3@K=ymv1(|BJdh$6!a6}xiy>J{WY8G-pIiy8R6MMtW31!IO1`R- > z1wmvcNeRkDYBG_jT$W<|@+w#MOO%kj+bPqc|42g4av+p^<QYxYOI`lQlV2*^DoYm3 > zQy%k>#9XCyLW#*-nQ|el%3U)3l}i>qQdU}`<RyR6%!D8kBCkT=ExGbaDdlZ%egm9B > zE>{(IQU#t+(aKch`ObRA^D6etCqA>H&wlcAEBn0XKm}URe-iYd1`TLJivmZ5((|G7 > zl&C)!T2Y8H)S?pw=tVP1QI398qX_NjMnf7=kCqgpQDG99nv~Lzwp1ws9ne8r$`=3t > zKs9a|sYpu#Q{y$%PCms@O?*1jp%#^<N!@9fkXjL;a)~T0L@7v~@>Hk(M23lK>E+g# > z&L_1o32*#I9{*TYvz`^LX>Ai()!J6K|GpKjW^LRyEm_w)(KSzaO-KOu=C(`LRj+9p > z<zLGbSTh;+aESenV!OmlGBI|qfYp;?e_6G<MHZdtRA=Dg+F8$j7POILYiLPZTGO6( > ztxy^2YQbdGpUP1puDz306(?IS**2yug%fTa$J;dd7N}-4q?5YJMk}$6Eu%&4a+%v) > z=Q{Mc(UtCWKifD~`Nx=}9BeG3i6%;JcT0|i8Y~HWOoM>8m)OE&2n$(WRjT)!?Tzo4 > z^ori^UW9e%)h~C`%irtv_pg-9uX&Ft-ene;OZUa)v<!>hFlm>-;U#Zm8;7^Prt@a& > z<jPb*dQuOQ^us*m@Q6cfQV^e5|HLJBsfw4nVicSBB`bZgi&H$~7T>tTHqLR4c?@G8 > z)A+|YCb5o(+~c3fcqy#qEp3~M65x)Bwk~OM*OHvlo^;8zP5x~{gO+8bzShYkiE?nO > zER!fg8LC88^H#Ln+BNe;uXtUOle?-$Dp^Ux(53F4`P^qdD%a0}7WAME2WSZ!v$6O+ > zteV){NHGmFP>t5of4}75$Fd2sf}Iwn6Aa60PIyf0t?z(T24FB{I>M1PFeWKk;eTY7 > zZ%@iFX9-<vTi@DGhQ{@-dEIL<(aEULOiKdDaggB!pp3>lcCs6k03h;d*w5CIT8b?u > z2qgdj;sUdxzeKWOSNq8p|G9-bwlvSDWzU%r<phzaswP>#s1@yqcNJ5VW>?7jW#p-l > zWi#Rm2O%Ve)Rsv-F5_BTYyx%TB6mV=;fmw<@YVD#w>uu5A~hbBS){4>w+G&!Eo@M@ > zcXN`Rjq65Q`x@mbSNWV)-g1|(e6Glp0>2NU30sLWKBRO^qzrbgUm!fulay?LBeIE` > zmlxp*t}D+?JxT%3+?O(^`9N%b-U;tlAe%s8`3lmCDIg&Nz_U77ypDjdCu9=|h;t>u > zTZVJJH`Vanc|o-P#(g0Gr<*|g{YqvF3E&i`KwyjWUdM%B+Cl*`YXjFNQuW{gAOJv+ > z!pkr@O15l)3|i&H|MN(>h1+GmAX|8yU#%_>&m#f!rXRh6bN-LDz3LY>Zwwg%F9lAq > zY7|El*h$QJ;ft$9!vtw3ky*UsAR8aZ$rp0*i+ud%D<ApGC;s$}yc0b~AN$Xje)Y4@ > zeD71=`QPWh_PY;$=#yXga9)V^P$U!)y}2yfE;lC$QgBwrwy`tCzm!X+Z=z1YHn8j> > z*w6n?i2(KQ&8mbV(83f1gbV1$6(Rscc*b~Kfj3BFMbLtRWTexCMj}Gvh*+YVXbJw_ > z%ni7pFL-A>K0`2`hXcEW6UYE9NZ~AQVG3l>%qnAvlz>9uEdJ0UUC!(wT!DTH;spER > zAV?ue%A$PA|3VBv<9~btFP3l$wqPKp;0bRb3R}tsQ6*Lu@CS8c1+Nf1Sc0pjgyg_V > ztaR<=+RzQ9iw)mU4(G760*O+J#`RDJbaDz|rXcWILm+N}1A4?U2<aX;z+mo13jl9L > zB83#TK=KlUXgJ07z9n7$MGLg#dq4*=`XW*S0NWbTfx?JHOfT?O#OtI$0hAC4ZSe23 > z;93w7Ql{-<wys3%0dyh{H6X<mkpLE}L<0V1HTn<~0iZSPV*+q05f@2T9<XE7#qA)) > z71U*E@TwWvWsz(FJ(9qaCh=qDWeZq=6eZDI>;qri0{|k%^b`OI#(*s@NRz&2OK#z6 > zM5zsA|7MYH0X^*Em&gw4O6F4V5nG5R3Tj~*J#h;_5eQ=A9=Jee+~8X#jRQbLkU)kl > zqDCMJV-mpvA`1y4PX<zoW>z9{AzZ;B{efWYAt6<RA-RAMI}(v@0Sbb~)nKh}X3Zet > zkPdf}C*4XXdlD#vvX8O^EtKHl?js{L1V0Eu3jFE^v5Yao0*Fe79dtuLi~}w1W7t}u > zG9*q%Tp<8l;W_G|e+1%-$|4D>@)ErWS8O2wS|Km$0eR#NEw-QtpVD|9Py>%{J!AwS > zS^+$^fa~@mLKdPaV<!Qs(jJNjL+V0G;^J<ct$tpk6*NHtw!kYR;)1jeGcm$}3PYD5 > z|KunIWC|caEel95Tj4jZ5DL`N0b$Wb-h&G`p)AIu6(&Ld0z|7yAp)X60yx1iJ`)M& > zZ}JoZ1C!u59OD*}U<@{a4AM@kekeP%2sEZZ?0lj!&J803BMP)CE=VEs)}lBHt}sv} > zA_Bk`z;igp^ESBgCTwsFB%wQ{ayqxj28+js_|9(7XiADp4HfApg;GEV6p#qiKo4|} > z>@aG;B0#!M0zM5a4pWpUpy#TD43YqaBw$j8hC(l*DKByZ#vvmSMggSeY8s;zB0vd% > zMln<+0x*;|TwwxuW?L}EAU<?lLNp`a=y+OzEnH!9D)S$#GizLd44`FPIy7Za|L}P@ > zCR&hy6*%pCvT2rFL0SZ&TRzP!F9ci?(IHeNLpPLLq(DN=QVIY7>WTvieAGN(iD;1X > z9%|+rfss?#=Xs=HUKT)VT;WjefkJ%-THKBmlHe9Dk^;uTXSk*~q~;fRr4+V6AB)9- > z{z`2;F%UV!9wa~#tVmg^1<^#sI=BY_N+FqwsYIh7E(GsJxs(F{0PiM%@(xTO++hl6 > zPHl?CeCk0`{sCb&;9P)|Yre}`B;^;lrXhb-0vPW<&ISpjpjP~&DI23zUo}<*f{03` > zCVfxVfG=>+PyE8q{I(BV$M0LuuUy5|T*I~e$aTh$Y+T<pUD;J$;T2u$|8-sI^<Lu@ > zU%gdbv4SY)<KbF?Ha<qIa!CrREku|t%n~+=n35d2X)=e+I|6oK2SNcNfJ@ecf#BoW > zx@a-L!yeodBMvqxH#R;<LkpsS3pPPywPTwcq70@dIrDH?P*!E1!&!LLV8QJ>5NkeY > z!!M)5Ms}xXR|i9iwl0*!DE)yiUXTT`@)ra)IGQInxF7<MKmqd2WhHhw6V5FvqZZC% > z8YxdG3Zx+h!#jW@I}N1EjL84ULNJ9eLu?@mU~xJwrr365i=y^!E5bL-b07*NLmrMn > zR5A&;Kwx!fX(`1n3>R@X=RKz&Ijkol{DEYFFfas2t1R~|x^*Jz{|pTk@JezLK}*+k > zD<?otS9MnxPZU%E=z&t&pkjfP=FlPv{Ayh`>6<zfmwHrI03#5OhG<}yVpb(uCLjr} > zbCrC>7Aoc+FmF|5C11uxtZsK!6`*-H6=QT}M<)Uib$7w+ffR(-^&nSPT95r`SDCDr > zQl>!fzN8E`AbDkF0VuIp!J--mLRJ5xA(ydONP!&_AP52g0cLiQhBt2JC0q#QXx^q) > zPUs@1mt4GYTm+A3MqvyvuV9I(Vfbri{Gk*AH~<2mR{qPB4sjV1qG7g)Xlh|Ycy*Pc > z^<A_LcC-{2EenzCpn=;r5gSyHq{o(6q*B;rZ5BcvbHx?V{{~XZQXp>k8XMzUMV54V > z$tJh;Z%&SNvsjDqXp6hpi(7Xhq+pADqhgnn^q5lYbfXB%O)nu%IzRR{azl*$fsAjX > zJ8>g3Q$z{Eqk+=HET<AYaN}COXo1`VKIG#&_F@VuR*em007;>WG&c~DGlmk`7D|Ua > z%vcD|Qg+^AF?EP9<^p?Nhz&F_H&RqHAcJ`_0+pu_KbBKF8)A;@cxwf@k!_?20Z@Eu > z(?@n^k=elwl%RmNXFkWJGqrFq1JN=jA_4YeJBt>KP-hd?m?hR$VsCURM?;daY6@nS > zEK)Rvx`jB9Cu-+rZQSM-1i+c2c5|EuNXq$S&bT1D|4;+>ry$OfNcq-&>6T>nbL9H- > ztBwncznGv48ckBypc7i5?a=j7V+!cjATZ^1swhIaR9B+lb*}CfFj}l+xKR+bf<Fcc > zlHj40<rX%e+Dzk^ScP_h_=~nwRt_}~`DImEdUNhkI{AWyTH%d0I-}Fg9=<0-#jXqx > zV50A#VQ`11BYKD^`eJ;vARr?FB;bAVYXihZb_pd4UUUj#6*2-~1FDsllrRCdfD%ik > zTF%<(Ml=B)I#X`pP#G94qH!RgR3F9V78vgrwBT>ZfD+392t;{PFNq8SKmh`eEmkN2 > z{5qE2!CFLV0**$IBp_Cs=5Nf}gYWRH_ZOxG|AGsE009D^ELY78wq^DV8>c+w=ZL2< > zR%k^J33+2_6F?{RT-z1ax@68qT;Kv%D!MT~JG8N*AjrUA{Db!t>6~Ki_pX?o=2c&x > z+g|zAxug4B^Od^$wYsHyU!~&uv>Upu8@#>yx~Dt5v)jC}8@<V!#73*C?1{;^8%^39 > z$5@O1*i5(3?5gtnqvGVfEsDMqFwG)Nwz`bW!mXROpcLvG%naP4!UQs|>I~DUKmT)z > z723itoJ$ZI!#A9DHQa{ZMCr;4)$YnqRxQ;=9Ki&l9k7kX0U(bt%}ch6O3lR5gh`of > z9KH+;vTXQ#&-zSIJl!Nrn2twFvQ=+1{|hI%g2R`5!!?}ApPWE}&D!c{o|uZhK&#5h > ztW270$=obY!tKho?Y30Q%Xf>+{JFR$T%c#k$)ViMfwIltT+Zjv)mSaXV;oE*jg#(t > zmrQNI*6Y$BluK$H&jG#iKyA?FOYZI*>v{>%{EN{eo#?uY(q;9=BkRyBy-SQqz`*y? > z_k6tUJjnfAzHob$k_)r6b+brUyvsYixBJz<yVYm?yw$tbVcogiRn~8P*JFLWaUIxu > z-PV2G)rZ~4bp1=XT+2()`tF;a#;n=fP1?oWzUOb+p`F^ve7*tP+85T_y^Os2+f3Rl > zo*+VQt?bJb+)JX9)jt;vLl@5m|9Z~n9YN!q-tS%I;t;#E>C`aYuDry=H$Bf6ZE*CQ > z-+A2A;;sWfz26tC|M>ix{vE{mM3Ml$;SHza&&0?tY}J(<-#>mp?H%Mt-qwaMOrrDN > z7#<^1UI=wAx01ajd~QqFy`E|z-vr0t@dVqqOy$=E|EB6?;rrTosmx}c%3glVVP3y8 > zDa-YYKS?g2O|HU8p6cnm>aSi7V{X9=%pQDUQCKBd^3K$T=I=shuT&)GN$I}=-ru2w > zYPhXRLbesq3z~$u?Da0hGuz=k-SY0?T0#BNv+K_P{M6;E&x7VUFHO`X{?c_E(Q(|; > zOV82y0uYBTz82HR`K!;n|HRZepTF!u?iYO^2K%yl57s(9{eu11g<bWFz4d9G*kRw; > zU!T{f-t}1@_g7!{Z-4h?pY~(F_IV#8kW3?-hn7@QNhTtnvw}z?LM1`2r_`&<oeKGd > zs*}tbsq{~m)x9RxL%$uu_#XoK1yBH0rL3)k@REQrErK9wfun(kAW)FQ0#PsPgtmmI > zJHHtKK)W5I2q4HL@#XKI!Gi{W{o9fx!6sY&ez^hwgd!P)2>snspd}(H0QM5z+fwge > z#3>XLOsQ4ypF)f-7m|8;PXd4hq)h&^CjkgfoiSlDbm*_3Jt;sGlmd!Tqk=6S0hDxU > zkEF?8D1A<`*OH8s|AYq>0>Jpoo-MQO&Yms%D@hWJV`0jjOSi7wyLj{J-OIPH-@kwZ > z3m#0ku;Igq6Dvj>cd_HgkRwZ;Ou4e<%a}9kZQSx;!F&H)NeKBlVQ0|4HjK)vS}y3h > zAz>N?T-sn~*-h!jW@<3xJr1L8-|Z}VH*SNrM(cddTkdY*!vD4UYLCpOfA?%9aaFWc > z?$5T6c)!)|VD^Tj<wkE_&=nU=eAgStI6tFwf#zLNXPRA!g7h%iGu=*GTzAia(0TUH > zMo9Q`&jdiU1X@9#abX}AU#(}*J#Iu{9v4fwBSlUNu@agDB<6OYha+7l(?9M=U}A#z > zDA3@85pJkc|AiMyqEl5VBGg!K|D@H5Ek>TTQvwOVrP-5EMk(c#R90!_m3s||<(6D_ > z>E)MThDljtzbL@cFQrJKQ9}SA;8RFaFbP1ASOr&?LUp$EOPoo}na~uhXcB-D-u<G) > znXN<-(H2{{#0gY$*&+o)tt1hFJy^*}B36M8S}30Ykb~T$NmQy_RSikO)t{tjco3=y > zNHLxg3AFhpfvFO7OPp;YkpQNk202{<NnE7fFI#l<%cBvI5!^4WC_6=~cOl#AOSbJ< > zDYh^fdS)q#aw;gHE^+Z}nwSDQZlNPJw1qGKplA@hwL~GVr(}iYk}HZPR7yt*x+LpB > zblE~~|7b~IS_Y(bX{1C1^%(qXrb!?ms!LOh5mBP9V%$nZWO1P}w!es+Z$bz23vxnR > zj6v5e4@Vqly2M)i=d&Cqq%urHVrtgUYWdvOTSRIp=Fvzet@P4NXJ&NMP)9BG)KoV; > z)@KT^gPkjHf!2xy6u$aT6s_!b4+X0+$J}Y=?u3!7pKX8yEBD}6qt?IJ(QjL=6c?~g > ztt3ziDM{E8g^35jjU5-_)lD>ZW0x(R-~wsaN_IKj!-a<%0!NooWK4owc?L~4+dsDC > zRNYslgUw&$2XS3!ik;~xo@i6uGWSSYKgh}$No?_4z!R`Dh7w$qRL~Y=@KoP&S#RC- > z{{*|^_fHv`_*2RR<!Roy^_%-s)mO}q{|_l!j8y&O_Qjq=XJsHEMHHm$HbwKw;Z~D9 > zq;T>}gV#&K{F?VStZ8BloFGGvAV;=>{DK4UvBd^llpBM%LIG0|+k@=1oC&N(f1~N& > z|60-#51L0oH98;{6nL8kE(n6C+d%fh(xA?SL`xw_OOnzO5_>SoN!-#`x8xE;unh5t > zCrctiiWn3n3eky1gklqyNW>~4k&0J*A{M2{#4cKqi(lNL7_~^oD2~yLWt?Ify_iKb > zx>1d9WFr{c2**0g@r-s1BN@kX$2ICPj(^M}9s3B#K?ahLf}Eow@rcMq9`ccf|BU1! > z{U}LBLUNLY#AG5ZSxHJBgpV#s0;2kby7%4Y7ygg}IY>5zXBlJ@#F9V~0ERIRf<+4D > zxXKh(5fOJx%uE|olR?~|1XUhn6A?&)ELCMNG<8G^A5@4jPx34u#zvZ}M4b|x>5naJ > z3U#CaU@cb$OJxesBMBfKLYk>Bh1g;g;xt%MsA$Y_8nK+oWSAQi5CyHpWD9XpQ$+a5 > zj$e>MRYUwm0{UstC}4_XZGqP>CK1bXdTV3h+GRxSfdEW#!JEIj*+wUJsEs0^9@E)V > z1iCp2PL1@MPm~BLcT%=nOhH+$SgD~15-^3_qHtT7!aWjDkFv1BER%|<|3L~D1qsw6 > zq%o-{R#GAbTclAtQB+8!tO}N`R`I94NGepdTGcAHrCw5aDomxCl>n60G*W0s5J8y` > zpz#n%J84UkgjBV@`t`4X4QyqeCfLFn_OOUG?A5TN0D3q`WnNQ`aL9&`TX=3Row)_t > z<f6E<jl)5@W9(Gg@VJ`1&pu(BjxE~YBPkfmATB(coqmduQdv*|tBu4zT)QACNMRFX > z5a%s6AdZe1<e&hM)j_^yoLx;SxGiC>T*&4d325zk_IXf)V&hE$`N((J`2`S+a~cWo > z(j^T_PD$+XAN7dVAf&|(+CU3HE?BBmpYb4QBXq&3{I3=wa>WLs|4^^X>GvMAwTm5Y > z6tM94!xj@*Z+mUS9|d^l9`+DtOTwcY+yc}eKv*JWv2)o<bxgV#(GEl)S4ytVrAAf- > z4f|9aHkj0CeEvWJMn<f;54Hjcu_=gcorI7a+Tut_>fsOhn%E{g`N>d58eyYM<tkhG > z%8ntE5@-<-aBX-X(HYSaTiG!@O-oOS+Z3Qi8j+HsDO~p;g(T!f7gucaU)}tsm@#%O > zYvO6QB(gI%y_qMO>WR#FLFbowiZNaq8aZHw$)59+7q7_6VMY1HP}9YVh-`W;SaI|{ > z64r(QAi!5QL0nTViY7I=^cS|67#B`zXRd@gBC)o|M(Tm<|C7RO45J{!794u(vOpEo > zT!W58R$7JxoVr#C@TaC%C9qCIt6$f2MFKPg)`<Bzs%pO0J>psyn9VaL2J;SBEJqd$ > z>q!dMHl0tE(CKn3!4}@07`R#G>A2$Q9#P2PCyQJbBn53Pm~sn|vCQy>JN)5!nKHyD > zPVtI2d9nYPK=xWOfzvL@JGI-b<Tivox7pY{WPlHI4ANQsE(m}c_lIbR))srjNSz@s > z4r>98kd}|bAh!rHg=H0z3ks;Uq}bqt<tCw&o4h&IAsvT%LsXONSAyd7q4E?&^kE{Q > zB)_0weSUhDbYjQlt^i=#3sMHoLGE;JKtewPSl6B8|4&6R|JqxAF@eu6C`T!G5E*== > z+1W-&bCJ-<=CCt`B(Os78gY4B{+8m$*@Fw^xy1y0L?q|6guUYYM;jWX`JY!M^BQ-} > zdtD;*#Hsrq9B@$^#!KPapZ&T(FI5k7|FH^(p0o~{Af&lN7tqwmzY8pgau?_0@xBJD > z^BK!-jJ!i7`Eai>(UK!OnMqD+a{J%Er1!_~N%D8U{OCWw`Zt+=_OBoO=6ApR;opAx > zztJGt7k~Zr|9<|@-~aaaf7~a4;WvN*cz^(yfCi|51=xV`=SXzrLqV_wrjrn)AWINq > z8d^Xn<AwzJ(?v(2OlwsNsDu^NVogPo029~_|59)ddvYzMAOUqI1qRh91p*e+@+b9% > zf>g0*l2U^&;tmQBJgk6FJ?McjwM2Rs0bS=3q>unW5CP@U3PZRBtguoR7)zuhO{{<| > z0zoVUG#Btz5a?oe%Jf#EM=NN^L~TGTlCWY20W*x&El;(AU*S?qf+?GzF$N)pc*ui8 > zu?Z5z2`BXnZgmpY_GxMqc2JlAz7i68=MZo>Z;WOr;c^gU@Gvdq60HyczVZ-gh7n3Y > z5XQDv>K1kgfB?iYa$Qsk45L>8;D{5$CbD=<N^pq=@r8mIN&yfA&z6KSVJMmQ569*Y > zTF`|UlZvaTC*4Id|Db(#6*O$2a7<=#|JHbo*!WixmyO)mjoyeQTH+lgQ4+853sa~d > z>VyzRf){ClS=ez4ouN)?;STHM8`8lP(Sb_z!D7gvBjqt0|M(rTqZ8#ZJnpzQ?o}H* > zViyTQVc7L{{74@F;x-DYA=I``u9A@vxf~p+kop4@D>fmJ^N{}lF9w&6NOF-n(p>2X > zbnW0Eo{^IC(U5<mPF`bh6WKm{f;czXj>v&zM`j-8ctWTnKH!05weWf;7mrLKaJSG3 > zwZKCMQ3_ij7dTWTM`mzGqGZ^If8Y3(U>TNDvW;R{mS%aDfk7r@#6*+fj9Imae(`6Z > zCKy4)mV{xKg@KoN5o-Ye1PIV3|1H@tbD0-z*-n5Vmvo_*hoP90RG5r}hJ&dWh)I}n > z85T!@T|HB9K=U)xXnD_wmY^A$qRAIzIhv+<nqrA@5Lp@sX?v%^k%D=0$AJ+mN0j(z > z7_C`+wh0>Z!5q)C9tmQQwD~!Jp*GI(HzN5w(?@xB@jSkHLc(D-Vq<&mD4V%4Jg-S4 > z)Tw&288$YAobKhC;rVpANnzh<dn6ej;3=LC$rLxVk2;i<Qh9wq#DEWIpZm9f`sslA > z$)EkVpZ5u%{TZPCNq_`;paYtqbXA}TD4_qzp!oTq2r8ioN}(2NpcRUt7uui@>Y(hm > zpj(7Uc(fM*SS??qmyGG5{~S7@8rq^F8liPGqJbo%Z%IeqhoWf|qB+`rI(jDx+C-X( > zlX#VIUHO@)d89~cmZh1bOxmOkx1`Mxo9bDeeqmfr`f$c&rRu4reGzh8%AD@_rC-`U > z<2jqvNF>#_l}l2jP<o|q`lg4~rf@o^bV@a9>7k2}qI{vJdRnJjGMPJyM19JmUD_9e > z`l18)7nz9`2PZVtIB<R%sgmkfayqG&dZ||OG+KI|{1|dlnw!8`Kw!F&RBD}`dYhOk > zB_f%d$vLX0N~N`_BXwb%>=~trHlD9)o4G2h`?#IF3aU=Bobd^iT&XYlr=l>rq8X~J > z8=9=nimcGuq0-8t|IF&5&niV;Nv+qat<lP@)4HtN3a;HMuHV|M<7%!DN>{vcEi9)1 > z0YEKwQLT|_qFJ;jwF0kog%f60Z%O2s<m#;e`>z7~ppLmRGkT&rxt1njuyvIL7Xzqq > zR1goLR7P}43CoZ^N=3s66lLMCZe$d~^On%47I)>S>|>;<nzAapC6&6eF8i{I(HO?J > z1*^fDmvyVYdJtUDaX_&v3IGHlo3mS>06|bcPv@Qgu(LXL7x&N_WU&dcp>!}CB?Z9+ > zvILS+U?U;BJT}6IG&u!BsBY|a695nZ001uvf;BEe1_00*zn}>jv;|~qgbDc%Q?L|n > zW3?x%w70PZ|FRUc&H-&@3lcs=wzYz`*`)w$+qPi=w;RF*WIF|0hd%3Zm1$~yO9rV^ > zySbbT87<qnpc}e$p?!!J7Nt9{c;N<Iux$rH3ZD`rWPvVN@mh1Cx~v<orQ1boCc2iP > z5N^N;9(O*S1~cc91x3YMAxdU2!><XkLNCKl5iv8m`z*;)W_acqZt#R?_PZX}Y7y~K > z7MoS1Ffw70K+hX4(n~~ShAhg0G0dyEd9|6+m~fu?xx9P7p=-bRyT4PbB^yMWo7F>i > zS2&^(Cl{D7wR#U+vzpj3c%u3s{)ANtyujJXA0}iQa$76vHe8yZ1<4hd{7V?8vA_md > z29gkY|Ljl@5`3FA!E2#GN<}vT!dVcShXl3Y3e>YaWZQd5J40@8vbG1ov`NAF@fNXx > zJ?+6dwhD=K8^gc3Ak<S2I1E4;3;?~SIX*mN9`c=*tCcvpruBKQ)+)x;TE^jut^te2 > z13Sj$%EoEz#^?v4Yi!13e8zCB#&~?kdQ8V{%*Su+$8k&+>ub6kge=dZ9}gA)ZNLQ( > zLuXy2G9+^#Lk6TSac`v1JjWJOki15x8&%na6#*A^-8;v249IzG%8B%7VR0aCRyd^4 > zDwzxxFar^#_-9g3P>GoYV*p%FxK`4ZF%KpMd`QW;Cme=GGMii$3Xl_<plV)p%gzYR > z|GB)&WYEjE^sp@x%)_i{#1_U>L9!&9R}HanAzaQOtiR@*&XcMomi0k#2pkg928z;S > zp`ie`&>^YRJPff+O8gITH%~?Pvqmc){{c|{5Mob^IyJNt_QlWd?9QFBd=KJx;+3B2 > z>=zD(5L5^R)B<B$aKf#ZU0aYnqOr7))-mig9fcEmu(LRA^1*h2VqvjhWN`;3&2mzZ > zTmly<Z*dzd3>y|E1%(p?Z=-IS^U_j44@)FUJOiKdnU%$QrWZZcn~Tm=ebr7%x`k{{ > zR6$g;92$4l5j1-zJA)B6MZVnvOdX3UiCNZmNoh?qX=|nsW{1^>xi6%IEj9%e|AxYA > znx=1NR251<ftScl<u(vnpep)?O(F#?v8*rbv_#1U7OL!0w@|%reZ66!*>eF_iT!QT > zluP5}*laCLfw`zc%9->_&UW3}mul6n9owO)nzF@@O`A5y^vWSC8u!o!iHE#oiZDL{ > zl3Dk38D}R8Y#ZVO6UjZEwQZc8MFF3MV6-ROd0`wJwwy0l5PfGsk8?UinKLBj&L?+# > zz7P`C&2m@#9tknYf5TomiB&$WW1rzv39&##l3AW(HXQJCWNLgEblwMT9G&$Z4uoIr > zeGj!MePT1LUre81ajbn@$`oG57Czw@{>K%L;T*2viHfqPyx|_+;iR15|0Pc1A#UO+ > zj^fs;ec-!9AGNUaMsC_PXfD!49Fu7i8+No*CLJ}E|M7x);)raW5Cc*z%o5F8cvD67 > zVHwNfs2t@YUVk~WC{~3muEtzK4s^t_8}49L+QF|o8NPAoX2iHw4-3qb<6lIy1T4X^ > z6&Q?&d=*MS1_co~vB4m77Z&tpu)nYb?;Augb0%kgOQMYxBhv~er8BsgePj`hsy#HU > zZQYdarm$V<nC^}KdkaxOES1&i?32R=AtI{<AFef%4IGO%aU+$I00CeX8`5#-_79vO > zO9TWOHQaQu2n3J%GN8Uh5<LZ)5CN}l(V31N8O-af-Vm>zK?fX$|4?kirPC0FV~-h^ > z!Uu^#?Y$e?;v10rm+)4%9FlllvFmPO3nLK#Kbr(M!s*F01p%4s2qDB22Eox0HpiFX > zJtsLw4eloL8Wv)eP7Rc53gPg3><E9Fm!9wp4{;JsMcgu`7@M;(CGprAqy0*;4Uf7N > zM6bbZqJx?j-PNKvYp576sL~+`OoZC<YqDaq@EafVRtEDjU-M6<n!6{R#s{0jNs|>^ > z@6&m-+-clFFP-Q)+`LMrKX03v9yZSDssMyQd%qKps<z6hp&G>Qb$TK}^hq!DIV+N{ > zNt0x1^c~+4yK)aT<~ChieNjDdQm*2AJmo7s;&OlDb07C~fA<tB$NzS}_iyj^Z4c#e > z-}iv;_kpjh3o1uAnn{J3_{5y?PwuZzUiX48`F}6@J$je%s_`N3_$N>D2)ll$d-#w( > zq<S^!Hh=opNb{(_`hlhTNOP*2%KB3(7-zrrT>|@FQn>@~#cQ9fvj6)I5B$RKa0|cr > zG>RB?nV1JV{F#4$mFfI#Sy-rDvIr;XQqBC>?_{o@{oL;*Qx>ZwZ|vIH@kNHKLC>l4 > z&hyV%tKFae6wmac%B#3P{<}Jy?{A&MF{<m27c;CE0CDf%Kz{@O6*Tw{z(0Ei8@96r > zu~kG{4ecdJVDOj1g#R{n^awH}$d4UMCOoN8WXgdoN19w2lmDblm@{qG#EFuo&YL`Q > zs_Y5$XUm;JeI7NMlxWbULti3o3iYYdsZpuA#9Gwq)~a2zGW9BQ;6kZiotiDHH7r=P > zSJk%V3U}?<w{_XtrF&OzUcPDf`t9rXui(IO>yi~((xuLi2o=+OEVilK#drZ<CXCte > z;LU~SYW|#AbZ5YkJ)1u1STbptmG@FMs(SV8kceB_t4I;Iy<fbOB<X0I_Hg3GjUPvz > zT={b5&7D7o9$or$>ea1Zr!5@&cJAH1e+M64{CM)^t!rO?Zwsk|(Gj+{UhqBf`R484 > z$8WzO!RhmFC1GW?`arU03wMf>Pcnf9T5vi402IqHDgPuB?jc(ylBgnvFbXa|4?hGk > zL=i_MvBc-lGciRKS7fn87hg;WMXCm}MGD2HU?sufPO3+WBof#JK9TI1;>RFYD-ekY > z*sIE)jmnTBg8h1v0Eh&bbPYx=x0LKjB@%eg9x1G3l13>KNTSS{%J_0hk#0#~6emeR > zV9Em9X=w@otdzo*;npLHj31>aXrnQH+`=_6gShgxEs{{6l-1yr^A|dK^XSez_e`Kq > zC;us93Ly=$jkn!$+pV`>eshQ;E?<Q;R#|7ARYqEG#Wh!5cdc%r0Xu5&y>_H%NT7Q* > z7{#Dpe>JET_QX6=NQ9haNQ!6^l5dM3#Vm6_A^%G$1GxPFwMT+hl;t(u5FIN;P=xN` > z;#mnK{l}Ih&i&<F{!9WFxcC13=N1K!C};v8-fa&dE`o%2!hyU^=$=+4+;3ur$k1e< > zHpJByT?P*#R|59b3(^*m4GK2kfeSwPV;vX%2i_>&1vE@oTZJe?h%DkT#?V*;`e&hG > z<7{ZlMl-r-qE9<I=%qn>8tJH=ZaQhEmF{qAtD&x1Yp=To8*8zzCY$TA%`RK&u+ygc > z?4%I;Wr|Uv=(HApZc)PSjUs60moouKK*@UIW<wJ+_5NoIB!INVNgO3XGe?v-*<!$y > ze&IxmPaj*g@n1@r*{Y3xaV2z<xEMpWt^d__yKS~tcO7ln6DvKz1%E-^Nh(qoNp;dq > z7i-lB*r_prhPJ>60CGmBVw8S9kAT^hq>$VrQYz0^^I!6IvUE2^&wYDeoSfKlkCNcK > zc;1jtKzZgX1$}diQ(8!|Q+JMtp;a^TSzZ7C2VekoWh(&|(0~U-pm}^1vk4%f9`+zb > z68r%#Io-n+QHa?;S|Ndk6yzQaY=u1pH4qL^Yzq^*#}tqdLNd&(APp(S0`Ro2R>4pL > z?C8jQ?xBEU-GX4)0+uZZX0s2LNqPxH;&UuGM0BNaWD{V;p|(&Dj|K6ElCg&ZG%*G) > zkl~60xfg>tAc_fY;98@KnFOe~AO9b+0$${ZnF8uT22T9qi@jrD1?5O1;D{xQTkPT; > z!x%FzkWoFNKuHoHHkOW92tyar&_(!1sw7s@l9$BfatN47PIl6h1%w-poOBW1xMD)T > z*a8_s;tC1MCvKH;1(*b;H(S&xZaE^IBKdeFe>hG8N%$qPb}6S%O=b#Pz>O4a=@nKM > zvsp>-MOQvK&DPW|C9Qy_%A&xITX2C`S^1sn67ruK@y&Rn`-c>8=n*|30RTy8+-hv0 > zgkH8pJ!c}+-IQR9O(4^FLBZzej_D8e`GlNNYR5T4qRverApl9R8&RY%!~Q+2H@ZQU > zs#sM~YIfA4AC=W6K{`^Bmj4t#3%msf#!)CBP_c9OP-C+ckf2*^V5J@*m@DL1C?wS4 > z78H<Ez*?5U0%eRq?GZ>(@pzA0NI?o)5F)}{7Bg26tU<VeC%{bFNr8B<suN6zz2f;n > zcCd7j8T2X>Y3M;7ZeeCXfLIh-Q78@^sTEgjP=j1Cg%=JaUJP+-%oadJCtZ<QVx5vx > zxA0Yvu+?Nz@C95GB3BQ>OoN6^r-ppyEeq)^l0B=<>%P`H*p+s)qFvkUV#hkvlJ>N! > zHEn8H3tL9Q_O-Lc?P_%!Ti42Vx3~SRZ+Y8V;@VcY8wq8l>`|wIlwk|4kOX{KsjRrI > zVw4D|$1Ma<i{~wOaQ{=VZg2ic5YmCEC*3n$ZwjRmCzV2Tw^?sXtZ5K?%@sbt9j<YE > zyWisGmYFjvNqsdk1yXz}0aVP?BEyN4|M()iq?9Nr1e&_Nty3)j>E-C?>7;v{Q<V8i > z?>#b1UnSs+U+;xUg9iuUg-V#6mHNjvi4p}&5k)u`#SQ*UwW>Z+a;rZEa*)9zQXv=F > z$Vbi&QYR3J37!yT5{QE2lBxh4dT<Lt_2C4$Ff;XF@Po+`Dq!){&FK1(3si(xnd_y1 > zq1LM*u8=^08e$7Jx2j$eWKex|gyc-#!VY+jnJp4vUxPIB9%aBFjaD|!Kl|gXDBGhz > z+WZT-GN_Ofy#MQ?Q3k~)gZZu#bOXWcF##~g@r#0X#lLFtW(5SOv2jH6FWMYqeNm9A > zNU#S&>?M*4{ll{p;n^f>%V%O2+t@xncCwWn<WOpXZkxD;Mz}CugODO{5%{j;E;33k > z{74gp`Y<Ui^~(fIBEBHmpttMk5hX;gC8Y@5M2WJ!M0<o2Xad0EeDh8K0ItT;Tz0Nd > znSdbV`@nch01!AetuIN!Pc3P;V~|&HeY>Q=nNsLLPR33I009B_V%F%~>zHy43UPvp > z(|fk(?(=mV0sgj>0?<d%H)T`25R(FyrqJ;Olvv^g21%)KjH-%gw8sr0a=?|obR;I7 > z=}(8c|Nj<v4-S-?KZnj(eRR~DDt7h0oJE<PGYAr|cd&{c(P|FWZnL$Dq(KoY&fCKt > z_44dFWkA#+Qq>N`k3CH6es>Jp$3FObP}<df7n0ZWULUh-J@IpA$loK+Ua^;guZ_H! > z-*X1Zooz_i9Kmnsk2`w)>bJkaExmt5pL*1vKEJrFAL(7c`q|6g^tV^N?Q_ri*S8+` > zzJD8e3B0+Gh_y>o$&yrp2bsh$e=^N~KJT^W6WT9-HPt5!YIgq;(@-D!p7B0ckm9$8 > zbKm(wv;K~}BBt+A(tf$YAD(>kd8gCe{`ZGf{qetl{;z22e@%#z)0Y)FqP;Vq0=&Nh > zRR6%_*t53dj|LnLx#PQpF+lEcy!uEgUc<a!`z-qFzYhdKF44abB*79K54dTL<ns^} > zG(quT!RL^R^rMXzOd=XYHvP#lqT8{Jy1^Fo!5=J-5d^{^BtlJ^JRz}&2!uQdG{CfD > z!n~@YUtvPQQ;sLZBN`HmDLgv_fjk7Pj=XyjFC-4R>npOmLL_9sEqp*}DUL@=yR^%~ > zxQoIHL?biYES#Z;M+!ETxG~)WzCY}}K*T;k97I9%Lqn9lLoCE$BSb}X#6*O|+Ka^B > zLqtimL`p0~<TAgYQNHuD!JE*&=-b3mG>cN4jV#H;wdua^OGQx(MSufEpl~KrbpOTg > zlR;ZN#TkhXRGh#WIikn7vHQs=-{>($EW%?%#^?~jWMsx>9JT^vLfVMNFH8<iGZ8VI > z4Q#}YU$MsjS-@9g4s5g;Z4|pbqr&HSL(H2%FYGlA+(V9X#(0!R3Yo`xw8x6V3Q;1C > z(Xklg_{3?n!=<<gHQC2k!5tV24=4GO006oGY?KjE$OK?Way&k;aE=$$KJkJhrwE=+ > zC_nFL49>ZTjl_xqiAJG2m6SBPR$0b-bjc5F$(NML`x6cfthxaK$X!Fcbrd@&q#^cW > z!=DtU`hY6w*aRQhf`phDj4&d|TS^5W$W1UWEIdQL!<g+@7L@52xhu-5RR6=r6A&(B > zkdE=Aj*!Z3{D-3ymx1^XG$DifILje16SrieGf9HC+(J4#k~Wa6mheb}AOpYwfGI%B > z<H5^|5X|;@IO&PYglJ4OqBEl^gGt+jS^!J{csVIRuz$FoqiV-I)D6$enn%3E?ZZUU > zJk3co&DCr~)ojhuw2Ihs%}Pv7*rZL?yv^6F&E3S!R_w1nYzfDyo*5j7R!A>htVNKR > zr<)ito%F}=8!`Ecg6H7^?1?2DvWEd=jO%2G0n8U486VLhuY*{YZqgp^`xkUF4E3bS > zeR(Bknu*j1$?=R%T;z#j5~l$}NSydiXv$BG;RHZS%MijFT=IyC@&8ZBD1{tps!Ab+ > zbHN+DTnPn$g6C1D_In>EIiX)H1wtVSPT-v`Nw@GkQ1A>n^l_34eZKcZC1GMG1NE8r > z%fTGepC9RvnZ(f|jL973(SSpYhLD0^dagXu0sv3|eW{Tu8Onc{0*u%)gV+K#8-s?Z > zw8)#7M*~telY(;*6N$7E;9#mC1p+NtBMI<QG#Lmf030CD&JHCLU_lu!$e2TOm@@db > zdvJy5f|pG|2<fpkZz%|qu{LYmAv7zUnArkW*a85EGmpTSf(VzGAuufff)az$ghM7; > z^ScdFkLv*f>N&%dDmC?3(^^}PMU8*}DO9RTG;eAyEs%_fIsX$<@R14HqklNmG_8^? > zpeU-lDai;_k<q2qz*DtK2+OLM2^drbK`J|KqB1z6h@sGWfHOQ>)P$=w4lpX?oDH(7 > zr}vl`G{YhSqc&Sm)?pnI1&9J_{iTAqRyMUXyQ;T<0iGUoB+z`v9_-P5g-L$(R}h37 > zE_kn2&>IzE2h0JmR=}4n&>kvS*ISqZ0Nb+XnF7t>f;y^Le;|cTz>4Z5A3xcMc^a{N > z!KE#z96s_1hVqC~KmzP6gD&Z?iGU}JlQ8O0NLJ7U34p4QTLPTbS<guvQox&aBMI*L > z8~`hlgWWPGsZI^rCw6LC+yO8u$b!LPSbx|8j3pGDjsF`cI9OW!MY$P}oXe9WI9RuG > z5(Utk`1yqngNT_G3tI>PSGczxk+3AFhxU?K?QsQ=eKG0zl=T?eQ}iB^_^_b0m--|L > ziGe7MxCd5fT#HkHG1vskRU~wh0+O79)Txn%yf7soQHi3tnGjsi4P2q!unS9qF&G8P > z&A4>ZxzvH!hTI5hyFpXIs8k`w8~xYbrAglfUb?f5HdrCEt0^oSDuHl?G9Va%S*K9D > zn*}JVTaYz-D1|NfAcEP10(+DQ0H#DEUN<^II4i^USYALImhw$gefg=NN{C1KB?)jC > z4T3U4^BXmdsU7O6duXAh8kkz(-$ZK#C18kK?f<r50#hm*2sdD|f3PVUu_`;oCJng< > z3b-j-IG9_IUY>#=>}?=_*xvRnh%8f&B=8aWMVYD-i7jo2ni>ZPw$<kav~(l1NAVFU > zz#urw&Wy>q9V#naC753@10)b%9a3S7u;K45h(9afN6V|k1sGSD00cYKa@kD5YXt=W > z07Ao6EoKEkMHhh?mMFkroU^ZA-~h}V%w81<ceuRy*flWjyDE+;T=g;@>be4+7&7k5 > z^(~e(ez_?a-6&YK4gtH((?ij;G28S_-qg+BL`_JZ%|@1FMvi27q~u4|<lN-sOzz}M > z{^Usp<*yNrHjsh<*iy$Su->qxd*Xs=v;POY8J}=!Fb+!qh}b77(1MAXA=7}8P9Zcm > zAYjM12WNX_K$|X@$XtjU&Z}s*khmoc1+XKvWdyq%CBP-O#T&b+WzjKO<8++xS}xw0 > zw#C7daOSW&!5g5x<#i6ycHo<6zPT%m8_!*+QV=L{#;#vbTk=USB}g=JA}A&J9L>>| > zRnZ%8ZZ3R^D0U*ql-Op1f+h^ri20H)37DaXhGOw^oRb|ej|K@@ekvB*2zB}e-KmsT > zuw9UFh1AsuGJsVE+b4<c2YqUsZZ^LL8|m>X>4Wu%<{=-RPN%-%Lz56j-Q_Xf4c?{p > zQKok4W&

1 0

[PATCH] Fix valgrind build with Musl
by Ivan Nikolaenko 18 May '22

18 May '22

Musl does not support __THROW macro, so need to define this empty. Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko(a)unikie.com> --- pkgs/development/tools/analysis/valgrind/default.nix | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/pkgs/development/tools/analysis/valgrind/default.nix b/pkgs/development/tools/analysis/valgrind/default.nix index 6e736df235f..8e04e0c8520 100644 --- a/pkgs/development/tools/analysis/valgrind/default.nix +++ b/pkgs/development/tools/analysis/valgrind/default.nix @@ -19,6 +19,10 @@ stdenv.mkDerivation rec { url = "https://bugsfiles.kde.org/attachment.cgi?id=143535"; sha256 = "036zyk30rixjvpylw3c7n171n4gpn6zcp7h6ya2dz4h5r478l9i6"; }) + (fetchpatch { + url = "https://bugsfiles.kde.org/attachment.cgi?id=148912"; + sha256 = "Za+7K93pgnuEUQ+jDItEzWlN0izhbynX2crSOXBBY/I="; + }) ]; outputs = [ "out" "dev" "man" "doc" ]; -- 2.25.1

2 1

[PATCH] host/rootfs: fix Weston hotplugging
by Alyssa Ross 11 May '22

11 May '22

On the Spectrum host, the kernel delivers uevents to mdevd. By default, if mdevd doesn't have a handler for an event, it won't do anything, and other processes that might be interested in the uevent hear about it. This means that Weston isn't notified about input device hotplugs. (And USB devices can appear after init has started, so any USB device counts as hotplugging even if it's attached before boot or even built in to the computer.) To fix this, we configure mdevd to rebroadcast events it doesn't handle on Netlink group 4. These events will then be picked up by Weston via libudev-zero, which listens on group 4 for hotplug events. Reported-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> Fixes: 372bacd ("etc/s6-rc: add weston service") --- Ville, does this fix the internal keyboard and mouse on your ThinkPad T560? host/rootfs/etc/s6-rc/mdevd/run | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/host/rootfs/etc/s6-rc/mdevd/run b/host/rootfs/etc/s6-rc/mdevd/run index f838d2d..c4615c8 100644 --- a/host/rootfs/etc/s6-rc/mdevd/run +++ b/host/rootfs/etc/s6-rc/mdevd/run @@ -2,4 +2,4 @@ # SPDX-License-Identifier: EUPL-1.2 # SPDX-FileCopyrightText: 2020-2022 Alyssa Ross <hi(a)alyssa.is> -mdevd -D3 -b134217728 +mdevd -D3 -O4 -b134217728 base-commit: 402c7fd0637e834f97b0ea1a2abb89c417e7469d -- 2.35.1

3 2

[PATCH] host: use a bigger kernel buffer for mdevd
by Alyssa Ross 23 Apr '22

23 Apr '22

This should fix a boot error on a Thinkpad P51s: > mdevd: fatal: unable to receive netlink message: No buffer space available From the mdevd documentation: > • -b kbufsz : try and reserve a kernel buffer of kbufsz bytes for > the netlink queue. Too large a buffer wastes kernel memory; too > small a buffer risks losing events. The default is 500 kB, which > should be enough for most systems; if you're getting "No buffer > space available" errors from mdevd at coldplug time, try > increasing this number. The new value of 128MiB is used by default by both udev and Busybox's mdev. --- I'm hoping this will fix the problem reported by terwiz in #spectrum. host/initramfs/etc/init | 2 +- host/rootfs/etc/s6-rc/mdevd/run | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/host/initramfs/etc/init b/host/initramfs/etc/init index bb58c84..e6236bc 100755 --- a/host/initramfs/etc/init +++ b/host/initramfs/etc/init @@ -19,7 +19,7 @@ if { mkfifo /dev/rootfs.poll } background { fdclose 3 - mdevd -C + mdevd -C -b134217728 } importas -iu mdevd_pid ! diff --git a/host/rootfs/etc/s6-rc/mdevd/run b/host/rootfs/etc/s6-rc/mdevd/run index 03ad5e3..22f0127 100644 --- a/host/rootfs/etc/s6-rc/mdevd/run +++ b/host/rootfs/etc/s6-rc/mdevd/run @@ -2,4 +2,4 @@ # SPDX-License-Identifier: EUPL-1.2 # SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi(a)alyssa.is> -mdevd -D3 +mdevd -D3 -b134217728 base-commit: 92e918a477dc7a16460a73d466fc2d42f7e7c421 -- 2.35.1

2 1

[PATCH] Documentation: document the binary cache
by Alyssa Ross 22 Apr '22

22 Apr '22

--- Documentation/Makefile | 1 + Documentation/binary-cache.adoc | 53 +++++++++++++++++++++++++++++ Documentation/getting-spectrum.adoc | 4 +++ 3 files changed, 58 insertions(+) create mode 100644 Documentation/binary-cache.adoc diff --git a/Documentation/Makefile b/Documentation/Makefile index 59bc3ae..e9d5886 100644 --- a/Documentation/Makefile +++ b/Documentation/Makefile @@ -8,6 +8,7 @@ datarootdir = $(prefix)/share docdir = $(datarootdir)/doc/spectrum SOURCES = \ + binary-cache.adoc \ creating-vms.adoc \ development.adoc \ getting-spectrum.adoc \ diff --git a/Documentation/binary-cache.adoc b/Documentation/binary-cache.adoc new file mode 100644 index 0000000..5ce6a4d --- /dev/null +++ b/Documentation/binary-cache.adoc @@ -0,0 +1,53 @@ +// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> +// SPDX-License-Identifier: GFDL-1.3-or-later OR CC-BY-SA-4.0 + += Binary cache + +Building Spectrum from source can take a very long time. To avoid +having to wait when building the system to try it out or test patches, +an x86_64 binary cache service is available. If configured to do so, +Nix will download build outputs from the cache, instead of building +them locally. + +Builds are run on the latest NixOS unstable on +https://www.vultr.com/[Vultr], and are signed to ensure authenticity. +The build infrastructure and signing key are accessible by Alyssa Ross +and Puck Meerburg. Storage is provided by Daniel Kuehn. Please +https://spectrum-os.org/participating.html[get in touch] if you +encounter any trouble with it. + +The binary cache is currently not able to provide logs, due to a +https://github.com/NixOS/nix/issues/6403[Nix bug]. + +== Setting up the binary cache + +=== NixOS + +The following configuration adds the Spectrum binary cache as a +substituter, and tells Nix to trust builds signed with its public key. + +[source,nix] +---- +{ ... }: + +{ + nix.settings.substituters = [ "https://cache.dataaturservice.se/spectrum/" ]; + nix.settings.trusted-public-keys = [ + "cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=" + "spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ=" + ]; +} +---- + +=== Non-NixOS systems + +Add the following configuration to /etc/nix/nix.conf: + +---- +substituters = https://cache.dataaturservice.se/spectrum/ https://cache.nixos.org/ +trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ= +---- + +If you have other binary caches configured besides cache.nixos.org, +make sure to add the Spectrum cache URL and public key to the existing +configuration lines in your nix.conf file. diff --git a/Documentation/getting-spectrum.adoc b/Documentation/getting-spectrum.adoc index 4ff6b05..fb69dce 100644 --- a/Documentation/getting-spectrum.adoc +++ b/Documentation/getting-spectrum.adoc @@ -10,6 +10,10 @@ from other operating systems, as long as they're not too invasive.] and have https://nixos.org/[Nix] and https://git-scm.com/[git] installed, it's very easy to do. +Before you start, consider setting up the +xref:binary-cache.adoc[Spectrum binary cache], which will save you a +lot of time waiting for builds. + == Trying Spectrum If you want to try Spectrum out to get a feel for it, without -- 2.35.1

2 1

[PATCH 0/2] img/combined: avoid chainloading in GRUB
by Alyssa Ross 21 Apr '22

21 Apr '22

Ville, could you give these patches a try on your P43s and see if "Try Spectrum" can boot now? Alyssa Ross (2): img/live: switch to systemd-boot img/combined: avoid chainloading in GRUB img/combined/default.nix | 10 ++++++++++ img/combined/grub.cfg.in | 5 +++-- img/live/Makefile | 23 +++++++++++------------ img/live/default.nix | 2 +- img/live/etc/os-release | 5 ----- 5 files changed, 25 insertions(+), 20 deletions(-) delete mode 100644 img/live/etc/os-release base-commit: 82212bb2207b9cb43f4510f373fb9ba8acfde21e -- 2.35.1

3 5

Binary cache
by Alyssa Ross 14 Apr '22

14 Apr '22

I'm happy to report that we finally have a Spectrum binary cache set up, which mitigate the very long build times that have been frustrating people attempting to try out Spectrum for themselves. URL: https://cache.dataaturservice.se/spectrum/ Signing key: spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ= The build infrastructure is hosted on Vultr, and is accessible by me and Puck. Storage is provided by Daniel Kuehn (lejonet). We don't have logs in the cache yet, due to a Nix bug[1]. My aim with this service is to provide low-cost and easy-to-run convenience for people interested in trying out Spectrum early, so don't assume this architecture is reflective of that of any future, higher stakes project infrastructure. Thanks to everybody who's helped me get this up and running over the last couple of weeks. I'll write some documentation about how to use it and link to it from the "Getting Spectrum" docs, and publish the configuration etc. for the build infrastructure soon but I've been pushing myself very hard to get this done in the first place, so I need a break from it first. In future I'd also like to explore possibilities for additional mirrors of the binary cache (lots of people have offered me storage hosting over the years!), but there are some additional technical challenges to solve before I can commit to that. Alyssa [1]: https://github.com/NixOS/nix/issues/6403

1 0

[PATCH] Don't assume /var/empty exists in Makefiles
by Alyssa Ross 10 Apr '22

10 Apr '22

It apparently doesn't on Ubuntu, and this way we also don't have to hack around it being missing in the Nix sandbox. Reported-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- host/rootfs/Makefile | 9 ++++++--- host/rootfs/default.nix | 5 ----- vm/app/catgirl/Makefile | 9 ++++++--- vm/app/catgirl/default.nix | 5 ----- vm/app/lynx/Makefile | 9 ++++++--- vm/app/lynx/default.nix | 5 ----- vm/sys/net/Makefile | 9 ++++++--- vm/sys/net/default.nix | 5 ----- 8 files changed, 24 insertions(+), 32 deletions(-) diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 87bc15e..d78cc4c 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -45,15 +45,18 @@ LINKS = bin sbin BUILD_FILES = build/etc/mdev/modalias.sh build/etc/s6-rc MOUNTPOINTS = dev ext run proc sys -build/rootfs.tar: $(PACKAGES_TAR) $(FILES) $(BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(FILES) $(BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(FILES) $(LINKS) echo $(BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty build/etc/mdev/modules.map: scripts/modprobe/gen_modules.map.awk mkdir -p $$(dirname $@) diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index cb02445..0bef7ba 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -94,11 +94,6 @@ stdenv.mkDerivation { MODULES_ORDER = "${kernel}/lib/modules/${kernel.modDirVersion}/modules.order"; PACKAGES_TAR = packagesTar; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' cp build/rootfs.ext4 $out ''; diff --git a/vm/app/catgirl/Makefile b/vm/app/catgirl/Makefile index 5029bb4..3ee5842 100644 --- a/vm/app/catgirl/Makefile +++ b/vm/app/catgirl/Makefile @@ -55,15 +55,18 @@ VM_LINKS = bin etc/ssl/certs/ca-certificates.crt VM_BUILD_FILES = build/etc/s6-rc VM_MOUNTPOINTS = dev run proc sys -build/rootfs.tar: $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(VM_FILES) $(VM_LINKS) echo $(VM_BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(VM_MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty VM_S6_RC_FILES = \ etc/s6-rc/catgirl/run \ diff --git a/vm/app/catgirl/default.nix b/vm/app/catgirl/default.nix index 230da00..bf35d04 100644 --- a/vm/app/catgirl/default.nix +++ b/vm/app/catgirl/default.nix @@ -74,11 +74,6 @@ stdenv.mkDerivation { PACKAGES_TAR = packagesTar; VMLINUX = "${kernel.dev}/vmlinux"; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' mv build/svc $out ''; diff --git a/vm/app/lynx/Makefile b/vm/app/lynx/Makefile index f449d27..016b1f5 100644 --- a/vm/app/lynx/Makefile +++ b/vm/app/lynx/Makefile @@ -55,15 +55,18 @@ VM_LINKS = bin etc/ssl/certs/ca-certificates.crt VM_BUILD_FILES = build/etc/s6-rc VM_MOUNTPOINTS = dev run proc sys -build/rootfs.tar: $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(VM_FILES) $(VM_LINKS) echo $(VM_BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(VM_MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty VM_S6_RC_FILES = \ etc/s6-rc/lynx/run \ diff --git a/vm/app/lynx/default.nix b/vm/app/lynx/default.nix index f7791d8..b188794 100644 --- a/vm/app/lynx/default.nix +++ b/vm/app/lynx/default.nix @@ -74,11 +74,6 @@ stdenv.mkDerivation { PACKAGES_TAR = packagesTar; VMLINUX = "${kernel.dev}/vmlinux"; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' mv build/svc $out ''; diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index 642c815..7103c0e 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -55,15 +55,18 @@ VM_LINKS = bin var/run VM_BUILD_FILES = build/etc/s6-rc VM_MOUNTPOINTS = dev run proc sys var/lib/connman -build/rootfs.tar: $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) +build/empty: + mkdir -p $@ + +build/rootfs.tar: build/empty $(PACKAGES_TAR) $(VM_FILES) $(VM_BUILD_FILES) cp --no-preserve=mode -f $(PACKAGES_TAR) $@ tar $(TARFLAGS) --append -f $@ $(VM_FILES) $(VM_LINKS) echo $(VM_BUILD_FILES) | cut -d/ -f2 | \ tar $(TARFLAGS) --append -f $@ -C build -T - for m in $(VM_MOUNTPOINTS); do \ - tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," /var/empty ; \ + tar $(TARFLAGS) --append -hf $@ --xform="s,.*,$$m," build/empty ; \ done - tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' /var/empty + tar $(TARFLAGS) --append -hf $@ --xform='s,.*,etc/service,' build/empty VM_S6_RC_FILES = \ etc/s6-rc/connman/dependencies \ diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index e3bd996..55ae9cb 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -83,11 +83,6 @@ stdenv.mkDerivation { PACKAGES_TAR = packagesTar; VMLINUX = "${kernel.dev}/vmlinux"; - postPatch = '' - mkdir $NIX_BUILD_TOP/empty - substituteInPlace Makefile --replace /var/empty $NIX_BUILD_TOP/empty - ''; - installPhase = '' mv build/svc $out ''; -- 2.35.1

2 1

Spectrum Status Update, March 2022
by Alyssa Ross 02 Apr '22

02 Apr '22

Hi all, I thought I'd try a different format of update. It's difficult to find the time for the big This Week in Spectrum updates I've tried to do before, but I'd like to provide some sort of account of what I've been doing. So, here's an overview of what I did in March. I'm happy to expand on any of it that sounds interesting — just hit Reply All and tell me what you'd like to hear more about! I'm also interested to hear what you think of this status update format. I'd like to get better about communicating what I'm working on, in a sustainable way. Let me know if you have any suggestions! Miscellaneous ------------- • Edited and published demo video[1] • Set up an IRC bot to post incoming mailing list messages • Switched from Busybox's modprobe to kmod • Removed unused dependencies • Various other cleanups and fixes • Started work towards CI for Spectrum • Prototyped a shared base image for application VMs [1]: https://diode.zone/w/dWAWHR38Zu3feRtDKjVEJb virtiofs investigation ---------------------- • Prototyped virtiofs VM filesystem access • Reported a bug: "Can't run unprivileged any more due to setgroups"[2] • Participated in discussion and testing of Musl port[3][4][5] [2]: https://gitlab.com/virtio-fs/virtiofsd/-/issues/36 [3]: https://github.com/slp/capng/pull/2#issuecomment-1059976861 [4]: https://github.com/slp/capng/pull/3 [5]: https://github.com/rust-lang/libc/pull/2713 Spectrum-related upstream Nixpkgs commits ----------------------------------------- • lvm2: don't use targetPlatform (05a6c124e65) • coreutils: add debug output (e30f0f31e8d) • pkgsMusl.systemd: fix build for 250.4 (39eee39fd92) • nghttp2: only run tests on GNU (8685cea963b) • python3.pkgs.importlib-metadata: fix cross (3c7b77e638b) • spidermonkey: use the same LLVM as rustc (3ff5f0eb764) • pkgsStatic.stdenv.cc.cc: put static libs in $lib (12c37aec377) • Revert "gcc: Always pass `--enable-shared` by default" (c6dd11ca39a) • libudev-zero: 1.0.0 -> 1.0.1 (c7b7ad77985) • linux_latest: 5.16.14 -> 5.17 (58ae11758e8) • crosvm: 81.12871.0.0-rc1 -> 99.14468.0.0-rc1 (6aefdafbed9) • shadow: 4.8.1 -> 4.8.11 (8d35d7e2bf1) • pkgsMusl.libnetfilter_conntrack: fix build (2cc5ec86571) • pkgsMusl.systemdMinimal: fix build (b8734c50e29) • linux.configfile: fix alts containing "/m" (fb079c3110d) • cloud-hypervisor: 21.0 -> 22.0 (36a211e1ee3) • edk2: 202108 -> 202202 (9222b68380e) • kmod: add dev and lib outputs (dc1303185f8) • systemd: update patchShebangs comment (a0bfc8e7c1f) • systemd: fix a whole bunch of typos (479b1cb510b) Pending Spectrum related Nixpkgs PRs ------------------------------------ • crosvm: add support for virgl_renderer{,_next} (#165128) • qemu: 6.2.0 -> 7.0.0 (#165291) Spectrum infra related upstream Nixpkgs commits ----------------------------------------------- • irccat: init at 0.4.8 (ce8cbe3c01f) • git: enable debug info (4345b27dedf) • cgit-pink: init at 1.3.0 (deab83e1167) • mailman-web: fix django version check removal (3512f5b7075) Demo video related upstream Nixpkgs commits ------------------------------------------- • ccsymbols: init at 2020-04-19 (cf7556eea5a)

1 0

[PATCH] img/live: increase ESP size
by Alyssa Ross 27 Mar '22

27 Mar '22

Switching from Busybox to kmod modprobe has left us needing a bit more space. Reported-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> Fixes: cf7e089 ("Busybox modprobe is banned") --- img/live/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/img/live/Makefile b/img/live/Makefile index e9b760c..319895a 100644 --- a/img/live/Makefile +++ b/img/live/Makefile @@ -34,7 +34,7 @@ build/bootx64.efi: etc/os-release build/cmdline $(INITRAMFS) $(EFI_STUB) $@ build/boot.fat: build/bootx64.efi - $(TRUNCATE) -s 209715200 $@ + $(TRUNCATE) -s 220200960 $@ $(MKFS_FAT) $@ $(MMD) -i $@ ::/EFI ::/EFI/BOOT $(MCOPY) -i $@ build/bootx64.efi ::/EFI/BOOT -- 2.35.1

2 1

[PATCH 1/2] Busybox modprobe is banned
by Alyssa Ross 22 Mar '22

22 Mar '22

Trying to use modprobe from Busybox has caused nothing but problems. The latest issue is that it doesn't implement softdep at all. rootfs.ext4 before: 509M rootfs.ext4 after: 513M Thanks-to: Puck Meerburg <puck(a)puckipedia.com> --- host/initramfs/default.nix | 8 +++++++- host/rootfs/default.nix | 10 ++++++++-- vm/app/catgirl/default.nix | 15 +++++++++++++-- vm/app/lynx/default.nix | 15 +++++++++++++-- vm/sys/net/default.nix | 15 +++++++++++++-- 5 files changed, 54 insertions(+), 9 deletions(-) diff --git a/host/initramfs/default.nix b/host/initramfs/default.nix index b207ace..5ce198a 100644 --- a/host/initramfs/default.nix +++ b/host/initramfs/default.nix @@ -16,7 +16,7 @@ let linux = rootfs.kernel; packages = [ - pkgsStatic.mdevd pkgsStatic.execline + pkgsStatic.execline pkgsStatic.kmod pkgsStatic.mdevd (pkgsStatic.cryptsetup.override { programs = { @@ -29,7 +29,13 @@ let (busybox.override { enableStatic = true; extraConfig = '' + CONFIG_DEPMOD n CONFIG_FINDFS n + CONFIG_INSMOD n + CONFIG_LSMOD n + CONFIG_MODINFO n + CONFIG_MODPROBE n + CONFIG_RMMOD n ''; }) ]; diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index b9fabee..2f2c368 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -4,7 +4,7 @@ { pkgs ? import <nixpkgs> {} }: pkgs.pkgsStatic.callPackage ( { lib, stdenv, runCommand, writeReferencesToFile, s6-rc, tar2ext4 -, busybox, cloud-hypervisor, cryptsetup, execline, jq, mdevd, s6 +, busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod, mdevd, s6 , s6-linux-utils, s6-portable-utils, socat, util-linuxMinimal, xorg }: @@ -20,7 +20,7 @@ let foot = pkgsGui.foot.override { allowPgo = false; }; packages = [ - cloud-hypervisor execline jq mdevd s6 s6-linux-utils + cloud-hypervisor execline jq kmod mdevd s6 s6-linux-utils s6-portable-utils s6-rc socat start-vm (cryptsetup.override { @@ -33,8 +33,14 @@ let (busybox.override { extraConfig = '' + CONFIG_DEPMOD n CONFIG_FINDFS n CONFIG_INIT n + CONFIG_INSMOD n + CONFIG_LSMOD n + CONFIG_MODINFO n + CONFIG_MODPROBE n + CONFIG_RMMOD n ''; }) ] ++ (with pkgsGui; [ foot westonLite ]); diff --git a/vm/app/catgirl/default.nix b/vm/app/catgirl/default.nix index ed897eb..716e7dd 100644 --- a/vm/app/catgirl/default.nix +++ b/vm/app/catgirl/default.nix @@ -9,7 +9,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenv, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4 -, busybox, cacert, catgirl, execline, mdevd, s6, s6-linux-utils +, busybox, cacert, catgirl, execline, kmod, mdevd, s6, s6-linux-utils , s6-portable-utils }: @@ -17,7 +17,18 @@ let inherit (lib) cleanSource cleanSourceWith concatMapStringsSep; packages = [ - busybox catgirl execline mdevd s6 s6-linux-utils s6-portable-utils s6-rc + catgirl execline kmod mdevd s6 s6-linux-utils s6-portable-utils s6-rc + + (busybox.override { + extraConfig = '' + CONFIG_DEPMOD n + CONFIG_INSMOD n + CONFIG_LSMOD n + CONFIG_MODINFO n + CONFIG_MODPROBE n + CONFIG_RMMOD n + ''; + }) ]; packagesSysroot = runCommand "packages-sysroot" { diff --git a/vm/app/lynx/default.nix b/vm/app/lynx/default.nix index 90fda0f..72bff7f 100644 --- a/vm/app/lynx/default.nix +++ b/vm/app/lynx/default.nix @@ -9,7 +9,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenv, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4 -, busybox, cacert, execline, lynx, mdevd, s6, s6-linux-utils +, busybox, cacert, execline, kmod, lynx, mdevd, s6, s6-linux-utils , s6-portable-utils }: @@ -17,7 +17,18 @@ let inherit (lib) cleanSource cleanSourceWith concatMapStringsSep; packages = [ - busybox execline lynx mdevd s6 s6-linux-utils s6-portable-utils s6-rc + execline kmod lynx mdevd s6 s6-linux-utils s6-portable-utils s6-rc + + (busybox.override { + extraConfig = '' + CONFIG_DEPMOD n + CONFIG_INSMOD n + CONFIG_LSMOD n + CONFIG_MODINFO n + CONFIG_MODPROBE n + CONFIG_RMMOD n + ''; + }) ]; packagesSysroot = runCommand "packages-sysroot" { diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index a9e1fdc..62a13bb 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -9,7 +9,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenv, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4, xorg -, busybox, connmanMinimal, dbus, execline, mdevd, nftables, s6 +, busybox, connmanMinimal, dbus, execline, kmod, mdevd, nftables, s6 , s6-linux-utils, s6-portable-utils }: @@ -19,9 +19,20 @@ let connman = connmanMinimal; packages = [ - busybox connman dbus execline mdevd s6 s6-linux-utils + connman dbus execline kmod mdevd s6 s6-linux-utils s6-portable-utils s6-rc + (busybox.override { + extraConfig = '' + CONFIG_DEPMOD n + CONFIG_INSMOD n + CONFIG_LSMOD n + CONFIG_MODINFO n + CONFIG_MODPROBE n + CONFIG_RMMOD n + ''; + }) + (nftables.override { withCli = false; }) ]; -- 2.35.1

2 3

[PATCH] use statically linked cryptsetup in initramfs
by Yureka 16 Feb '22

16 Feb '22

--- host/initramfs/default.nix | 31 +++++++++++++++++++++++++++---- 1 file changed, 27 insertions(+), 4 deletions(-) diff --git a/host/initramfs/default.nix b/host/initramfs/default.nix index ba6ede2..91e598f 100644 --- a/host/initramfs/default.nix +++ b/host/initramfs/default.nix @@ -2,15 +2,38 @@ { lib, stdenv, runCommand, writeReferencesToFile, pkgsStatic , busybox, cpio, cryptsetup, linux, lvm2 +, fetchpatch }: -let - cryptsetup' = cryptsetup; -in let inherit (lib) cleanSource cleanSourceWith concatMapStringsSep; - cryptsetup = cryptsetup'.override { lvm2 = lvm2.override { udev = null; }; }; + # https://github.com/NixOS/nixpkgs/pull/148948 + cryptsetup = (pkgsStatic.cryptsetup.override { + lvm2 = (pkgsStatic.lvm2.override { + udev = null; + }).overrideAttrs (old: { + patches = (old.patches or []) ++ [ + (fetchpatch { + url = "https://raw.githubusercontent.com/NixOS/nixpkgs/bf8b0d818a6c5fa65f318603756…"; + sha256 = "sha256-t67Osvcn92R7q9V+KDW+SyJDmajzVbnTPONlokcBU9U="; + }) + ]; + configureFlags = old.configureFlags ++ [ "--enable-static_link" ]; + makeFlags = old.makeFlags ++ [ "libdm.device-mapper" ]; + installPhase = '' + install -D -t $out/lib libdm/ioctl/libdevmapper.a + make -C libdm install_include + make -C libdm install_pkgconfig + ''; + outputs = [ "out" ]; + }); + }).overrideAttrs (old: { + NIX_LDFLAGS = ""; + configureFlags = old.configureFlags ++ [ + "--enable-static-cryptsetup" + ]; + }); packages = [ cryptsetup pkgsStatic.mdevd pkgsStatic.execline -- 2.34.0

2 1

[PATCH] installer: operate on FAT image directly
by Alyssa Ross 01 Feb '22

01 Feb '22

--- It feels cleaner to me to operate directly on the FAT image, rather than constructing the directory on the build filesystem and then copying it into the FAT image at the end. But, I'm wondering if maybe there's a reason it's better to do it the other way that I've missed? installer/default.nix | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/installer/default.nix b/installer/default.nix index f7e84ac..0818091 100644 --- a/installer/default.nix +++ b/installer/default.nix @@ -24,6 +24,7 @@ let esp = runCommand "esp.img" { nativeBuildInputs = [ grub libfaketime dosfstools mtools ]; + grubTargetDir = "${grub}/lib/grub/${grub.grubTarget}"; # Definition copied from util/grub-install-common.c. # Last checked: GRUB 2.06 pkglib_DATA = [ @@ -31,23 +32,23 @@ let "parttool.lst" "video.lst" "crypto.lst" "terminal.lst" "modinfo.sh" ]; } '' - mkdir -p files/grub/${grub.grubTarget} - cp ${grubCfg} files/grub/grub.cfg - cp ${grub}/lib/grub/${grub.grubTarget}/*.mod files/grub/${grub.grubTarget} + truncate -s 15M $out + faketime "1970-01-01 00:00:00" mkfs.vfat -i 0x2178694e -n EFI $out + mmd -i $out ::/EFI ::/EFI/BOOT \ + ::/grub ::/grub/${grub.grubTarget} ::/grub/fonts + + mcopy -i $out ${grubCfg} ::/grub/grub.cfg + mcopy -i $out $grubTargetDir/*.mod ::/grub/${grub.grubTarget} for file in $pkglib_DATA; do - path="${grub}/lib/grub/${grub.grubTarget}/$file" - ! [ -e "$path" ] || cp "$path" files/grub/${grub.grubTarget} + path="$grubTargetDir/$file" + ! [ -e "$path" ] || mcopy -i $out "$path" ::/grub/${grub.grubTarget} done + mcopy -i $out ${grub}/share/grub/unicode.pf2 ::/grub/fonts - install -D ${grub}/share/grub/unicode.pf2 files/grub/fonts/unicode.pf2 grub-mkimage -o grubx64.efi -p "(hd0,gpt1)/grub" -O ${grub.grubTarget} part_gpt fat - install -D grubx64.efi files/EFI/BOOT/BOOTX64.EFI + mcopy -i $out grubx64.efi ::/EFI/BOOT/BOOTX64.EFI - img=$out - truncate -s 15M $img - faketime "1970-01-01 00:00:00" mkfs.vfat -i 0x2178694e -n EFI $img - (cd files; mcopy -psvm -i $img ./* ::) - fsck.vfat -vn $img + fsck.vfat -n $out ''; installer = runCommand "installer.img" { -- 2.33.0

1 0

[PATCH] installer: more closely match grub-install
by Alyssa Ross 12 Jan '22

12 Jan '22

The main motivation here is that I noticed we're installing a lot of *.module files that grub-install didn't include. After this change, we're still differing from what grub-install does in the following ways: • Locale files are still not included • grub/x86_64-efi/core.efi or grub/x86_64-efi/grub.efi are not included. I'm not sure where those executables come from, but they don't seem to be required. • We're still not creating a grubenv file, which should be fine because there's no data that might need to be persisted anyway. --- installer/default.nix | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/installer/default.nix b/installer/default.nix index 9fc7493..f7e84ac 100644 --- a/installer/default.nix +++ b/installer/default.nix @@ -24,10 +24,22 @@ let esp = runCommand "esp.img" { nativeBuildInputs = [ grub libfaketime dosfstools mtools ]; + # Definition copied from util/grub-install-common.c. + # Last checked: GRUB 2.06 + pkglib_DATA = [ + "efiemu32.o" "efiemu64.o" "moddep.lst" "command.lst" "fs.lst" "partmap.lst" + "parttool.lst" "video.lst" "crypto.lst" "terminal.lst" "modinfo.sh" + ]; } '' - install -D ${grubCfg} files/grub/grub.cfg - cp -r ${grub}/lib/grub/${grub.grubTarget} files/grub/ - cp ${grub}/share/grub/unicode.pf2 files/grub/ + mkdir -p files/grub/${grub.grubTarget} + cp ${grubCfg} files/grub/grub.cfg + cp ${grub}/lib/grub/${grub.grubTarget}/*.mod files/grub/${grub.grubTarget} + for file in $pkglib_DATA; do + path="${grub}/lib/grub/${grub.grubTarget}/$file" + ! [ -e "$path" ] || cp "$path" files/grub/${grub.grubTarget} + done + + install -D ${grub}/share/grub/unicode.pf2 files/grub/fonts/unicode.pf2 grub-mkimage -o grubx64.efi -p "(hd0,gpt1)/grub" -O ${grub.grubTarget} part_gpt fat install -D grubx64.efi files/EFI/BOOT/BOOTX64.EFI base-commit: 907294e4a85e77e5db116a73ecfdf9a406be663b -- 2.33.0

1 0

[PATCH] fix imports after repo merge
by Yureka 16 Dec '21

16 Dec '21

--- host/initramfs/extfs.nix | 6 +++--- host/initramfs/live.nix | 2 +- host/rootfs/default.nix | 6 +++--- installer/default.nix | 2 +- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index b26f6e1..c50df21 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -1,17 +1,17 @@ { pkgs, runCommand, s6-rc, tar2ext4 }: let - netvm = import ../spectrum-netvm { + netvm = import ../../vm/sys/net { inherit pkgs; # inherit (foot) terminfo; }; - appvm-catgirl = import ../spectrum-appvm-catgirl { + appvm-catgirl = import ../../vm/app/catgirl { inherit pkgs; # inherit (foot) terminfo; }; - appvm-lynx = import ../spectrum-appvm-lynx { + appvm-lynx = import ../../vm/app/lynx { inherit pkgs; # inherit (foot) terminfo; }; diff --git a/host/initramfs/live.nix b/host/initramfs/live.nix index 27d868b..6446593 100644 --- a/host/initramfs/live.nix +++ b/host/initramfs/live.nix @@ -6,7 +6,7 @@ let initramfs = callPackage ./. { }; - host-rootfs = import ../spectrum-rootfs { inherit pkgs; }; + host-rootfs = import ../rootfs { inherit pkgs; }; extfs = pkgsStatic.callPackage ./extfs.nix { inherit pkgs; }; kernelTarget = stdenv.hostPlatform.linux-kernel.target; diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index fc347aa..21a9818 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -62,17 +62,17 @@ let -T ${writeReferencesToFile packagesSysroot} . ''; - netvm = import ../spectrum-netvm { + netvm = import ../../vm/sys/net { inherit pkgs; inherit (foot) terminfo; }; - appvm-catgirl = import ../spectrum-appvm-catgirl { + appvm-catgirl = import ../../vm/app/catgirl { inherit pkgs; inherit (foot) terminfo; }; - appvm-lynx = import ../spectrum-appvm-lynx { + appvm-lynx = import ../../vm/app/lynx { inherit pkgs; inherit (foot) terminfo; }; diff --git a/installer/default.nix b/installer/default.nix index 183a962..0685bdd 100644 --- a/installer/default.nix +++ b/installer/default.nix @@ -8,7 +8,7 @@ let inherit (pkgs.lib) removePrefix; inherit (nixos ./configuration.nix) config; - image = import ../spectrum-initramfs/live.nix { inherit pkgs; }; + image = import ../host/initramfs/live.nix { inherit pkgs; }; grub = grub2_efi; -- 2.34.0

2 1

[PATCH 1/2] increase partition sizes
by Yureka 16 Dec '21

16 Dec '21

--- host/initramfs/live.nix | 2 +- installer/default.nix | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/host/initramfs/live.nix b/host/initramfs/live.nix index 6446593..5dc4f87 100644 --- a/host/initramfs/live.nix +++ b/host/initramfs/live.nix @@ -29,7 +29,7 @@ let nativeBuildInputs = [ dosfstools mtools ]; passthru = { inherit uki; }; } '' - truncate -s ${toString (100 * 1024 * 1024)} $out + truncate -s ${toString (150 * 1024 * 1024)} $out mkfs.vfat $out mmd -i $out ::/EFI ::/EFI/BOOT mcopy -i $out ${uki} ::/EFI/BOOT/BOOTX64.EFI diff --git a/installer/default.nix b/installer/default.nix index 0685bdd..39933d9 100644 --- a/installer/default.nix +++ b/installer/default.nix @@ -35,7 +35,7 @@ let eosimages = vmTools.runInLinuxVM (runCommand "eosimages.img" { nativeBuildInputs = [ exfatprogs kmod util-linux ]; } '' - truncate -s 4G "$out" + truncate -s 5G "$out" mkfs.exfat -L eosimages "$out" mkdir /mnt modprobe loop -- 2.34.0

2 2

[PATCH linux] usbip: tools: usbipd: implement authorization
by Alyssa Ross 18 Oct '21

18 Oct '21

Devices are filtered before all usbipd operations, so that to each client, it's as if the only usbip-able devices are the ones it's been authorized to see. Authorizing a client is as simple as e.g.: ln -s /sys/bus/usb/devices/3-2.2 /var/run/usbipd/2001:db8::1/ Which will make the USB device 3-2.2 available to the client connecting from 2001:db8::1. I selected this filesystem based mechanism because of its simplicity to implement -- no need to parse any files, and granting and revoking permissions is atomic. Technically, it doesn't matter what's actually at that path, as long as it exists, but I've chosen to require it to be a symlink into sysfs, because I think that's the most clear, and actually checking that minimizes valid states, which is generally good practice. Currently, only IP addresses are supported (IPv4 and IPv6). It would be possible to extend this to support hostnames as well, but that would require various additional security considerations. This shouldn't make a difference to any USB functionality when a client _is_ authorized, but just to make sure, I've tested with various USB 2 and USB 3 devices, including hubs. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- tools/usb/usbip/src/usbipd.c | 112 +++++++++++++++++++++++++++++++---- 1 file changed, 100 insertions(+), 12 deletions(-) diff --git a/tools/usb/usbip/src/usbipd.c b/tools/usb/usbip/src/usbipd.c index 48398a78e88a..c380e785106f 100644 --- a/tools/usb/usbip/src/usbipd.c +++ b/tools/usb/usbip/src/usbipd.c @@ -16,6 +16,7 @@ #include <unistd.h> #include <netdb.h> #include <string.h> +#include <stdbool.h> #include <stdlib.h> #include <sys/types.h> #include <sys/stat.h> @@ -45,6 +46,7 @@ #define MAIN_LOOP_TIMEOUT 10 #define DEFAULT_PID_FILE "/var/run/" PROGNAME ".pid" +#define PROG_STATE_PATH "/var/run/" PROGNAME static const char usbip_version_string[] = PACKAGE_STRING; @@ -57,6 +59,14 @@ static const char usbipd_help_string[] = " -6, --ipv6\n" " Bind to IPv6. Default is both.\n" "\n" + " -A, --authorization\n" + " Apply authorization rules to clients. Clients\n" + " will only be able to attach to devices, or see\n" + " devices in the device list, that they are\n" + " authorized to access. To authorize a client to\n" + " access a device:\n" + " ln -s /sys/bus/usb/devices/$USB_PATH " PROG_STATE_PATH "/auth/$CLIENT_IP/\n" + "\n" " -e, --device\n" " Run in device mode.\n" " Rather than drive an attached device, create\n" @@ -82,12 +92,78 @@ static const char usbipd_help_string[] = " Show version.\n"; static struct usbip_host_driver *driver; +static bool enable_authorization; static void usbipd_help(void) { printf("%s\n", usbipd_help_string); } +static bool device_visible(struct usbip_exported_device *edev, int connfd) +{ + bool r = false; + int e; + struct sockaddr *addr = NULL; + socklen_t addr_len = 0; + char name[NI_MAXHOST]; + char *path = NULL, *real_path = NULL; + char *expected_path = NULL, *expected_real_path = NULL; + + if (!enable_authorization) + return true; + + if (getpeername(connfd, NULL, &addr_len) == -1) + goto err; + if (!(addr = malloc(addr_len))) + goto err; + if (getpeername(connfd, addr, &addr_len) == -1) + goto err; + + if ((e = getnameinfo(addr, addr_len, + name, sizeof name, NULL, 0, NI_NUMERICHOST))) { + err("getting name of peer: %s", gai_strerror(e)); + goto out; + } + + // Since we're mandating numeric hosts, we don't have to worry + // about what happens if name contains things like slashes. + // If we were to use names as well, we might have to be more + // careful, and use openat2, etc. + if (asprintf(&path, "%s/auth/%s/%s", + PROG_STATE_PATH, name, edev->udev.busid) == -1) + goto err; + + if (!(real_path = realpath(path, NULL))) { + if (errno == ENOENT) + goto out; + goto err; + } + + if (asprintf(&expected_path, "/sys/bus/usb/devices/%s", + edev->udev.busid) == -1) + goto err; + + if (!(expected_real_path = realpath(expected_path, NULL))) + goto err; + + r = !strcmp(real_path, expected_real_path); + goto out; + +err: + err("checking device visibility: %s", strerror(errno)); +out: + free(addr); + free(path); + free(real_path); + free(expected_path); + free(expected_real_path); + return r; +} + +#define CHECK_DEVICE_VISIBILITY(edev, connfd) \ + if (!device_visible(edev, connfd)) \ + continue + static int recv_request_import(int sockfd) { struct op_import_request req; @@ -109,6 +185,7 @@ static int recv_request_import(int sockfd) list_for_each(i, &driver->edev_list) { edev = list_entry(i, struct usbip_exported_device, node); + CHECK_DEVICE_VISIBILITY(edev, sockfd); if (!strncmp(req.busid, edev->udev.busid, SYSFS_BUS_ID_SIZE)) { info("found requested device: %s", req.busid); found = 1; @@ -176,6 +253,7 @@ static int send_reply_devlist(int connfd) /* number of exported devices */ list_for_each(j, &driver->edev_list) { edev = list_entry(j, struct usbip_exported_device, node); + CHECK_DEVICE_VISIBILITY(edev, connfd); if (edev->status != SDEV_ST_USED) reply.ndev += 1; } @@ -196,6 +274,7 @@ static int send_reply_devlist(int connfd) list_for_each(j, &driver->edev_list) { edev = list_entry(j, struct usbip_exported_device, node); + CHECK_DEVICE_VISIBILITY(edev, connfd); if (edev->status == SDEV_ST_USED) continue; @@ -586,17 +665,18 @@ static int do_standalone_mode(int daemonize, int ipv4, int ipv6) int main(int argc, char *argv[]) { static const struct option longopts[] = { - { "ipv4", no_argument, NULL, '4' }, - { "ipv6", no_argument, NULL, '6' }, - { "daemon", no_argument, NULL, 'D' }, - { "daemon", no_argument, NULL, 'D' }, - { "debug", no_argument, NULL, 'd' }, - { "device", no_argument, NULL, 'e' }, - { "pid", optional_argument, NULL, 'P' }, - { "tcp-port", required_argument, NULL, 't' }, - { "help", no_argument, NULL, 'h' }, - { "version", no_argument, NULL, 'v' }, - { NULL, 0, NULL, 0 } + { "ipv4", no_argument, NULL, '4' }, + { "ipv6", no_argument, NULL, '6' }, + { "authorization", no_argument, NULL, 'A' }, + { "daemon", no_argument, NULL, 'D' }, + { "daemon", no_argument, NULL, 'D' }, + { "debug", no_argument, NULL, 'd' }, + { "device", no_argument, NULL, 'e' }, + { "pid", optional_argument, NULL, 'P' }, + { "tcp-port", required_argument, NULL, 't' }, + { "help", no_argument, NULL, 'h' }, + { "version", no_argument, NULL, 'v' }, + { NULL, 0, NULL, 0 } }; enum { @@ -620,7 +700,7 @@ int main(int argc, char *argv[]) cmd = cmd_standalone_mode; driver = &host_driver; for (;;) { - opt = getopt_long(argc, argv, "46DdeP::t:hv", longopts, NULL); + opt = getopt_long(argc, argv, "46ADdeP::t:hv", longopts, NULL); if (opt == -1) break; @@ -632,6 +712,9 @@ int main(int argc, char *argv[]) case '6': ipv6 = 1; break; + case 'A': + enable_authorization = true; + break; case 'D': daemonize = 1; break; @@ -663,6 +746,11 @@ int main(int argc, char *argv[]) if (!ipv4 && !ipv6) ipv4 = ipv6 = 1; + if (enable_authorization && driver != &host_driver) { + err("authorization is only supported when using host driver"); + goto err_out; + } + switch (cmd) { case cmd_standalone_mode: rc = do_standalone_mode(daemonize, ipv4, ipv6); base-commit: 620b74d01b9d4393bef6742bf121908322c2fe0b -- 2.33.0

1 0

[DEMO PATCH linux 0/3] usbip: tools: support USB over vsock(7)
by Alyssa Ross 13 Oct '21

13 Oct '21

This series implements USB over VSOCK sockets, as an alternative to USB over IP. This would allow, for example, a host system to share its USB controller with a VM over VFIO, and then selectively attach to certain specific USB devices exported by the VM over a VSOCK connection. If inter-guest VSOCK were to be implemented in the kernel, or using a program like vsock-bridge[1] or socat[2], USB devices could also be shared between VMs in this way, without having to have a full-on networking setup between those VMs. I've taken different approaches to implement the VSOCK support in each of the usbip client and server. For the server, I added a new program, in.usbipd, that implements an inetd-style interface, taking a socket from its environment. This way, the socket could be a TCP socket, a VSOCK socket, or something else entirely, and the program doesn't have to care. The usbip client, on the other hand, cares about what type of socket it's communicating over, because it records address information for the remote server. Since this meant I had to teach it about VSOCK anyway, to tell it how to find that information for VSOCK sockets, it was easier to just extend the existing program with an option to choose an alternative protocol family than to have it take a socket from its environment. I've tested this series by running in.usbipd in a NixOS VM, and then attaching a device from it to my host system. The following NixOS configuration sets up an in.usbipd service listening on a VSOCK socket: systemd.sockets.usbipd = { wantedBy = [ "sockets.target" ]; listenStreams = [ "vsock::3240" ]; socketConfig.Accept = "yes"; }; systemd.services."usbipd@" = { serviceConfig.ExecStart = "${usbip}/bin/in.usbipd"; serviceConfig.StandardInput = "socket"; serviceConfig.StandardOutput = "socket"; serviceConfig.StandardError = "journal"; }; [1]: https://github.com/stefano-garzarella/vsock-bridge [2]: http://www.dest-unreach.org/socat/ Alyssa Ross (3): usbip: tools: in.usbipd: add usbip: tools: usbip: record protocol usbip: tools: usbip: support vsock(7) connections tools/usb/usbip/libsrc/vhci_driver.c | 26 ++- tools/usb/usbip/src/Makefile.am | 4 +- tools/usb/usbip/src/in.usbipd.c | 326 +++++++++++++++++++++++++++ tools/usb/usbip/src/usbip_attach.c | 54 ++++- tools/usb/usbip/src/usbip_list.c | 43 +++- tools/usb/usbip/src/usbip_network.c | 51 ++++- tools/usb/usbip/src/usbip_network.h | 4 +- 7 files changed, 470 insertions(+), 38 deletions(-) create mode 100644 tools/usb/usbip/src/in.usbipd.c base-commit: 620b74d01b9d4393bef6742bf121908322c2fe0b -- 2.32.0

1 3

This Week in Spectrum, 2021-W33
by Alyssa Ross 22 Aug '21

22 Aug '21

Hi! I've had a busy day today and am pretty tired, so I'm not sure how coherent my writing is at the moment. But I'd rather get this out on time, especially since tomorrow is also busy today. As I said last week[1], I took some time off this week as a preventative measure against burnout. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/87lf51ybwk.fsf@alyssa… rust-vmm -------- One of my patches[2] was accepted, but another[3] is still waiting. As I said last week, I have more fixes for rust-vmm planned, but want to let them catch up with the changes I've already sent upstream first so I know what base I'm using for the next stuff. I expect the one that's still waiting to be accepted next week. [2]: https://github.com/rust-vmm/vmm-sys-util/pull/135 [3]: https://github.com/rust-vmm/vhost/pull/69 spectrum-live ------------- Last week, I'd just integrated dm-verity into the Spectrum live image I've been working on. When it came time to work on the actual root filesystem, instead of the initramfs, I hit a bit of a brick wall. I realised that trying to generate a whole operating system image using Nix was giving me real writers' block. There was too much inbetween me and how the files ended up on disk, and that meant there was too much overhead to keep in mind when I was thinking about how things should be designed and laid out. It might feel like making a Linux root filesystem should be a solved problem, but Spectrum has a bunch of special requirements. You might want to do something like start a VM for each hardware device of a specific type, and that's something that isn't really addressed by most standard stuff. All this stuff is definitely solveable, but it requires some experimentation to get right, and Nix was getting a bit in the way of that. So I created a new directory, and I wrote a Makefile that builds an ext4 image, and I just started putting files in an etc/ directory. This made reasoning about the system way easier, and I was immediately making progress. Nix is great for building known targets, and for making customisable systems (there's no way my Makefile-based system would allow the amount of customisation I'd easily be able to provide with Nix), but for experimentation, it's a lot nicer to be closer to the end product. So once I know how all this should look I'll make it Nix-aware. Currently, I have a root filesystem with a service manager that can respond to hardware appearing and disappearing. The next (more exciting) step will be to have it start some VMs, and assign hardware to them appropriately. I'm looking forward to getting to that next week. An interesting challenge I'll have to solve will be figuring out simple categories (e.g. "ethernet device") from the huge amount of very specific information the kernel provides. I think I might be able abuse the modules.alias file from the kernel, that defines the mappings from PCI etc. information to default drivers. Then all I'll have to do will be to write mappings from default drivers to whatever categories I come up with, or what VM I want to assign them to, or whatever. One neat thing I'm using for the first time here is tar2ext4[4], a utility program that's part of a larger Microsoft open source project I don't entirely understand the purpose of. It's really useful for me it builds ext4 images entirely in userspace, which will be great for using in Nix derivations where it's not possible to just mount an ext4 image and write directly to it. For previous Spectrum experiments, I'd always used SquashFS[5], entirely because I already knew of a tar2sqfs program[6] that made creating filesystem images really easy. I've added tar2ext4 to Nixpkgs[7], which will hopefully help other people who have similar problems discover it. [4]: https://github.com/microsoft/hcsshim/blob/master/cmd/tar2ext4/tar2ext4.go [5]: https://en.wikipedia.org/wiki/SquashFS [6]: https://github.com/AgentD/squashfs-tools-ng [7]: https://github.com/NixOS/nixpkgs/pull/134650 musl ---- While I was working on the root filesystem, I noticed that mount -a, which mounts all filesystems described in fstab(5), wasn't working. This turned out to be because of a bug in Musl's implementation of getmntent(3), a libc function for parsing fstab files. So I wrote some tests and a fix and sent them to Musl[8]. [8]: https://inbox.vuxu.org/musl/20210821085420.474615-1-hi@alyssa.is/ Hope that all made sense. Next week, I'll continue working on Spectrum live, and maybe fix the next rust-vmm issue I have on my todo list if my final outstanding PR is merged.

1 0

[PATCH www] todo.html: refresh
by Alyssa Ross 21 Aug '21

21 Aug '21

The old tasks listed here aren't really relevant any more. But I would love me a static veritysetup -- it would massively reduce initramfs size. --- todo.html | 25 +++++++------------------ 1 file changed, 7 insertions(+), 18 deletions(-) diff --git a/todo.html b/todo.html index 67cf072..dc768e3 100644 --- a/todo.html +++ b/todo.html @@ -1,5 +1,5 @@ <!doctype html> - +   <html lang="en"> @@ -44,23 +44,12 @@ might be able to help out. <ol> <li> - Implement - <a href="https://wayland.freedesktop.org/docs/html/apa.html#protocol-spec-wl_composi…">wl_compositor - v4</a> in - <a href="https://spectrum-os.org/doc/developer-manual.html#_sommelier">Sommelier</a>. - This will mean we no longer have to - <a href="https://spectrum-os.org/git/nixpkgs/tree/pkgs/development/libraries/wlroots…">patch</a> - wlroots to downgrade it to an older protocol. - - <li> - Implement a ceiling for host memory allocated by the - crosvm <a href="https://spectrum-os.org/doc/developer-manual.html#_the_memfd_server">memfd - server</a>. - - <li> - Create a minimal VM kernel configuration. This will make it much - easier to iterate on the VM kernel, by substantially reducing clean - build times. + <a href="https://github.com/NixOS/nixpkgs/issues/134948">nixpkgs + #134984</a>: cryptsetup cannot be built statically. This would let + us massively reduce the size of Spectrum's initramfs. For Spectrum, + we only need the <code>veritysetup</code> program from that package + at the moment, so it's okay for us if other features have to be + disabled to make static linking work. </ol> <p> -- 2.32.0

2 2

[PATCH www] bibliography.html: link to cgit atom feed
by Alyssa Ross 16 Aug '21

16 Aug '21

This is a very Worse is Better approach to letting people subscribe to new links, but it also has zero setup cost, and my budget for this right now is by coincidence pretty much zero. :) --- bibliography.html | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/bibliography.html b/bibliography.html index e2445a0..1ac6bc1 100644 --- a/bibliography.html +++ b/bibliography.html @@ -22,6 +22,10 @@ <h1>Bibliography</h1> +<p>If you'd like to find out when new links are posted, subscribe to +the <a href="https://spectrum-os.org/git/www/atom/bibliography.html">cgit +atom feed for this page</a>. + <ul> <li><cite><a href="https://flatkill.org/">Flatpak — a security -- 2.32.0

1 0

This (Last) Week in Spectrum, 2021-W32
by Alyssa Ross 16 Aug '21

16 Aug '21

It's a short update this week, because most of what I did was a continuation of stuff from last week. rust-vmm -------- Last week, I mentioned I'd identified some Rust safety issues in rust-vmm. Most of the patches for these are now up[1][2][3]. The first has been accepted already, and I expect another to be accepted later today. There's still a UB issue I'm aware of and haven't sent a fix for yet, because there are a number of ways to fix it and I wanted to get my other patches in first before I decided how to fix that one. I deliberately haven't made any progress on using cloud-hypervisor's vhost-user-net backend with crosvm, which is what got me looking at this code in the first place. I want to make sure I can work on rust-vmm-adjacent things at a pace where I don't get overwhelmed with having to keep track of loads of patches and whether I've got them upstream yet. So I'll be putting that work on hold until the current round of patches are upstreamed. [1]: https://github.com/rust-vmm/vhost/pull/68 [2]: https://github.com/rust-vmm/vmm-sys-util/pull/135 [3]: https://github.com/rust-vmm/vhost/pull/69 spectrum-live ------------- For the past little while, in the time when I wasn't writing regular updates, I've been working on a live system for testing Spectrum. This will be especially useful for testing things like GPU support, because I can just build a live image with everything I might need, plug it into all the computers I want to test, and have everything be automatic from there. It will also probably evolve directly into what becomes the Spectrum base system that we'll hopefully all be running as the host system on our machines at some point. I shifted my focus back to this this week because of wanting to not get ahead of myself with rust-vmm. (I have a funding milestone for GPU support, so getting that checked off soon would be good.) The main thing I did this week was integrate dm-verity[4], which I did mostly for fun and to satisfy my curiosity. dm-verity is a Linux mechanism to efficiently ensure that a read-only filesystem hasn't been tampered with, by constructing a Merkle tree out of filesystem block hashes, and providing the root hash to the kernel when the filesystem is mounted. dm-verity is a _great_ fit for Nix, because we can generate the hashes at the same time as creating the filesystem image, and then embed the hash into the initramfs we're also building. Getting this all working took less than a day. The idea is that (long) in the future, we'll also implement Secure Boot, which will make sure the kernel and initramfs haven't been tampered with, and dm-verity will extend that integrity guarantee to the host system's root filesystem. I recommend reading "Producing a trustworthy x86-based Linux appliance"[5] by Matthew Garrett for an overview of how this all comes together. dm-verity is something that's particularly exciting to me, because it's very useful to us, but it's something that's generally used to frustrate end user attempts to control computers they own. In Spectrum, it's instead a tool that protects the end user against malicious filesystem changes, while being almost completely transparent to the user if they do want to modify their own system. Protecting against root filesystem tampering (which would require a VM escape or physical device access) is hardly the biggest priority for Spectrum, but integrating dm-verity was fun, interesting, and provided good motivation for working on the live image, which is one of the highest priority bits of the system. (Because I'm tired of having to say "you can't" when people ask me how they can try out Spectrum.) [4]: https://lwn.net/Articles/459420/ [5]: https://mjg59.dreamwidth.org/57199.html This week, I'm going to take a bit of time off as an anti-burnout defense, but probably not the whole week. I'll still keep an eye on the rust-vmm patches throughout this time as well, to make sure they're not delayed in getting accepted upstream.

1 0

[PATCH nixpkgs] chromiumOSPackages.libqmi: don't enable doc twice
by Alyssa Ross 16 Aug '21

16 Aug '21

This is already enabled in the original derivation, so overriding it to add it here just means we end up passing the option twice. --- pkgs/os-specific/linux/chromium-os/libqmi/default.nix | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkgs/os-specific/linux/chromium-os/libqmi/default.nix b/pkgs/os-specific/linux/chromium-os/libqmi/default.nix index b96b5224b57..7328642d0b4 100644 --- a/pkgs/os-specific/linux/chromium-os/libqmi/default.nix +++ b/pkgs/os-specific/linux/chromium-os/libqmi/default.nix @@ -15,7 +15,7 @@ libqmi.overrideAttrs ( # ModemManager tests fail with QRTR in Chromium OS 91. # Will hopefully be fixed in CrOS 92. - configureFlags = configureFlags ++ [ "--enable-gtk-doc" "--disable-qrtr" ]; + configureFlags = configureFlags ++ [ "--disable-qrtr" ]; passthru = passthru // { updateScript = ../update.py; -- 2.32.0

2 2

[PATCH nixpkgs] spectrumPackages: update execline backtick uses
by Alyssa Ross 12 Aug '21

12 Aug '21

execline 2.8.0.0 changed how backtick's options work: * -i (exit with the program's error code on failure) and -n (chomp off a trailing newline from the captured output) are now the defaults. * -E has been introduced, which does importas automatically. -E is fantastic, because we followed every single use of backtick with importas. Even we didn't have -n before, it's okay that chomping will now be enabled from the default, because in all those cases, there won't be a trailing newline anyway. --- .../linux/spectrum/testhost/default.nix | 14 ++++---------- pkgs/os-specific/linux/spectrum/vm/app/default.nix | 3 +-- pkgs/os-specific/linux/spectrum/vm/net/default.nix | 6 ++---- 3 files changed, 7 insertions(+), 16 deletions(-) diff --git a/pkgs/os-specific/linux/spectrum/testhost/default.nix b/pkgs/os-specific/linux/spectrum/testhost/default.nix index de62f0add67..d81ff8b826e 100644 --- a/pkgs/os-specific/linux/spectrum/testhost/default.nix +++ b/pkgs/os-specific/linux/spectrum/testhost/default.nix @@ -52,18 +52,14 @@ let # routers from clients. Other bits are reserved. # 5-6 Last two octets of client's IP (in 100.64.0.0/16). # - backtick -i router_mac { + backtick -E router_mac { pipeline { printf %.4x ${toString sys-vms.app.vmID} } sed s/^\$..\$\$..\$$/0A:B3:EC:80:\\1:\\2/ } - backtick -i client_mac { + backtick -E client_mac { pipeline { printf %.4x ${toString sys-vms.app.vmID} } sed s/^\$..\$\$..\$$/0A:B3:EC:00:\\1:\\2/ } - multisubstitute { - importas -iu router_mac router_mac - importas -iu client_mac client_mac - } # Create the net VM end, and attach it to the net VM. # @@ -143,12 +139,11 @@ let # is allowed to fail because it might already know that, in # which case it'll return EEXIST. if { modprobe vfio-pci } - backtick -in device_id { + backtick -E device_id { if { dd bs=2 skip=1 count=2 status=none if=''${PCI_PATH}/vendor } if { printf " " } dd bs=2 skip=1 count=2 status=none if=''${PCI_PATH}/device } - importas -iu device_id device_id foreground { redirfd -w 1 /sys/bus/pci/drivers/vfio-pci/new_id printf "%s" $device_id @@ -205,8 +200,7 @@ writeScriptBin "spectrum-testhost" '' if { redirfd -w 1 /proc/sys/net/ipv4/ip_forward echo 1 } importas -iu runtime_dir XDG_RUNTIME_DIR - backtick -in TOP { mktemp -dp $runtime_dir spectrum.XXXXXXXXXX } - importas -iu top TOP + backtick -E top { mktemp -dp $runtime_dir spectrum.XXXXXXXXXX } if { echo $top } if { rsync -r --chmod=Du+w ${servicesDir}/ ''${top}/service } background { diff --git a/pkgs/os-specific/linux/spectrum/vm/app/default.nix b/pkgs/os-specific/linux/spectrum/vm/app/default.nix index f8ff480932c..9ffed7e5b13 100644 --- a/pkgs/os-specific/linux/spectrum/vm/app/default.nix +++ b/pkgs/os-specific/linux/spectrum/vm/app/default.nix @@ -26,12 +26,11 @@ runCommand "vm-app" rec { oneshot ''; up = writeText "net-up" '' - backtick -i LOCAL_IP { + backtick -E LOCAL_IP { pipeline { ip -j link show eth0 } pipeline { jq -r ".[0].address | split(\":\") | .[4:6] | \"0x\" + .[]" } xargs printf "100.64.%d.%d" } - importas -iu LOCAL_IP LOCAL_IP if { ip address add ''${LOCAL_IP}/32 dev eth0 } if { ip link set eth0 up } diff --git a/pkgs/os-specific/linux/spectrum/vm/net/default.nix b/pkgs/os-specific/linux/spectrum/vm/net/default.nix index 5921b62fcf7..75c86b8f92c 100644 --- a/pkgs/os-specific/linux/spectrum/vm/net/default.nix +++ b/pkgs/os-specific/linux/spectrum/vm/net/default.nix @@ -54,12 +54,11 @@ runCommand "vm-net" rec { # Our IP is encoded in the NIC-specific portion of the # interface's MAC address. - backtick -i CLIENT_IP { + backtick -E CLIENT_IP { pipeline { ip -j link show $INTERFACE } pipeline { jq -r ".[0].address | split(\":\") | .[4:6] | \"0x\" + .[]" } xargs printf "100.64.%d.%d" } - importas -iu CLIENT_IP CLIENT_IP if { ip address add 169.254.0.1/32 dev $INTERFACE } if { ip link set $INTERFACE up } @@ -121,7 +120,7 @@ runCommand "vm-net" rec { ''; run = writeScript "connman-run" '' #! ${execline}/bin/execlineb -S0 - backtick -in HARDWARE_INTERFACES { + backtick -E HARDWARE_INTERFACES { pipeline { find -L /sys/class/net -mindepth 2 -maxdepth 2 -name address -print0 } @@ -132,7 +131,6 @@ runCommand "vm-net" rec { # Extract the interface names from the address file paths. awk -F/ "{if (NR > 1) printf \",\"; printf \"%s\", $5}" } - importas -iu HARDWARE_INTERFACES HARDWARE_INTERFACES ${connman}/bin/connmand -ni $HARDWARE_INTERFACES ''; base-commit: d2e147bedf251976ed99b94b6c905d6761f7a892 -- 2.32.0

2 2

This Week in Spectrum, 2021-W31
by Alyssa Ross 08 Aug '21

08 Aug '21

What a week. Progress has felt a bit slow, but the work has been consistently interesting, and there have been some exciting developments in the ecosystem. Nixpkgs ------- Last week I'd posted the patches required to get all of Spectrum working with a more up-to-date Nixpkgs. After last week's email, Cole reviewed the patch set, so I applied it, and merged the nixpkgs-update branch into master, so now Spectrum is using a reasonably up-to-date Nixpkgs for the first time in a long time. I also said > Once that's done, it's time for another chromiumOSPackages upgrade, but > that should be pretty easy this time because we're only one version behind. Alas, it's never that simple. Firstly, the chromiumOSPackages update script broke, because the information about the currently released Chromium OS build seems to have gone missing. Google is apparently[1] serving build number 13982, but their published build metadata[2] includes builds 13981 and 13983, but not 13982. This means it's not possible for me to know what Git revisions are used in the currently released Chromium OS. Assuming this is just a one time thing, I hacked the update script to just look at the previous build, but we should keep an eye on this. If it ever happens again we should probably implement some sort of mitigation in the update script. Once I had new versions of the Chromium OS packages it was time to get them to build, which was straightforward enough for everything except crosvm. For Spectrum, we have a patch[3] for crosvm to make it support VIRTIO_NET_F_MAC[4], which is a mechanism by which the host system can indicate to the guest kernel what it should set as the MAC address of a virtual network device. After the update, this patch no longer applied, because all of a sudden crosvm has two different virtual network device implementations. This turns out to be because crosvm has implemented vhost-user[5], a protocol to allow virtual devices to be implemented outside the VMM program! This is great news, but it's also surprising. virtiofs[6] was designed to be implemented with vhost-user, but when crosvm implemented it a while ago, they became the only implementer to do so in-VMM. It's great to see them moving in the vhost-user direction, because it makes it much easier to mix and match virtual device implementation and VMMs. Most excitingly, I saw a reference to vhost-user-wl, meaning a standalone implementation of Virtio Wayland. This would allow us to use Virtio Wayland with other, non-crosvm VMMs, which is great because I think cloud-hypervisor[7] is probably going to end up being a better fit for Spectrum, but Virtio Wayland was crosvm's killer app. I'd even thought about trying to port crosvm's virtio wayland implementation to vhost-user myself, so it's great to know that when the time comes, that'll already have been done for me. While I could have fixed my crosvm patch for all this new virtual network device code, the introduction of vhost-user support means we should be able to drop the patch altogether. cloud-hypervisor provides a vhost-user-net implementation[8] that already supports VIRTIO_NET_F_MAC, so if we can just get crosvm to talk to cloud-hypervisor's vhost-user-net implementation we shouldn't have to carry any patch for this any more. [1]: https://cros-updates-serving.appspot.com/ [2]: https://chromium.googlesource.com/chromiumos/manifest-versions [3]: https://spectrum-os.org/git/nixpkgs/tree/pkgs/os-specific/linux/chromium-os… [4]: https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html [5]: https://qemu.readthedocs.io/en/latest/interop/vhost-user.html [6]: https://virtio-fs.gitlab.io/index.html#overview [7]: https://github.com/cloud-hypervisor/cloud-hypervisor [8]: https://github.com/cloud-hypervisor/cloud-hypervisor/tree/db2159b5638eb4ccf… cloud-hypervisor / rust-vmm --------------------------- So I started looking at cloud-hypervisor to try to hook this all up, but it looks like cloud-hypervisor still has some issues where it doesn't quite follow the vhost-user specification. As I was trying to debug this, I noticed some UB in rust-vmm[9] (the shared utility code project for crosvm and its derivatives like cloud-hypervisor and Firecracker[10]). Being a good citizen, I started working on a fix for this, and then I encountered some more issues with functions that should have been marked as unsafe but weren't. So that's going to need to be fixed too. But the rust-vmm code is otherwise very high quality and easy to work with, and they're very responsive, so it shouldn't take long to get these issues fixed. The affected code is fortunately all to do with communication between vhost-user backends and the VMM, so it's very very unlikely that it's anything that could be exploited by a guest. These sorts of problems are exactly the sort of thing that Rust is supposed to prevent, so it was disappointing to discover these issues. But on the other hand, Rust made them stand out like sore thumbs to me, so even though these issues managed to sneak in, there's definitely still a huge benefit to using Rust for these sorts of programs. [9]: https://github.com/rust-vmm [10]: https://firecracker-microvm.github.io/ In the next week, I'm hoping to get all the rust-vmm issues I've discovered fixed, and maybe get cloud-hypervisor's backends to speak proper vhost-user. I'm getting my second vaccination next week as well though, so we'll see. I might just end up being sick.

1 0

[COMMIT] nixpkgs-update merged into master
by Alyssa Ross 04 Aug '21

04 Aug '21

Everything seems to be working. I'm going to start another merge immediately, so we're fully up to date. But the Nixpkgs I merged in was only from last week, so it shouldn't be too hard.

1 0

[PATCH nixpkgs 0/6] Fix build failures after Nixpkgs update
by Alyssa Ross 03 Aug '21

03 Aug '21

Alyssa Ross (6): chromiumOSPackages.vm_protos: set full go_package chromiumOSPackages.modemmanager: disable patches chromiumOSPackages.modemmanager-next: add gtk-doc chromiumOSPackages.libqmi: disable QRTR linux_cros: fix build wlroots: update Spectrum patches ...ayland-downgrade-to-wl_compositor-v3.patch | 64 ++++++----- ...til-support-virtio_wl-shm-allocation.patch | 11 +- ...-mk-don-t-leak-source-absolute-paths.patch | 4 +- ...ommon-mk-.gn-don-t-hardcode-env-path.patch | 4 +- .../os-specific/linux/chromium-os/default.nix | 3 +- .../linux/chromium-os/libqmi/default.nix | 4 +- .../chromium-os/modem-manager/default.nix | 2 + .../linux/chromium-os/modem-manager/next.nix | 6 +- ...er-don-t-leak-source-absolute-paths.patch} | 6 +- ...tools-sommelier-Switch-to-the-stabl.patch} | 6 +- .../linux/chromium-os/sommelier/default.nix | 4 +- ...-goproto_library-source_relative-opt.patch | 48 +++++++++ ...tools-proto-set-go_package-correctly.patch | 102 ++++++++++++++++++ .../linux/chromium-os/vm_protos/default.nix | 5 + 14 files changed, 222 insertions(+), 47 deletions(-) rename pkgs/os-specific/linux/chromium-os/sommelier/{0003-sommelier-don-t-leak-source-absolute-paths.patch => 0005-sommelier-don-t-leak-source-absolute-paths.patch} (82%) rename pkgs/os-specific/linux/chromium-os/sommelier/{0004-Revert-Revert-vm_tools-sommelier-Switch-to-the-stabl.patch => 0006-Revert-Revert-vm_tools-sommelier-Switch-to-the-stabl.patch} (99%) create mode 100644 pkgs/os-specific/linux/chromium-os/vm_protos/0003-common-mk-add-goproto_library-source_relative-opt.patch create mode 100644 pkgs/os-specific/linux/chromium-os/vm_protos/0004-vm_tools-proto-set-go_package-correctly.patch base-commit: cab13e087c80c4e7476bb7df432e2b7deb56563d -- 2.32.0

2 8

This (Last) Week in Spectrum, 2021-W30
by Alyssa Ross 02 Aug '21

02 Aug '21

Hi, it's been a long time since I've done one of these, but so much interesting stuff happened this week, and there's been a bit of renewed interest in the project, that it _really_ needed an update. So here it is. As always when I'm not keeping up with TWiS, this update is limited to only things that happened in the last week, because otherwise it would just be far too long. Wayland ------- A problem I've been thinking about for a long time is Wayland access control. The Linux desktop ecosystem is moving towards access controls for most functionality through xdg-desktop-portal[1] (which sadly doesn't seem to have a website). But it doesn't cover stuff that's part of the core Wayland protocol, like access to the clipboard. And some compositors (wlroots-based ones) provide extra Wayland protocols for things like screenshots. And up to now, Wayland compositors haven't really done any authorization for these potentially dangerous APIs. (All of this is only really meaningful for sandboxed applications -- running in Flatpak or a Spectrum VM or something. Linux isn't really set up to do separation between processes running as the same user without namespaces.) So earlier this week, I posted[2] to the Wayland mailing list with an idea I'd been thinking about for a while[3], which was to place a proxy program in front of the Wayland compositor, that would intercept client->compositor requests and handle access control. I was quickly convinced that a proxy wasn't a good idea, but there was a lot of discussion, and it was really helpful to me figuring out what the right way to do it might be. There are really two problems to be solved here, one of which I hadn't even thought much about. The first is securely identifying a Wayland client. A compositor needs to be able to form the question "Should client X be able to do this?", and to do that it needs to be able to identify a client as client X, and know if it tries to interact with client X later, that it'll be talking to the same client. In a non-virtualized system, the obvious way to do this would be getting the pid of the client from the connection socket and then looking it up in proc(5) to find out executable path, but this approach is fundamentally insecure[4]. The way forward here (and one that would work for Spectrum) appears to be the proposed Security Contexts protocol[5], which would allow a sandbox implementation to provide a security context identifier for a client before handing off the Wayland connection to the client. Once the security context had been set up, it wouldn't be allowed to be modified, so once the sandboxed client was given the connection, it wouldn't be able to change the security context identifier. In Spectrum, the security context identifier here would likely be a unique, user-provided name for the VM, and the security context setup would be done by the virtio wayland implementation in the VMM. The second part of this puzzle is how the compositor should decide whether a client should be allowed to perform a particular operation, like a paste or going fullscreen (which is risky because it might spoof a desktop). There was actually a previous attempt at this a long time ago. libwsm[7] (short for Wayland Security Module) was a library that compositors would have integrated to make authorization decisions. But it wasn't adopted by compositors, and it some things we now know[8] to be bad ideas, like setting policy based on the executable path. It also made compositors responsible for any sort of authorization UI. In my opinion, it's better to have that done by the external piece, so that compositors have as little work to do as possible and therefore authorization is implemented as widely as possible. The compositor could implement an authorization system entirely on its own, but this would be a lot of code for each compositor to write, and it would limit the user to whatever permissions system the compositor came up with, which might not be able to accommodate their needs. (An example of this would be a Spectrum user that wanted to allow pasting between two applications, but not allow pasting between either of those and any other application.) It could also be implemented by having the compositors integrate with something like polkit[6], but compositor authors are reluctant to integrate directly with a single system like that, and even polkit might not support everything that would be desirable in an authorization system. (For example, it might be nice to implement libwsm's concept of a soft allow, where an action is permitted, but a notification is shown so the user is aware it's happened.) So a third solution, suggested by ifreund, a Wayland compositor author, in the Spectrum IRC channel, would be to have a privileged Wayland client that the compositor could ask authorization questions to, with a new protocol. The compositor would know that this particular client should be privileged because it either wouldn't be in a security context, or would be given a special security context identifier known to the compositor ahead of time. Then, implementations of this protocol could do authorization however they wanted, with the only limitation being the questions the compositor was programmed to ask them. I think this is a good way forward, but it'd be important to discuss with more compositor authors before getting to excited about it. Next steps from here with Wayland are: * Figure out what needs to happen to move the security contexts proposal forward. If it needs an experimental implementation, maybe we could help with that? * Inquire about the authorization protocol idea, and see how other compositor offers would feel about it. If there's a generally positive reaction, figure out how to move forward with it. [1]: https://github.com/flatpak/xdg-desktop-portal/ [2]: https://lists.freedesktop.org/archives/wayland-devel/2021-July/041896.html [3]: https://spectrum-os.org/lists/archives/spectrum-discuss/8735ueudel.fsf@alys… [4]: https://gitlab.freedesktop.org/wayland/weston/-/issues/206#note_176699 [5]: https://gitlab.freedesktop.org/wayland/wayland-protocols/-/merge_requests/68 [6]: https://www.freedesktop.org/software/polkit/docs/latest/polkit.8.html [7]: https://github.com/mupuf/libwsm [8]: https://lore.kernel.org/lkml/87r1xplsku.fsf@x220.int.ebiederm.org/ Nixpkgs ------- I created a branch in Spectrum's nixpkgs repository[9] for the long-overdue merge with upstream, did the merge, and posted some patches[10] to spectrum-devel that fix all the builds that broke as a result. I've been letting them sit for a few days hoping for a review. Once that's done, it's time for another chromiumOSPackages upgrade, but that should be pretty easy this time because we're only one version behind. [9]: https://spectrum-os.org/git/nixpkgs [10]: https://spectrum-os.org/lists/archives/spectrum-devel/20210729100928.196534… Thanks for keeping up with Spectrum. :)

1 0

Updating Spectrum's nixpkgs
by Alyssa Ross 29 Jul '21

29 Jul '21

I've just pushed a branch to our nixpkgs repository called nixpkgs-update, where I've merged in yesterday's nixos-unstable. From here, I'm going to send some patches targeting that branch, until everything builds again and we can merge it back into master. And then we're never going to go this long between updates ever again.

1 0

[PATCH nixpkgs 0/2] spectrumPackages.spectrum-vm: switch compositor from Wayfire to tinywl
by Alyssa Ross 01 Jul '21

01 Jul '21

This is another change I want to get in before updating Spectrum's Nixpkgs, because once we do that the current approach will no longer work. As explained in patch 2's commit message, this change only affects this demo, not the eventual Spectrum system. Alyssa Ross (2): tinywl: init at 0.11.0 spectrumPackages.sys-vms.comp: switch to tinywl .../window-managers/tinywl/default.nix | 30 ++++++++++++++++++ .../linux/spectrum/vm/comp/default.nix | 31 ++++++------------- pkgs/top-level/all-packages.nix | 2 ++ 3 files changed, 41 insertions(+), 22 deletions(-) create mode 100644 pkgs/applications/window-managers/tinywl/default.nix base-commit: 129d329e3bcdcc972454ce284e17da16b28bd401 -- 2.32.0

1 2

[PATCH nixpkgs 0/2] Pre-merge fixes for Nixpkgs update
by Alyssa Ross 01 Jul '21

01 Jul '21

I'm working on merging in a new version of upstream Nixpkgs. These are a couple of small fixes that came up during that work, but can be taken care of first since they make improvements to the current version. Alyssa Ross (2): chromiumOSPackages.modemmanager-next: fix libqmi spectrumPackages.sys-vms.comp: don't pin Emacs .../linux/chromium-os/modem-manager/next.nix | 12 ++++++++++-- pkgs/os-specific/linux/spectrum/vm/comp/default.nix | 4 ++-- 2 files changed, 12 insertions(+), 4 deletions(-) base-commit: 129d329e3bcdcc972454ce284e17da16b28bd401 -- 2.31.1

2 5

[PATCH platform2 0/2] Make vm_protos compatible with protoc-gen-go 1.5.x
by Alyssa Ross 30 Jun '21

30 Jun '21

My previous attempt[1] at this was totally wrong, because I didn't understand what change needed to be made, and didn't have any way of testing it. To make sure this time that I got it right, I wrote a package for tremplin[2], which was the only Chromium OS component I could find that was written in Go and actually used vm_protos. With these changes, I was able to add "${vm_protos}/lib/gopath" to tremplin's extraSrcPaths, and get a successful build. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20210627165035.899276… [2]: https://chromium.googlesource.com/chromiumos/platform/tremplin Alyssa Ross (2): common-mk: add goproto_library source_relative opt vm_tools: proto: set go_package correctly common-mk/proto_library.gni | 7 +++++++ vm_tools/proto/BUILD.gn | 5 +++++ vm_tools/proto/tremplin.proto | 2 +- vm_tools/proto/vm_crash.proto | 2 +- vm_tools/proto/vm_guest.proto | 1 + vm_tools/proto/vm_host.proto | 1 + 6 files changed, 16 insertions(+), 2 deletions(-) -- 2.31.1

2 4

[PATCH platform2] vm_tools: proto: fix go_package import path
by Alyssa Ross 27 Jun '21

27 Jun '21

This is required by protoc-gen-go >= 1.4.0. --- I don't really have any idea what I'm doing here -- this is my best guess. If somebody could tell me if it looks sensible I'd be much obliged. vm_tools/proto/common.proto | 1 + vm_tools/proto/tremplin.proto | 2 +- vm_tools/proto/vm_crash.proto | 2 +- vm_tools/proto/vm_guest.proto | 1 + vm_tools/proto/vm_host.proto | 1 + 5 files changed, 5 insertions(+), 2 deletions(-) diff --git a/vm_tools/proto/common.proto b/vm_tools/proto/common.proto index 39a3e39528..f11d128ef1 100644 --- a/vm_tools/proto/common.proto +++ b/vm_tools/proto/common.proto @@ -8,6 +8,7 @@ option cc_enable_arenas = true; // Common definitions used by both host and guest messages. package vm_tools; +option go_package = "chromiumos/vm_tools"; // gRPC requires that every RPC has an argument and a return value. This empty // message is used when an argument or return value is not required. diff --git a/vm_tools/proto/tremplin.proto b/vm_tools/proto/tremplin.proto index aac76f7a9e..0a8968fe8d 100644 --- a/vm_tools/proto/tremplin.proto +++ b/vm_tools/proto/tremplin.proto @@ -8,7 +8,7 @@ option cc_enable_arenas = true; // This file defines services for tremplin, the container springboard service. package vm_tools.tremplin; -option go_package = "tremplin_proto"; +option go_package = "chromiumos/vm_tools/tremplin/tremplin_proto"; // This needs to be duplicated because the gyp rule for building // go code makes it difficult to have imports. diff --git a/vm_tools/proto/vm_crash.proto b/vm_tools/proto/vm_crash.proto index 6e4f62fe13..70d620fde8 100644 --- a/vm_tools/proto/vm_crash.proto +++ b/vm_tools/proto/vm_crash.proto @@ -7,7 +7,7 @@ syntax = "proto3"; option cc_enable_arenas = true; package vm_tools.cicerone; -option go_package = "vm_crash"; +option go_package = "chromiumos/vm_tools/cicerone/vm_crash"; import "common.proto"; diff --git a/vm_tools/proto/vm_guest.proto b/vm_tools/proto/vm_guest.proto index 34d2fd3d60..b3a742ec07 100644 --- a/vm_tools/proto/vm_guest.proto +++ b/vm_tools/proto/vm_guest.proto @@ -8,6 +8,7 @@ option cc_enable_arenas = true; // This file defines services that will be running in the guest VM. package vm_tools; +option go_package = "chromiumos/vm_tools"; import "common.proto"; import "google/protobuf/timestamp.proto"; diff --git a/vm_tools/proto/vm_host.proto b/vm_tools/proto/vm_host.proto index a8bd066f61..de31fa747f 100644 --- a/vm_tools/proto/vm_host.proto +++ b/vm_tools/proto/vm_host.proto @@ -8,6 +8,7 @@ option cc_enable_arenas = true; // This file defines services that will be running on the host for the VM. package vm_tools; +option go_package = "chromiumos/vm_tools/vm_host"; import "common.proto"; -- 2.31.1

1 0

[PATCH doc] Add note about spectrum-vm Mesa version mismatches
by Alyssa Ross 24 Jun '21

24 Jun '21

Cc: Cole Helbling <cole.e.helbling(a)outlook.com> --- developer-manual.adoc | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/developer-manual.adoc b/developer-manual.adoc index da37ef0..9db8b3c 100644 --- a/developer-manual.adoc +++ b/developer-manual.adoc @@ -121,6 +121,14 @@ spectrumPackages.spectrum-vm --run spectrum-vm` in a terminal, which will start up an instance of crosvm from Nixpkgs running Wayfire in a new window. +**** +If no Wayfire window appears, and you see error messages about EGL, +the cause is likely to be a mismatch between the Mesa version on your +host system, and the one Spectrum's Nixpkgs. If Spectrum's Mesa is +out of date, please report it using any of the project's +https://spectrum-os.org/participating.html[communication channels]. +**** + **** Be advised that Wayfire's default keymap is set to Dvorak https://spectrum-os.org/git/nixpkgs/tree/pkgs/os-specific/linux/spectrum/ro… -- 2.31.1

1 1

[PATCH nixpkgs 1/4] kernelPatches: drop export_kernel_fpu_functions
by Alyssa Ross 03 Jun '21

03 Jun '21

From: Atemu <atemu.main(a)gmail.com> Hasn't been necessary since ZFS 0.8.3 (cherry picked from commit 88f877e07def90a2c77106b86802b143ca8f97b3) Backporting this for Spectrum because it won't apply to future CrOS kernels. --- .../export_kernel_fpu_functions_4_14.patch | 60 ------------------- .../export_kernel_fpu_functions_5_3.patch | 42 ------------- pkgs/os-specific/linux/kernel/patches.nix | 11 ---- pkgs/top-level/all-packages.nix | 6 -- 4 files changed, 119 deletions(-) delete mode 100644 pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_4_14.patch delete mode 100644 pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_5_3.patch diff --git a/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_4_14.patch b/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_4_14.patch deleted file mode 100644 index 205497aee2d..00000000000 --- a/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_4_14.patch +++ /dev/null @@ -1,60 +0,0 @@ -From 245e0f743d814c9ff2d1c748175e321301eb16cf Mon Sep 17 00:00:00 2001 -From: =?UTF-8?q?J=C3=B6rg=20Thalheim?= <joerg(a)thalheim.io> -Date: Thu, 2 May 2019 05:28:08 +0100 -Subject: [PATCH] x86/fpu: Export __kernel_fpu_{begin,end}() - -This partially undo commit: - -12209993 x86/fpu: Don't export __kernel_fpu_{begin,end}() - -We need this symbol in zfs for AES-NI/AVX support. ---- - arch/x86/include/asm/fpu/api.h | 2 ++ - arch/x86/kernel/fpu/core.c | 6 ++++-- - 2 files changed, 6 insertions(+), 2 deletions(-) - -diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h -index b56d504af6545..7d53388d266ea 100644 ---- a/arch/x86/include/asm/fpu/api.h -+++ b/arch/x86/include/asm/fpu/api.h -@@ -18,6 +18,8 @@ - * If you intend to use the FPU in softirq you need to check first with - * irq_fpu_usable() if it is possible. - */ -+extern void __kernel_fpu_begin(void); -+extern void __kernel_fpu_end(void); - extern void kernel_fpu_begin(void); - extern void kernel_fpu_end(void); - extern bool irq_fpu_usable(void); -diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c -index 2e5003fef51a9..2ea85b32421a0 100644 ---- a/arch/x86/kernel/fpu/core.c -+++ b/arch/x86/kernel/fpu/core.c -@@ -93,7 +93,7 @@ bool irq_fpu_usable(void) - } - EXPORT_SYMBOL(irq_fpu_usable); - --static void __kernel_fpu_begin(void) -+void __kernel_fpu_begin(void) - { - struct fpu *fpu = &current->thread.fpu; - -@@ -111,8 +111,9 @@ static void __kernel_fpu_begin(void) - __cpu_invalidate_fpregs_state(); - } - } -+EXPORT_SYMBOL(__kernel_fpu_begin); - --static void __kernel_fpu_end(void) -+void __kernel_fpu_end(void) - { - struct fpu *fpu = &current->thread.fpu; - -@@ -121,6 +122,7 @@ static void __kernel_fpu_end(void) - - kernel_fpu_enable(); - } -+EXPORT_SYMBOL(__kernel_fpu_end); - - void kernel_fpu_begin(void) - { diff --git a/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_5_3.patch b/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_5_3.patch deleted file mode 100644 index dc9ca64bdc1..00000000000 --- a/pkgs/os-specific/linux/kernel/export_kernel_fpu_functions_5_3.patch +++ /dev/null @@ -1,42 +0,0 @@ -From 1e010beda2896bdf3082fb37a3e49f8ce20e04d8 Mon Sep 17 00:00:00 2001 -From: =?UTF-8?q?J=C3=B6rg=20Thalheim?= <joerg(a)thalheim.io> -Date: Thu, 2 May 2019 05:28:08 +0100 -Subject: [PATCH] x86/fpu: Export kernel_fpu_{begin,end}() with - EXPORT_SYMBOL_GPL -MIME-Version: 1.0 -Content-Type: text/plain; charset=UTF-8 -Content-Transfer-Encoding: 8bit - -We need these symbols in zfs as the fpu implementation breaks userspace: - -https://github.com/zfsonlinux/zfs/issues/9346 -Signed-off-by: Jörg Thalheim <joerg(a)thalheim.io> ---- - arch/x86/kernel/fpu/core.c | 4 ++-- - 1 file changed, 2 insertions(+), 2 deletions(-) - -diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c -index 12c70840980e..352538b3bb5d 100644 ---- a/arch/x86/kernel/fpu/core.c -+++ b/arch/x86/kernel/fpu/core.c -@@ -102,7 +102,7 @@ void kernel_fpu_begin(void) - } - __cpu_invalidate_fpregs_state(); - } --EXPORT_SYMBOL_GPL(kernel_fpu_begin); -+EXPORT_SYMBOL(kernel_fpu_begin); - - void kernel_fpu_end(void) - { -@@ -111,7 +111,7 @@ void kernel_fpu_end(void) - this_cpu_write(in_kernel_fpu, false); - preempt_enable(); - } --EXPORT_SYMBOL_GPL(kernel_fpu_end); -+EXPORT_SYMBOL(kernel_fpu_end); - - /* - * Save the FPU state (mark it for reload if necessary): --- -2.23.0 - diff --git a/pkgs/os-specific/linux/kernel/patches.nix b/pkgs/os-specific/linux/kernel/patches.nix index 8ce1ac2b587..678cec34b70 100644 --- a/pkgs/os-specific/linux/kernel/patches.nix +++ b/pkgs/os-specific/linux/kernel/patches.nix @@ -76,17 +76,6 @@ }; }; - export_kernel_fpu_functions = { - "4.14" = { - name = "export_kernel_fpu_functions"; - patch = ./export_kernel_fpu_functions_4_14.patch; - }; - "5.3" = { - name = "export_kernel_fpu_functions"; - patch = ./export_kernel_fpu_functions_5_3.patch; - }; - }; - # patches from https://lkml.org/lkml/2019/7/15/1748 mac_nvme_t2 = rec { name = "mac_nvme_t2"; diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix index c22a62456db..6ba0505cb8a 100644 --- a/pkgs/top-level/all-packages.nix +++ b/pkgs/top-level/all-packages.nix @@ -17500,7 +17500,6 @@ in # when adding a new linux version kernelPatches.cpu-cgroup-v2."4.11" kernelPatches.modinst_arg_list_too_long - kernelPatches.export_kernel_fpu_functions."4.14" ]; }; @@ -17509,7 +17508,6 @@ in [ kernelPatches.bridge_stp_helper kernelPatches.request_key_helper kernelPatches.modinst_arg_list_too_long - kernelPatches.export_kernel_fpu_functions."4.14" ]; }; @@ -17517,7 +17515,6 @@ in kernelPatches = [ kernelPatches.bridge_stp_helper kernelPatches.request_key_helper - kernelPatches.export_kernel_fpu_functions."5.3" ]; }; @@ -17525,7 +17522,6 @@ in kernelPatches = [ kernelPatches.bridge_stp_helper kernelPatches.request_key_helper - kernelPatches.export_kernel_fpu_functions."5.3" ]; }; @@ -17533,7 +17529,6 @@ in kernelPatches = [ kernelPatches.bridge_stp_helper kernelPatches.request_key_helper - kernelPatches.export_kernel_fpu_functions."5.3" ]; }; @@ -17563,7 +17558,6 @@ in kernelPatches = [ kernelPatches.bridge_stp_helper kernelPatches.request_key_helper - kernelPatches.export_kernel_fpu_functions."5.3" ]; }; -- 2.31.1

1 3

[PATCH nixpkgs v2 0/8] chromiumOSPackages: 83.13020.0.0-rc1 -> 89.13729.0.0-rc1
by Alyssa Ross 02 Jun '21

02 Jun '21

Hi everyone, This is a continuation of the effort to update our chromiumOSPackages, which was started by Jamie a couple of months ago[1]. I've taken it over, and I think it should be good now. Specifically, since Jamie's original submission I: * Re-enabled xdg-shell in Sommelier, using the reverted upstream implementation[2] instead of Puck's. * Fixed the crosvm build by fixing its Cargo.lock, and stopping the boot test trying to open /dev/log, which doesn't exist in Nix builds. * Rebased a crosvm patch that was added to Spectrum in the meantime. * Removed a kernel patch that was added to Spectrum in the meantime but this update makes obsolete. I've tested that everything in spectrumPackages and chromiumOSPackages builds, and that spectrumPackages.spectrum-vm still runs. Jamie, thank you so much for getting us most of the way there with this -- you saved me a lot of time and I really appreciate it. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/0100017891d36f4f-0c34… [2]: https://spectrum-os.org/lists/archives/spectrum-devel/87wntkwq7z.fsf@alyssa… Alyssa Ross (3): chromiumOSPackages.sommelier: update patches crosvm: fix build spectrumPackages.linux: drop evged patch Jamie McClymont (5): chromiumOSPackages: 83.13020.0.0-rc1 -> 89.13729.0.0-rc1 chromiumOSPackages.linux: fix config chromiumOSPackages.common-mk: update patches chromiumOSPackages.vm_protos: drop GN patch chromiumOSPackages.common-mk: disable clang-only warnings ...-mk-don-t-leak-source-absolute-paths.patch | 115 +- ...ommon-mk-.gn-don-t-hardcode-env-path.patch | 8 +- .../linux/chromium-os/common-mk/default.nix | 6 +- .../crosvm/Regenerate-Cargo.lock.patch | 681 ++++++++++++ ...om-tap-fd.patch => VIRTIO_NET_F_MAC.patch} | 202 ++-- .../linux/chromium-os/crosvm/default.nix | 20 +- ...er-don-t-leak-source-absolute-paths.patch} | 10 +- ...tools-sommelier-Switch-to-the-stabl.patch} | 981 ++++++++++-------- ...mmelier-make-building-demos-optional.patch | 100 -- .../linux/chromium-os/sommelier/default.nix | 5 +- .../linux/chromium-os/upstream-info.json | 44 +- ...03-vm_tools-proto-fix-parallel-build.patch | 39 - .../linux/chromium-os/vm_protos/default.nix | 2 - pkgs/os-specific/linux/kernel/linux-cros.nix | 6 +- pkgs/os-specific/linux/kernel/patches.nix | 9 - pkgs/os-specific/linux/spectrum/linux/vm.nix | 3 - 16 files changed, 1445 insertions(+), 786 deletions(-) create mode 100644 pkgs/os-specific/linux/chromium-os/crosvm/Regenerate-Cargo.lock.patch rename pkgs/os-specific/linux/chromium-os/crosvm/{0001-crosvm-support-setting-guest-MAC-from-tap-fd.patch => VIRTIO_NET_F_MAC.patch} (56%) rename pkgs/os-specific/linux/chromium-os/sommelier/{0004-sommelier-don-t-leak-source-absolute-paths.patch => 0003-sommelier-don-t-leak-source-absolute-paths.patch} (69%) rename pkgs/os-specific/linux/chromium-os/sommelier/{0005-sommelier-use-stable-xdg-shell-protocol.patch => 0004-Revert-Revert-vm_tools-sommelier-Switch-to-the-stabl.patch} (66%) delete mode 100644 pkgs/os-specific/linux/chromium-os/sommelier/0006-sommelier-make-building-demos-optional.patch delete mode 100644 pkgs/os-specific/linux/chromium-os/vm_protos/0003-vm_tools-proto-fix-parallel-build.patch -- 2.31.1

1 9

[PATCH www] participating.html: Freenode -> Libera
by Alyssa Ross 30 May '21

30 May '21

The Matrix bridge is "in beta", which I think is good enough. --- participating.html | 44 ++++++++------------------------------------ 1 file changed, 8 insertions(+), 36 deletions(-) diff --git a/participating.html b/participating.html index 0e8ba65..7b47e8a 100644 --- a/participating.html +++ b/participating.html @@ -68,42 +68,14 @@ development</a>. <p> There is a single IRC -channel, <a href="ircs://chat.freenode.net:6997/spectrum">#spectrum</a> -on Freenode, for all kinds of Spectrum discussions. If you're new to -IRC, you might find -Freenode's <a href="https://webchat.freenode.net/#spectrum">web -chat</a> the easiest way to get started. The channel -is <strong>logged</strong>, and the logs -are <a href="https://logs.spectrum-os.org/spectrum/">published</a> on -the web. - -<p> -<strong>We will be moving -to <a href="https://libera.chat/">Libera.Chat</a> as soon as -a <a href="https://github.com/matrix-org/matrix-appservice-irc/issues/1324">Matrix -bridge</a> is available.</strong> - -<h3>Note for Matrix users</h3> - -<p> -There appears to be an issue with Matrix's IRC bridged room that prevents -Matrix users from joining the channel with the normal alias -#freenode_#spectrum:matrix.org. There are the following workarounds: - -<ul> - <li>Join the room via it's internal name: - <kbd>/join !ZsIiqZmCTSWHpTQWaw:matrix.org</kbd>. - <li>Join the room via - <a href="https://matrix.to/#/!ZsIiqZmCTSWHpTQWaw:matrix.org">matrix.to link</a>. - <li>Ask the bridges appservice to invite you to the room: - <kbd>/msg @appservice-irc:matrix.org !join #spectrum</kbd> and accept the invite. -</ul> - -<p> -If you experience this issue, kindly report it to the people who run -the Matrix IRC bridge. If you are able to join the channel without -the workaround, please report that in #spectrum so the website can be -updated. +channel, <a href="ircs://irc.libera.chat:6697/spectrum">#spectrum</a> +on <a href="https://libera.chat/">Libera.Chat</a>, for all kinds of +Spectrum discussions. You can also join +through <a href="https://matrix.org/">Matrix</a> +as <a href="https://matrix.to/#/#spectrum:libera.chat">#spectrum:libera.chat</a>. +If you're new to IRC, you might Matrix the easiest way to get started. +The channel is <strong>logged</strong>, but the logs are not available +online yet. <h2 id="mailing-lists">Mailing lists</h2> -- 2.31.1

2 3

[PATCH crosvm v2] crosvm: support setting guest MAC from tap-fd
by Alyssa Ross 30 May '21

30 May '21

This adds a mac= option to crosvm's --tap-fd option. The virtio-net driver in the guest will read the desired MAC from virtio configuration space. See the documentation for VIRTIO_NET_F_MAC in the Virtio spec[1]. [1]: https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html Thanks-to: Puck Meerburg <puck(a)puckipedia.com> --- v1: https://spectrum-os.org/lists/archives/spectrum-devel/20210409222026.13886-… This is a new version of the patch, updated for a much more recent crosvm. I'm posting it again because I had to change it enough that I feel it could use a re-review. The biggest change is that I gave up on a NetOptions struct, since upstream has added a new option, and it seems that their approach is going to be just passing ever more function arguments around. No use fighting that for my single addition. Also, change from RawFd to crosvm's RawDescriptor. It would be nice if we could use SafeDescriptor earlier here (which represents fd ownership), but I don't think that's really easily doable. devices/src/virtio/net.rs | 12 ++++++-- src/crosvm.rs | 8 +++-- src/linux.rs | 12 ++++---- src/main.rs | 63 ++++++++++++++++++++++++++++++--------- 4 files changed, 71 insertions(+), 24 deletions(-) diff --git a/devices/src/virtio/net.rs b/devices/src/virtio/net.rs index 92368c440..4d0ea1560 100644 --- a/devices/src/virtio/net.rs +++ b/devices/src/virtio/net.rs @@ -419,6 +419,7 @@ where } pub struct Net<T: TapT> { + mac_address: Option<MacAddress>, queue_sizes: Box<[u16]>, workers_kill_evt: Vec<Event>, kill_evts: Vec<Event>, @@ -439,6 +440,7 @@ where ip_addr: Ipv4Addr, netmask: Ipv4Addr, mac_addr: MacAddress, + guest_mac_addr: Option<MacAddress>, vq_pairs: u16, ) -> Result<Net<T>, NetError> { let multi_queue = if vq_pairs > 1 { true } else { false }; @@ -450,12 +452,12 @@ where tap.enable().map_err(NetError::TapEnable)?; - Net::from(base_features, tap, vq_pairs) + Net::with_tap(base_features, tap, vq_pairs, guest_mac_addr) } /// Creates a new virtio network device from a tap device that has already been /// configured. - pub fn from(base_features: u64, tap: T, vq_pairs: u16) -> Result<Net<T>, NetError> { + pub fn with_tap(base_features: u64, tap: T, vq_pairs: u16, mac_address: Option<MacAddress>) -> Result<Net<T>, NetError> { let taps = tap.into_mq_taps(vq_pairs).map_err(NetError::TapOpen)?; // This would also validate a tap created by Self::new(), but that's a good thing as it @@ -488,7 +490,12 @@ where workers_kill_evt.push(worker_kill_evt); } + if mac_address.is_some() { + avail_features |= 1 << virtio_net::VIRTIO_NET_F_MAC; + } + Ok(Net { + mac_address, queue_sizes: vec![QUEUE_SIZE; (vq_pairs * 2 + 1) as usize].into_boxed_slice(), workers_kill_evt, kill_evts, @@ -503,6 +510,7 @@ where let vq_pairs = self.queue_sizes.len() as u16 / 2; VirtioNetConfig { + mac: self.mac_address.map(|m| m.octets()).unwrap_or_else(Default::default), max_vq_pairs: Le16::from(vq_pairs), // Other field has meaningful value when the corresponding feature // is enabled, but all these features aren't supported now. diff --git a/src/crosvm.rs b/src/crosvm.rs index 04e267d6c..1dde7d769 100644 --- a/src/crosvm.rs +++ b/src/crosvm.rs @@ -171,6 +171,10 @@ impl Default for SharedDir { } } +pub struct TapFdOptions { + pub mac: Option<net_util::MacAddress>, +} + /// Aggregate of all configurable options for a running VM. pub struct Config { pub vcpu_count: Option<usize>, @@ -194,7 +198,7 @@ pub struct Config { pub mac_address: Option<net_util::MacAddress>, pub net_vq_pairs: Option<u16>, pub vhost_net: bool, - pub tap_fd: Vec<RawFd>, + pub tap_fd: BTreeMap<RawFd, TapFdOptions>, pub cid: Option<u64>, pub wayland_socket_paths: BTreeMap<String, PathBuf>, pub wayland_dmabuf: bool, @@ -253,7 +257,7 @@ impl Default for Config { mac_address: None, net_vq_pairs: None, vhost_net: false, - tap_fd: Vec::new(), + tap_fd: BTreeMap::new(), cid: None, #[cfg(feature = "gpu")] gpu_parameters: None, diff --git a/src/linux.rs b/src/linux.rs index f452fef3f..42c07df4f 100644 --- a/src/linux.rs +++ b/src/linux.rs @@ -75,7 +75,7 @@ use vm_memory::{GuestAddress, GuestMemory}; #[cfg(all(target_arch = "x86_64", feature = "gdb"))] use crate::gdb::{gdb_thread, GdbStub}; -use crate::{Config, DiskOption, Executable, SharedDir, SharedDirKind, TouchDeviceOption}; +use crate::{Config, DiskOption, Executable, SharedDir, SharedDirKind, TapFdOptions, TouchDeviceOption}; use arch::{ self, LinuxArch, RunnableLinuxVm, SerialHardware, SerialParameters, VcpuAffinity, VirtioDeviceStub, VmComponents, VmImage, @@ -675,7 +675,7 @@ fn create_balloon_device(cfg: &Config, socket: BalloonControlResponseSocket) -> }) } -fn create_tap_net_device(cfg: &Config, tap_fd: RawDescriptor) -> DeviceResult { +fn create_tap_net_device(cfg: &Config, tap_fd: RawDescriptor, options: &TapFdOptions) -> DeviceResult { // Safe because we ensure that we get a unique handle to the fd. let tap = unsafe { Tap::from_raw_descriptor( @@ -691,7 +691,7 @@ fn create_tap_net_device(cfg: &Config, tap_fd: RawDescriptor) -> DeviceResult { vq_pairs = 1; } let features = virtio::base_features(cfg.protected_vm); - let dev = virtio::Net::from(features, tap, vq_pairs).map_err(Error::NetDeviceNew)?; + let dev = virtio::Net::with_tap(features, tap, vq_pairs, options.mac).map_err(Error::NetDeviceNew)?; Ok(VirtioDeviceStub { dev: Box::new(dev), @@ -725,7 +725,7 @@ fn create_net_device( .map_err(Error::VhostNetDeviceNew)?; Box::new(dev) as Box<dyn VirtioDevice> } else { - let dev = virtio::Net::<Tap>::new(features, host_ip, netmask, mac_address, vq_pairs) + let dev = virtio::Net::<Tap>::new(features, host_ip, netmask, mac_address, None, vq_pairs) .map_err(Error::NetDeviceNew)?; Box::new(dev) as Box<dyn VirtioDevice> }; @@ -1311,8 +1311,8 @@ fn create_virtio_devices( devs.push(create_balloon_device(cfg, balloon_device_socket)?); // We checked above that if the IP is defined, then the netmask is, too. - for tap_fd in &cfg.tap_fd { - devs.push(create_tap_net_device(cfg, *tap_fd)?); + for (tap_fd, options) in &cfg.tap_fd { + devs.push(create_tap_net_device(cfg, *tap_fd, options)?); } if let (Some(host_ip), Some(netmask), Some(mac_address)) = diff --git a/src/main.rs b/src/main.rs index 5d02af02f..f8bc0d14e 100644 --- a/src/main.rs +++ b/src/main.rs @@ -28,7 +28,7 @@ use base::{ }; use crosvm::{ argument::{self, print_help, set_arguments, Argument}, - platform, BindMount, Config, DiskOption, Executable, GidMap, SharedDir, TouchDeviceOption, + platform, BindMount, Config, DiskOption, Executable, GidMap, SharedDir, TapFdOptions, TouchDeviceOption, DISK_ID_LEN, }; #[cfg(feature = "gpu")] @@ -1319,17 +1319,52 @@ fn set_argument(cfg: &mut Config, name: &str, value: Option<&str>) -> argument:: } "vhost-net" => cfg.vhost_net = true, "tap-fd" => { - cfg.tap_fd.push( - value - .unwrap() - .parse() - .map_err(|_| argument::Error::InvalidValue { - value: value.unwrap().to_owned(), - expected: String::from( - "this value for `tap-fd` must be an unsigned integer", - ), - })?, - ); + let mut components = value.unwrap().split(','); + + let fd: RawDescriptor = components + .next() + .and_then(|x| x.parse().ok()) + .ok_or_else(|| argument::Error::InvalidValue { + value: value.unwrap().to_owned(), + expected: String::from("this value for `tap-fd` must be an unsigned integer"), + })?; + + let mut mac = None; + for c in components { + let mut kv = c.splitn(2, '='); + let (kind, value) = match (kv.next(), kv.next()) { + (Some(kind), Some(value)) => (kind, value), + _ => { + return Err(argument::Error::InvalidValue { + value: c.to_owned(), + expected: String::from("option must be of the form `kind=value`"), + }) + } + }; + match kind { + "mac" => { + mac = Some(value.parse().map_err(|_| argument::Error::InvalidValue { + value: value.to_owned(), + expected: String::from( + "`mac` needs to be in the form \"XX:XX:XX:XX:XX:XX\"", + ), + })?) + } + _ => { + return Err(argument::Error::InvalidValue { + value: kind.to_owned(), + expected: String::from("unrecognized option"), + }) + } + } + } + if cfg.tap_fd.contains_key(&fd) { + return Err(argument::Error::TooManyArguments(format!( + "TAP FD already used: '{}'", + name + ))); + } + cfg.tap_fd.insert(fd, TapFdOptions { mac }); } #[cfg(feature = "gpu")] "gpu" => { @@ -1644,8 +1679,8 @@ writeback=BOOL - Indicates whether the VM can use writeback caching (default: fa Argument::value("plugin-gid-map-file", "PATH", "Path to the file listing supplemental GIDs that should be mapped in plugin jail. Can be given more than once."), Argument::flag("vhost-net", "Use vhost for networking."), Argument::value("tap-fd", - "fd", - "File descriptor for configured tap device. A different virtual network card will be added each time this argument is given."), + "FD[,mac=MAC]", + "File descriptor for configured tap device. A different virtual network card will be added each time this argument is given. MAC is the MAC address that will be set in the guest."), #[cfg(feature = "gpu")] Argument::flag_or_value("gpu", "[width=INT,height=INT]", base-commit: f35d2c43ff19520855cffee761dc8899c5a439a1 -- 2.31.1

2 5

[PATCH www] participating.html: add note about leaving Freenode
by Alyssa Ross 26 May '21

26 May '21

--- participating.html | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/participating.html b/participating.html index d04c2e5..aed91ab 100644 --- a/participating.html +++ b/participating.html @@ -77,6 +77,12 @@ is <strong>logged</strong>, and the logs are <a href="https://logs.spectrum-os.org/spectrum/">published</a> on the web. +<p> +<strong>We will be moving +to <a href="https://libera.chat/">Libera.Chat</a> as soon as +a <a href="https://github.com/matrix-org/matrix-appservice-irc/issues/1324">Matrix +bridge</a> is available. + <h3>Note for Matrix users</h3> <p> -- 2.31.0

2 1

[PATCH nixpkgs] spectrumPackages: use link-local router addresses
by Alyssa Ross 14 May '21

14 May '21

There's no need for every interface on a router to have it's own unique IP. Using link-local addresses, we can assign every interface on a single router the same IP, and save the effort of adding/subtracting one all the time to infer the address of one side of a link from the other. This means that routers and clients are both only interested in what the IP of the client should be, since that's the only thing that is different between different links. So we change our MAC format to communicate the client IP to both sides. If we just made the whole NIC portion of the MAC the last three octets of an IP address like we did before, then both ends of the link would have the same MAC, so we still need to set a bit to encode whether a MAC is at the router side of the link or the client side. I've decided to simplify the implementation by using the most significant bit of the NIC part for that, and just declaring the rest of that octet to be reserved. This way, no calculation is required in the guests -- they just need to look at the last two octets. And we can still identify 65536 links in this scheme, which should be enough for anybody, right? --- I'd like to move the networking to be IPv6. That's a bit more complicated, but shouldn't get in the way of smaller improvements like this. .../linux/spectrum/testhost/default.nix | 25 ++++++++----------- .../linux/spectrum/vm/app/default.nix | 15 ++++------- .../linux/spectrum/vm/net/default.nix | 13 +++++----- 3 files changed, 23 insertions(+), 30 deletions(-) diff --git a/pkgs/os-specific/linux/spectrum/testhost/default.nix b/pkgs/os-specific/linux/spectrum/testhost/default.nix index 21c585f1490..de62f0add67 100644 --- a/pkgs/os-specific/linux/spectrum/testhost/default.nix +++ b/pkgs/os-specific/linux/spectrum/testhost/default.nix @@ -45,23 +45,20 @@ let if { ip link set br0 up } # Calculate the MACs for our TAP and the router's TAP. - backtick -in router_nic_dec { - expr ${toString sys-vms.app.vmID} * 2 + 64 * 256 * 256 - } - backtick -in client_nic_dec { - expr ${toString sys-vms.app.vmID} * 2 + 64 * 256 * 256 + 1 - } - multisubstitute { - importas -iu router_nic_dec router_nic_dec - importas -iu client_nic_dec client_nic_dec - } + # MAC address format, by octet: + # + # 0-3 Static OUI for Spectrum + # 4 Most significant bit is used to differentiate + # routers from clients. Other bits are reserved. + # 5-6 Last two octets of client's IP (in 100.64.0.0/16). + # backtick -i router_mac { - pipeline { printf %x $router_nic_dec } - sed s/^\$..\$\$..\$\$..\$$/0A:B3:EC:\\1:\\2:\\3/ + pipeline { printf %.4x ${toString sys-vms.app.vmID} } + sed s/^\$..\$\$..\$$/0A:B3:EC:80:\\1:\\2/ } backtick -i client_mac { - pipeline { printf %x $client_nic_dec } - sed s/^\$..\$\$..\$\$..\$$/0A:B3:EC:\\1:\\2:\\3/ + pipeline { printf %.4x ${toString sys-vms.app.vmID} } + sed s/^\$..\$\$..\$$/0A:B3:EC:00:\\1:\\2/ } multisubstitute { importas -iu router_mac router_mac diff --git a/pkgs/os-specific/linux/spectrum/vm/app/default.nix b/pkgs/os-specific/linux/spectrum/vm/app/default.nix index 65dbb51f5e1..f8ff480932c 100644 --- a/pkgs/os-specific/linux/spectrum/vm/app/default.nix +++ b/pkgs/os-specific/linux/spectrum/vm/app/default.nix @@ -28,20 +28,15 @@ runCommand "vm-app" rec { up = writeText "net-up" '' backtick -i LOCAL_IP { pipeline { ip -j link show eth0 } - pipeline { jq -r ".[0].address | split(\":\") | .[3:6] | \"0x\" + .[]" } - xargs printf "100.%d.%d.%d" + pipeline { jq -r ".[0].address | split(\":\") | .[4:6] | \"0x\" + .[]" } + xargs printf "100.64.%d.%d" } importas -iu LOCAL_IP LOCAL_IP - backtick -i REMOTE_IP { - jq -jn --arg localip $LOCAL_IP - "$localip | split(\".\") | .[3] |= tonumber - 1 | join(\".\")" - } - importas -iu REMOTE_IP REMOTE_IP - - if { ip address add ''${LOCAL_IP}/31 dev eth0 } + if { ip address add ''${LOCAL_IP}/32 dev eth0 } if { ip link set eth0 up } - ip route add default via $REMOTE_IP + if { ip route add 169.254.0.1 dev eth0 } + ip route add default via 169.254.0.1 dev eth0 ''; }; diff --git a/pkgs/os-specific/linux/spectrum/vm/net/default.nix b/pkgs/os-specific/linux/spectrum/vm/net/default.nix index 1deb7031caf..5921b62fcf7 100644 --- a/pkgs/os-specific/linux/spectrum/vm/net/default.nix +++ b/pkgs/os-specific/linux/spectrum/vm/net/default.nix @@ -54,15 +54,16 @@ runCommand "vm-net" rec { # Our IP is encoded in the NIC-specific portion of the # interface's MAC address. - backtick -i LOCAL_IP { + backtick -i CLIENT_IP { pipeline { ip -j link show $INTERFACE } - pipeline { jq -r ".[0].address | split(\":\") | .[3:6] | \"0x\" + .[]" } - xargs printf "100.%d.%d.%d" + pipeline { jq -r ".[0].address | split(\":\") | .[4:6] | \"0x\" + .[]" } + xargs printf "100.64.%d.%d" } - importas -iu LOCAL_IP LOCAL_IP + importas -iu CLIENT_IP CLIENT_IP - if { ip address add ''${LOCAL_IP}/31 dev $INTERFACE } - ip link set $INTERFACE up + if { ip address add 169.254.0.1/32 dev $INTERFACE } + if { ip link set $INTERFACE up } + ip route add $CLIENT_IP dev $INTERFACE } { -- 2.31.1

2 1

[DEMO] virtio-vhost-user between QEMU and crosvm
by Alyssa Ross 13 May '21

13 May '21

Introduction ------------ Virtio-vhost-user[1] is a promising virtualisation technology that allows virtual devices that are exposed to VMs to themselves be implemented in VMs. Let's break down its name a bit to understand how it works: * Virtio[2] is a standard driver interface for virtualisation. Interfaces are available for all sorts of types of virtual devices, e.g. virtio-net, virtio-blk, and virtio-scsi. Typically, virtio devices are implemented by a virtual machine monitor (VMM). * Vhost[3] is kernelspace implementation of virtio virtual devices, created for their performance benefit. Instead of implementing the virtual devices itself, the VMM talks to the kernel implementation of them using a special ioctl protocol. * Vhost-user[4] allows another process to implement the vhost protocol, instead of the kernel, by using a UNIX socket instead of ioctls on a special character device. This doesn't provide the raw performance of vhost, but it serves a different purpose -- it allows virtual devices to be implemented by external programs, in a standardised way so they're portable between VMMs. Virtio-vhost-user allows the program implementing the virtual device to run in a VM of its own, by having the VMM for that VM create the vhost-user socket, and transferring messages over it to its guest using virtio. This is exciting for Spectrum, because it would mean that the host system doesn't have to interact with physical hardware directly beyond the PCI level, and can instead pass it through to a VM, which is responsible for implementing the virtual device backed by that physical hardware, which can be exposed to other VMs. Last year I spent a while looking into virtio-vhost-user[5][6][7]. It's a long way from being ready to use, and it seems to be maturing very slowly. It might be useful to us eventually for driver isolation, or something else might come along. My conclusion from my research was that we should decide later, once the ecosystem has had a chance to develop. But I wanted something to come out of the research I did anyway, and so I've prepared a demonstration. [1]: https://wiki.qemu.org/Features/VirtioVhostUser [2]: https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html [3]: https://blog.vmsplice.net/2011/09/qemu-internals-vhost-architecture.html [4]: https://qemu.readthedocs.io/en/latest/interop/vhost-user.html [5]: https://spectrum-os.org/lists/archives/spectrum-devel/87pn8rezqn.fsf@alyssa… [6]: https://spectrum-os.org/lists/archives/spectrum-devel/87blk1pwph.fsf@alyssa… [7]: https://spectrum-os.org/lists/archives/spectrum-devel/87wo2glkg0.fsf@alyssa… What the demo does ------------------ Using any sort of non-host-based virtual device implementation is going to have to start with taking the virtual device implementation out of the VMM running an application VM, and vhost-user the clear solution to this. Vhost-user isn't supported by crosvm -- its focus is on doing all the virtualisation required by Chromium OS, so there's no need for it to allow other programs to provide virtual device implementations. So another part of the research I did was to try to port the vhost-user implementation from cloud-hypervisor to crosvm, which I was able to do successfully[8]. This has implications beyond vhost-user, and even beyond crosvm, because it demonstrates that it's practical to port features between rust-vmm[9] VMMs, which means we don't have to worry about finding one that provides every feature we need (which is just as well, because there isn't one). So here I demonstrate not just a "standard" virtio-vhost-user setup (to the extent that such a thing can exist at this early stage), but also that my patched crosvm with vhost-user support is capable of interoperating with the experimental virtio-vhost-user implementation for QEMU, because both are speaking the standardised vhost-user protocol. The demo sets up two VMs. One is run with my patched crosvm, and expects a virtual ethernet device to be provided by a vhost-user socket. When it boots, it brings up its network interface, tries to run a DHCP client, and then exits. The other VM is run with Nikos Dragazis and Stefan Hajnoczi's experimental virtio-vhost-user implementation in QEMU[10]. It gets a standard virtual ethernet device (backed by a TAP device on the host), and the virtio-vhost-user device hooked up to the socket that crosvm will be connecting to. Inside the VM, a userspace networking stack (DPDK, again modified to support virtio-vhost-user by Nikos Dragazis[11]) implements the device side of virtio-vhost-user, and forwards packets sent by crosvm's guest to the virtual ethernet device backed by the host TAP. +-----------------------------------------------------------------------------+ | | | +------------------------+ +------------------------+ | | | | | | | | | +----------------+ | | +----------------+ | | | | | | | | | | | | | +-----+ | | +--------+ | | | | | | | | | TAP +------+---+---+ DPDK +---+---+------+---+ | | | | +-----+ | | +--------+ | | | | | | | | | | | | | | | | | | | | Linux | | | | Linux | | | | | +----------------+ | | +----------------+ | | | | | | | | | | QEMU | | crosvm | | | +------------------------+ +------------------------+ | | | | Linux | +-----------------------------------------------------------------------------+ A complicating factor is that the virtio-vhost-user implementation for DPDK only supports outgoing traffic[12]. So packets coming from crosvm will be relayed to the TAP, but not the other way around. This means that we can't just use ping inside the crosvm VM to verify that the connection is working. Instead, we have to tcpdump on the host and verify that the packets the DHCP client inside the crosvm VM is sending are arriving on the TAP. For this to be useful for our intended purpose of isolating drivers for physical devices, we'd pass through the device here rather than using a TAP. It would otherwise work exactly the same, but it's more difficult to test it's working correctly. (I have tested it though -- for the first version of this I got working last year, I verified it worked by checking the logs of my local network's DHCP server.) [8]: https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540… [9]: https://github.com/rust-vmm [10]: https://github.com/ndragazis/qemu/tree/virtio-vhost-user [11]: https://github.com/ndragazis/dpdk-next-virtio/tree/virtio-vhost-user [12]: https://github.com/ndragazis/dpdk-next-virtio/blob/2d60e63/drivers/virtio_v… Running the demo ---------------- First, create a TAP device for QEMU to use: # ip tuntap add qemutap mode tap # ip link set qemutap up Start tcpdump, so we can see if packets arrive on the TAP: # tcpdump -i qemutap Start the QEMU VM: $ $(nix-build -A qemuVm /path/to/demo.nix) When you see "Press enter to exit", DPDK is ready to receive a virtio-vhost-user connection. Start the crosvm VM: $ $(nix-build -A crosvmVm /path/to/demo.nix) Once that VM boots, you should see some "BOOTP/DHCP" lines in the tcpdump output. This demonstrates that traffic from the crosvm guest has been relayed over virtio-vhost-user to DPDK, and then to the TAP on the host over virtio-net. You'll want to press enter to shut down the QEMU VM now, because DPDK pegs a CPU core (for reasons[*] unrelated to virtio-vhost-user that are out of scope here). Then you can remove the TAP device: # ip link delete qemutap Nix expression for the demo --------------------------- # SPDX-License-Identifier: MIT OR Apache-2.0 # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> let pinned = builtins.fetchTarball { url = "https://github.com/NixOS/nixpkgs/tarball/b14062b75c4e8ef4dd4110282f7105be87…"; sha256 = "1hzs0w6pcwwbzl2gkqyk46yrzizzm03mph4kggws02a6vlwphsib"; }; in { pkgs ? import pinned {} }: with pkgs; rec { linux = pkgs.linux.override { structuredExtraConfig = with lib.kernel; { "9P_FS" = yes; NET_9P = yes; NET_9P_VIRTIO = yes; PACKET = yes; VFIO = yes; VFIO_NOIOMMU = yes; VFIO_PCI = yes; VIRTIO_NET = yes; VIRTIO_PCI = yes; }; }; dpdk = stdenv.mkDerivation { name = "dpdk-virtio-vhost-user"; src = fetchFromGitHub { owner = "ndragazis"; repo = "dpdk-next-virtio"; rev = "0a46582dc1d02c0dc5069347ffff1a64239385f2"; sha256 = "169cxdps9k764jj420q44262x3291h2jcqsbrh7038hqjczjkgif"; }; buildInputs = [ numactl ]; configurePhase = '' runHook preConfigure make $makeFlags defconfig runHook postConfigure ''; enableParallelBuilding = true; RTE_KERNELDIR = "${linux.dev}/lib/modules/${linux.modDirVersion}/build"; NIX_CFLAGS_COMPILE = [ "-Wno-error=implicit-fallthrough" "-Wno-error=incompatible-pointer-types" ]; makeFlags = [ "RTE_OUTPUT=$(out)/lib" "kerneldir=$(out)/lib/modules/${linux.modDirVersion}/build" "prefix=$(out)" ]; inherit (pkgs.dpdk) meta; }; # DPDK is huge! We just need one program from it. testpmd = runCommandNoCC "testpmd" {} '' mkdir -p $out/bin cp ${dpdk}/bin/testpmd $out/bin ''; # qemu has changed build system since the virtio-vhost-user branch # was last updated, so it's simpler to just make a new derivation # and inherit the bits that are the same than to override the # existing one. qemu = stdenv.mkDerivation { name = "qemu-virtio-vhost-user"; src = fetchFromGitHub { owner = "ndragazis"; repo = "qemu"; rev = "f9ab08c0c8cfc58036ed95b895f9780397448071"; sha256 = "0p6v4i7gj70d6x7s28x3i3x9z8vlswcbbqdwfbhlx87bbnxjrn3b"; fetchSubmodules = true; }; enableParallelBuilding = true; nativeBuildInputs = lib.subtractLists [ ninja meson ] qemu_kvm.nativeBuildInputs; postPatch = '' sed -i '/$(INSTALL_DIR) "$(DESTDIR)$(qemu_localstatedir)/d' Makefile # The virtio-vhost-user implementation tries to allocate a huge # PCI bar, that's bigger than some CPUs can support! If you see # a kernel panic in vp_reset(), lower this further. substituteInPlace hw/virtio/virtio-vhost-user-pci.c \ --replace '1ULL << 36' '1ULL << 34' ''; inherit (qemu_kvm) buildInputs configureFlags meta; }; qemuInitramfs = makeInitrd { contents = [ { symlink = "/init"; object = writeScript "init" '' #!${busybox}/bin/sh -eux export PATH=${busybox}/bin mkdir -p /nix/store /run /var mount -t sysfs none /sys mount -t proc none /proc mount -t tmpfs none /run mount -t devtmpfs none /dev mkdir /dev/hugepages mount -t hugetlbfs none /dev/hugepages ln -s /run /var # Unbind the virtio-net (host TAP) and virtio-vhost-user devices # from their default drivers, since we'll be passing them # through to DPDK. echo 0000:00:04.0 > /sys/bus/pci/devices/0000:00:04.0/driver/unbind echo 0000:00:05.0 > /sys/bus/pci/devices/0000:00:05.0/driver/unbind # Tell the vfio-pci driver it can support virtio-net and # virtio-vhost-user devices. Since our devices are not # bound to any driver at the moment, doing this will bind # them to vfio-pci automatically. echo 1af4 1000 > /sys/bus/pci/drivers/vfio-pci/new_id echo 1af4 1017 > /sys/bus/pci/drivers/vfio-pci/new_id echo 256 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages ${testpmd}/bin/testpmd \ -l 0-1 \ -w 0000:00:05.0 \ --vdev net_vhost0,iface=0000:00:05.0,virtio-transport=1 \ -w 0000:00:04.0 poweroff -f ''; } ]; }; qemuVm = writeShellScript "qemu-vm" '' exec ${qemu}/bin/qemu-system-x86_64 -enable-kvm -cpu host -smp 2 -m 1G \ -M q35,kernel-irqchip=split \ -initrd ${qemuInitramfs}/initrd \ -netdev tap,id=net0,ifname=qemutap,script=no,downscript=no \ -device virtio-net-pci,netdev=net0,addr=04 \ -chardev socket,id=chardev0,path="$XDG_RUNTIME_DIR/vhost-user0.sock",server,nowait \ -device virtio-vhost-user-pci,addr=05,chardev=chardev0 \ -kernel ${linux}/${stdenv.hostPlatform.linux-kernel.target} \ -append "console=ttyS0 vfio.enable_unsafe_noiommu_mode=1" \ -nographic ''; # Can't use overrideAttrs because of cargoSha256. crosvm = rustPlatform.buildRustPackage rec { name = "crosvm-virtio-vhost-user"; src = fetchFromGitiles { url = "https://chromium.googlesource.com/chromiumos/platform/crosvm"; rev = "8a7e4e902a4950b060ea23b40c0dfce7bfa1b2cb"; sha256 = "1lm6psp0xakb66nhgmmh94valc4wzbb967chk80msk8bcvsfpdn4"; }; unpackPhase = let origSrc = pkgs.crosvm.passthru.src; in builtins.replaceStrings [ "${origSrc}" origSrc.name ] [ "$src" src.name ] pkgs.crosvm.unpackPhase; cargoPatches = [ (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "0yzqrpgq35s9wxvbf9s3dgs5cpyxgdc5hr14hsdjr0gd18a6camg"; }) ]; patches = pkgs.crosvm.patches ++ [ (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "0g2rvqqa4lvq7bjq0s1ynsjx7lmrxql7lsdv8wyzb7d2z9j6mj13"; }) (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "051sz87i8kzc5sbygk2bpiqp4g32y9fxswg2yax1nd3lg4rxh43r"; }) (fetchpatch { url = "https://spectrum-os.org/lists/archives/spectrum-devel/20210512170812.192540…"; sha256 = "1jpas65masn2xg9jxha16vi0y7scarzhl221y9wxh4chi4aa4m3f"; }) ]; cargoSha256 = "07yizbhs64jrb05fq5g7sx812xbz2989bsficacq5l19ziax5164"; passthru = pkgs.crosvm.passthru // { inherit src; }; inherit (pkgs.crosvm) sourceRoot postPatch nativeBuildInputs buildInputs preBuild postInstall CROSVM_CARGO_TEST_KERNEL_BINARY meta; }; crosvmInitramfs = makeInitrd { contents = [ { symlink = "/init"; object = writeScript "init" '' #!${busybox}/bin/sh -eux export PATH=${busybox}/bin mount -t sysfs none /sys mount -t proc none /proc ip link set eth0 up udhcpc -n || : reboot -f ''; } ]; }; crosvmVm = writeShellScript "crosvm-vm" '' # In our patched crosvm, suppling --mac without --host_ip or # --netmask will put it into vhost-user mode. exec ${crosvm}/bin/crosvm run \ --mac 0A:B3:EC:FF:FF:FF \ -i ${crosvmInitramfs}/initrd \ ${linux}/${stdenv.hostPlatform.linux-kernel.target} ''; }

1 0

[RFC PATCH crosvm 0/4] Implement vhost-user-net
by Alyssa Ross 12 May '21

12 May '21

This is a proof of concept, so no need for code review on it. Here I have taken cloud-hypervisor's implementation of vhost-user-net, and ported it to crosvm. I based this on old versions of crosvm and cloud-hypervisor, with the idea being that it would be easier to get an initial working version from a time when the codebases had diverged less, which could then be gradually brought forward. A slight complication is that the cloud-hypervisor version I ported the implementation from didn't quite behave in the way the vhost-user specification[1] said it should, so patches 2 and 3 are backports of cloud-hypervisor fixes that happened later. Alyssa Ross (4): devices: port vhost-user-net from cloud-hypervisor vhost_rs: vhost_user: Add missing protocol features devices: vhost-user: Vring should be enabled after initialization crosvm: use vhost-user-net instead of virtio-net Cargo.lock | 731 ++++++++++------ Cargo.toml | 2 +- devices/Cargo.toml | 8 + devices/src/virtio/mod.rs | 2 + .../virtio/vhost_user/cloud_hypervisor/mod.rs | 1 + .../vhost_user/cloud_hypervisor/net_util.rs | 162 ++++ devices/src/virtio/vhost_user/handler.rs | 123 +++ devices/src/virtio/vhost_user/mod.rs | 82 ++ devices/src/virtio/vhost_user/net.rs | 230 +++++ .../src/virtio/vhost_user/vu_common_ctrl.rs | 110 +++ src/linux.rs | 32 +- src/main.rs | 2 +- vhost_rs/Cargo.toml | 26 + vhost_rs/LICENSE | 202 +++++ vhost_rs/LICENSE-BSD | 27 + vhost_rs/LICENSE-MIT | 24 + vhost_rs/README.md | 10 + vhost_rs/src/backend.rs | 130 +++ vhost_rs/src/lib.rs | 122 +++ vhost_rs/src/vhost_kern/mod.rs | 320 +++++++ vhost_rs/src/vhost_kern/vhost_binding.rs | 405 +++++++++ vhost_rs/src/vhost_kern/vsock.rs | 84 ++ vhost_rs/src/vhost_user/connection.rs | 737 ++++++++++++++++ vhost_rs/src/vhost_user/dummy_slave.rs | 250 ++++++ vhost_rs/src/vhost_user/master.rs | 757 ++++++++++++++++ vhost_rs/src/vhost_user/master_req_handler.rs | 258 ++++++ vhost_rs/src/vhost_user/message.rs | 823 ++++++++++++++++++ vhost_rs/src/vhost_user/mod.rs | 251 ++++++ vhost_rs/src/vhost_user/slave.rs | 48 + vhost_rs/src/vhost_user/slave_req_handler.rs | 582 +++++++++++++ vhost_rs/src/vhost_user/sock_ctrl_msg.rs | 464 ++++++++++ vhost_rs/src/vsock.rs | 30 + virtio-bindings/Cargo.toml | 6 + virtio-bindings/src/lib.rs | 15 + virtio-bindings/src/virtio_blk.rs | 486 +++++++++++ virtio-bindings/src/virtio_net.rs | 734 ++++++++++++++++ virtio-bindings/src/virtio_ring.rs | 453 ++++++++++ 37 files changed, 8461 insertions(+), 268 deletions(-) create mode 100644 devices/src/virtio/vhost_user/cloud_hypervisor/mod.rs create mode 100644 devices/src/virtio/vhost_user/cloud_hypervisor/net_util.rs create mode 100644 devices/src/virtio/vhost_user/handler.rs create mode 100644 devices/src/virtio/vhost_user/mod.rs create mode 100644 devices/src/virtio/vhost_user/net.rs create mode 100644 devices/src/virtio/vhost_user/vu_common_ctrl.rs create mode 100644 vhost_rs/Cargo.toml create mode 100644 vhost_rs/LICENSE create mode 100644 vhost_rs/LICENSE-BSD create mode 100644 vhost_rs/LICENSE-MIT create mode 100644 vhost_rs/README.md create mode 100644 vhost_rs/src/backend.rs create mode 100644 vhost_rs/src/lib.rs create mode 100644 vhost_rs/src/vhost_kern/mod.rs create mode 100644 vhost_rs/src/vhost_kern/vhost_binding.rs create mode 100644 vhost_rs/src/vhost_kern/vsock.rs create mode 100644 vhost_rs/src/vhost_user/connection.rs create mode 100644 vhost_rs/src/vhost_user/dummy_slave.rs create mode 100644 vhost_rs/src/vhost_user/master.rs create mode 100644 vhost_rs/src/vhost_user/master_req_handler.rs create mode 100644 vhost_rs/src/vhost_user/message.rs create mode 100644 vhost_rs/src/vhost_user/mod.rs create mode 100644 vhost_rs/src/vhost_user/slave.rs create mode 100644 vhost_rs/src/vhost_user/slave_req_handler.rs create mode 100644 vhost_rs/src/vhost_user/sock_ctrl_msg.rs create mode 100644 vhost_rs/src/vsock.rs create mode 100644 virtio-bindings/Cargo.toml create mode 100644 virtio-bindings/src/lib.rs create mode 100644 virtio-bindings/src/virtio_blk.rs create mode 100644 virtio-bindings/src/virtio_net.rs create mode 100644 virtio-bindings/src/virtio_ring.rs base-commit: 8a7e4e902a4950b060ea23b40c0dfce7bfa1b2cb -- 2.31.0

1 4

[PATCH nixpkgs] spectrumPackages.makeRootfs: fix messagebus group
by Alyssa Ross 22 Apr '21

22 Apr '21

/etc/passwd set the messagebus user's primary group to 1, but there was no group with that id in /etc/group. The messagebus group had id 4. The 4 comes from NixOS. I think it's probably a good idea to use the same user and group ids where possible -- there's no particular reason to diverge -- so let's fix it in /etc/passwd instead of changing the id in /etc/group to 1. And while we're here, for consistency, let's change the user id of messagebus to 4 as well, so that's consistent with NixOS as well. I don't think the user ID is used anywhere else. This fixes the DBus "Permission denied" errors that have been showing up in vm-net since forever (but never seemed to cause any real harm...). --- pkgs/os-specific/linux/spectrum/rootfs/etc/passwd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pkgs/os-specific/linux/spectrum/rootfs/etc/passwd b/pkgs/os-specific/linux/spectrum/rootfs/etc/passwd index c76fb8f2a99..ddb1f854a6e 100644 --- a/pkgs/os-specific/linux/spectrum/rootfs/etc/passwd +++ b/pkgs/os-specific/linux/spectrum/rootfs/etc/passwd @@ -1,3 +1,3 @@ root:x:0:0:System administrator:/:/bin/sh -messagebus:x:1:1:D-Bus system message bus daemon user:/run/dbus:/bin/sh +messagebus:x:4:4:D-Bus system message bus daemon user:/run/dbus:/bin/sh user:x:1000:1000:User:/:/bin/sh -- 2.31.0

3 4

[PATCH nixpkgs] spectrumPackages.spectrum-testhost: fix driver bind
by Alyssa Ross 22 Apr '21

22 Apr '21

The following operations are available to us in /sys/bus/pci/drivers/vfio-pci: bind, unbind, new_id, remove_id bind and unbind control attachment of a device to a driver. new_id and remove_id control whether a device should be considered "supported" by a driver. But when an unassigned device is newly supported by a driver, the kernel will automatically bind it to that driver. We were relying on the new_id operation binding the device to the driver in this way. But if the driver is in the supported list, but not bound to anything, this won't happen. new_id won't do anything, because it's already in the supported list. So we can't rely on new_id binding for us, and need to try bind as well (at least if new_id fails). To reproduce: echo 8086 1502 > /sys/bus/pci/drivers/vfio-pci/new_id echo 0000:00:19.0 > /sys/bus/pci/drivers/vfio-pci/unbind (Use lspci -nn to find the right values for your hardware.) This will leave you with a device that is supported by vfio-pci but not bound to it, which would previously cause vm-net to fail to start, but should not after this change. --- .../linux/spectrum/testhost/default.nix | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/pkgs/os-specific/linux/spectrum/testhost/default.nix b/pkgs/os-specific/linux/spectrum/testhost/default.nix index 7e1a973e8c6..21c585f1490 100644 --- a/pkgs/os-specific/linux/spectrum/testhost/default.nix +++ b/pkgs/os-specific/linux/spectrum/testhost/default.nix @@ -142,7 +142,9 @@ let printf "%s" $PCI_LOCATION } - # (Re)bind the device to the VFIO PCI driver. + # Tell the VFIO driver it should support our device. This + # is allowed to fail because it might already know that, in + # which case it'll return EEXIST. if { modprobe vfio-pci } backtick -in device_id { if { dd bs=2 skip=1 count=2 status=none if=''${PCI_PATH}/vendor } @@ -155,6 +157,20 @@ let printf "%s" $device_id } + # Bind the device to the VFIO driver. This is allowed to + # fail because the new_id operation we just tried will have + # bound it automatically for us if it succeeded. In such a + # case, the kernel will return ENODEV (conistency!). + foreground { + redirfd -w 1 /sys/bus/pci/drivers/vfio-pci/bind + printf "%s" $PCI_LOCATION + } + + # Because we allow both new_id and bind to fail, we need to + # manually make sure now that at least one of them succeeded + # and the device is actually attached to the vfio-driver. + if { test -e /sys/bus/pci/drivers/vfio-pci/''${PCI_LOCATION} } + foreground { mkdir env } ${cloud-hypervisor}/bin/cloud-hypervisor -- 2.31.0

2 2

[PATCH nixpkgs 00/16] Inter-guest networking
by Alyssa Ross 14 Apr '21

14 Apr '21

In Spectrum, we want the host kernel to include as few drivers as possible, to reduce attack service. To accomplish this, we need to move as much hardware interaction as possible into VMs. This series introduces proof-of-concept network hardware isolation by passing through network devices to a VM, and having that VM handle all interaction with that hardware instead of the host system. Background ---------- Ideally, the Spectrum host system wouldn't have to have support for networking at all. Network hardware could be handled by a VM, which would act a bit like a router, and export virtual network devices directly to other VMs. The hard part of that ideal is the bit where a VM exports a virtual device to another VM. There's work going on in the Linux virtualisation ecosystem to make that possible, through a protocol called virtio-vhost-user[1]. But it looks like it's going to be a long time before virtio-vhost-user is baked enough to be useful to us, and inter-guest networking is so fundamentally important to Spectrum that we can't afford to wait that long. So we need a plan B. Because we have no way to have a VM provide a virtual device, the host is going to need to do it. And this means that some amount of networking is going to have to happen on the host. But we can still get most of what we were going for: * Pass hardware network devices through to a dedicated VM. * Attach a virtual network device to the same VM. * Connect htat virtual network device to virtual network devices for other VMs on the host. With this approach, the host is still doing networking, but there's a lot less code involved, because instead of having loads of drivers for every kind of Wi-Fi card and so on available on the host, the host only ever needs to use the driver for the virtual network devices. If we connect each network client VM to the router VM using a bridge (i.e. a virtual Ethernet switch), we don't even need to worry about IP addresses or routing tables on the host. (There are some drawbacks to this approach, which are elaborated later in the series.) So that's what's implemented here. [1]: https://wiki.qemu.org/Features/VirtioVhostUser Implementation -------------- This series starts with some cherry-picks from upstream Nixpkgs. (We should sync with upstream soon, but I didn't want to delay this any longer by blocking it on a Nixpkgs sync.) Then there's some important refactoring to make it possible to fit multiple VM definitions into spectrumPackages. (Until now, there's only been one.) Finally, I implement two VMs -- one to act as the router and one to be the client -- as well as a service manager that can set everything up on the host to be able to run those VMs, and run them. Don't read too much into the structure of the Nix code for the VMs. I just needed _some_ structure, and this was what I came up with. As we explore the configuration aspect of Spectrum more, I expect it to change dramatically. There's quite a bit of duplication between VM definitions, but I don't think it's worth spending time getting rid of that for now when the entire structure could change before it becomes an issue. The important thing here is what's going on at runtime on the host and in the VMs, not the Nix code used to build that. Everything in the router VM is set up to be able to handle multiple clients coming and going at runtime (with the exception of a small issue with clients going away that I'll get into later in the series). The only reason the client VM is defined up front is because that's what fits our Nix code. There's no reason from the networking point of view that client VMs couldn't be instantiated entirely at runtime. One thing that might jump out to anybody skimming the series is that we're using cloud-hypervisor here. This is because there's a feature it supports that crosvm doesn't (I'll go into detail about what exactly later in the series), and it's more expedient to just use both VMMs as required by the needs of the VMs they're running than it would be to port features from cloud-hypervisor to crosvm or vice versa. Obviously using multiple different VMMs on the host is not good in the long term, but things are so in flux at the moment that it's very likely that any porting work will no longer be in use by the time we need to pare down to a single VMM anyway. Testing ------- If you want to try this out for yourself (and I encourage you to do so and reply to me with a Tested-by!), here's what you need to do: 1. Identify the PCI location of a physical Ethernet device on your system. `lspci -n' should help with this. 2. Modify pkgs/os-specific/linux/spectrum/testhost/default.nix to define PCI_LOCATION to be your one. In future, we can either have a configuration option for this sort of thing, or try to figure it out at runtime. 3. Start the service manager: sudo env XDG_RUNTIME_DIR=/run $(nix-build -A spectrumPackages.spectrum-testhost)/bin/spectrum-testhost A temporary directory will be created for service manager state. Its location will be printed to the terminal. 4. At another terminal, tell the service manager to start the "application VM". This is the client that will be connected to the router VM. s6-rc -u -l /run/spectrum.sy2huQuC3x/s6-rc/live change vm-app Remember to substitute in your temporary directory. You'll need the s6-rc command available (it's in Nixpkgs). This tells the service manager to start the service named vm-app. This service has a dependency on another VM called vm-net, which is the router VM that deals with the network hardware, so both will be started when you ask s6-rc to start vm-app. 5. The terminal running the service manager will be connected to the serial console of client VM. After a few seconds (because the router VM will probably have to sort out DHCP first), you should be able to ping hosts on the internet. Note that the client VM doesn't come with DNS set up. 6. Once you're done, you can tell the service manager to stop the VMs: s6-rc -da -l /run/spectrum.sy2huQuC3x/s6-rc/live change Again, remember to substitute your temporary directory. The service manager itself won't respond to ^C. I believe this is handled better in more recent versions of s6, which we'll get when we sync Nixpkgs. After this, it's a good idea to reboot, to restore all the network devices to their default state. Alyssa Ross (16): linux: enable Xen everywhere it can be cloud-hypervisor: 0.8.0 -> 0.14.1 mdevd: init at 0.1.3.0 spectrumPackages.linux_vm: fix cloud-hypervisor hotplug spectrumPackages.linux_vm: allow config overrides crosvm: support setting guest MAC from --tap-fd spectrumPackages: export makeRootfs spectrumPackages.rootfs: add s6-rc support spectrumPackages.rootfs: make /var/lib and /var/run spectrumPackages.rootfs: add dbus configuration spectrumPackages.rootfs: add connman dbus services spectrumPackages.sys-vms.comp: init spectrumPackages.makeRootfs: move to default.nix spectrumPackages.sys-vms.net: init spectrumPackages.sys-vms.app: init spectrumPackages.spectrum-testhost: init .../cargo-lock-vendor-fix.patch | 53 ---- .../cloud-hypervisor/default.nix | 15 +- ...upport-setting-guest-MAC-from-tap-fd.patch | 294 ++++++++++++++++++ .../linux/chromium-os/crosvm/default.nix | 1 + .../linux/kernel/common-config.nix | 13 +- pkgs/os-specific/linux/kernel/patches.nix | 9 + pkgs/os-specific/linux/mdevd/default.nix | 28 ++ pkgs/os-specific/linux/spectrum/default.nix | 6 +- pkgs/os-specific/linux/spectrum/linux/vm.nix | 7 +- .../linux/spectrum/rootfs/default.nix | 92 +++--- .../linux/spectrum/rootfs/etc/group | 1 + .../linux/spectrum/rootfs/etc/passwd | 1 + .../linux/spectrum/rootfs/generic.nix | 48 --- .../linux/spectrum/rootfs/rc-services.nix | 26 ++ .../linux/spectrum/rootfs/stage1.nix | 25 +- .../linux/spectrum/spectrum-vm/default.nix | 6 +- .../linux/spectrum/testhost/default.nix | 205 ++++++++++++ .../linux/spectrum/vm/app/default.nix | 63 ++++ .../linux/spectrum/vm/comp/default.nix | 86 +++++ .../os-specific/linux/spectrum/vm/default.nix | 9 + .../linux/spectrum/vm/net/default.nix | 165 ++++++++++ pkgs/top-level/aliases.nix | 6 + pkgs/top-level/all-packages.nix | 12 +- 23 files changed, 976 insertions(+), 195 deletions(-) delete mode 100644 pkgs/applications/virtualization/cloud-hypervisor/cargo-lock-vendor-fix.patch create mode 100644 pkgs/os-specific/linux/chromium-os/crosvm/0001-crosvm-support-setting-guest-MAC-from-tap-fd.patch create mode 100644 pkgs/os-specific/linux/mdevd/default.nix delete mode 100644 pkgs/os-specific/linux/spectrum/rootfs/generic.nix create mode 100644 pkgs/os-specific/linux/spectrum/rootfs/rc-services.nix create mode 100644 pkgs/os-specific/linux/spectrum/testhost/default.nix create mode 100644 pkgs/os-specific/linux/spectrum/vm/app/default.nix create mode 100644 pkgs/os-specific/linux/spectrum/vm/comp/default.nix create mode 100644 pkgs/os-specific/linux/spectrum/vm/default.nix create mode 100644 pkgs/os-specific/linux/spectrum/vm/net/default.nix -- 2.30.0

2 19

[PATCH crosvm] crosvm: support setting guest MAC from tap-fd
by Alyssa Ross 14 Apr '21

14 Apr '21

This adds a mac= option to crosvm's --tap-fd option. The virtio-net driver in the guest will read the desired MAC from virtio configuration space. See the documentation for VIRTIO_NET_F_MAC in the Virtio spec[1]. [1]: https://docs.oasis-open.org/virtio/virtio/v1.1/virtio-v1.1.html --- This will be important for host-based networking in Spectrum. devices/src/virtio/net.rs | 31 ++++++++++++++++--- src/crosvm.rs | 8 +++-- src/linux.rs | 20 +++++++----- src/main.rs | 64 ++++++++++++++++++++++++++++++--------- 4 files changed, 96 insertions(+), 27 deletions(-) diff --git a/devices/src/virtio/net.rs b/devices/src/virtio/net.rs index 44a39abd..fe71371f 100644 --- a/devices/src/virtio/net.rs +++ b/devices/src/virtio/net.rs @@ -22,7 +22,9 @@ use virtio_sys::virtio_net::{ }; use virtio_sys::{vhost, virtio_net}; -use super::{DescriptorError, Interrupt, Queue, Reader, VirtioDevice, Writer, TYPE_NET}; +use super::{ + copy_config, DescriptorError, Interrupt, Queue, Reader, VirtioDevice, Writer, TYPE_NET, +}; const QUEUE_SIZE: u16 = 256; const NUM_QUEUES: usize = 3; @@ -373,7 +375,13 @@ where } } +#[derive(Default)] +pub struct NetOptions { + pub guest_mac: Option<net_util::MacAddress>, +} + pub struct Net<T: TapT> { + config: Vec<u8>, workers_kill_evt: Option<EventFd>, kill_evt: EventFd, worker_thread: Option<thread::JoinHandle<Worker<T>>>, @@ -392,6 +400,7 @@ where ip_addr: Ipv4Addr, netmask: Ipv4Addr, mac_addr: MacAddress, + options: NetOptions, ) -> Result<Net<T>, NetError> { let tap: T = T::new(true).map_err(NetError::TapOpen)?; tap.set_ip_addr(ip_addr).map_err(NetError::TapSetIp)?; @@ -401,18 +410,18 @@ where tap.enable().map_err(NetError::TapEnable)?; - Net::from(tap) + Net::with_tap(tap, options) } /// Creates a new virtio network device from a tap device that has already been /// configured. - pub fn from(tap: T) -> Result<Net<T>, NetError> { + pub fn with_tap(tap: T, options: NetOptions) -> Result<Net<T>, NetError> { // This would also validate a tap created by Self::new(), but that's a good thing as it // would ensure that any changes in the creation procedure are matched in the validation. // Plus we still need to set the offload and vnet_hdr_size values. validate_and_configure_tap(&tap)?; - let avail_features = 1 << virtio_net::VIRTIO_NET_F_GUEST_CSUM + let mut avail_features = 1 << virtio_net::VIRTIO_NET_F_GUEST_CSUM | 1 << virtio_net::VIRTIO_NET_F_CSUM | 1 << virtio_net::VIRTIO_NET_F_CTRL_VQ | 1 << virtio_net::VIRTIO_NET_F_CTRL_GUEST_OFFLOADS @@ -422,8 +431,18 @@ where | 1 << virtio_net::VIRTIO_NET_F_HOST_UFO | 1 << vhost::VIRTIO_F_VERSION_1; + if options.guest_mac.is_some() { + avail_features |= 1 << virtio_net::VIRTIO_NET_F_MAC; + } + + let config = options + .guest_mac + .map(|mac| mac.octets().to_vec()) + .unwrap_or_default(); + let kill_evt = EventFd::new().map_err(NetError::CreateKillEventFd)?; Ok(Net { + config, workers_kill_evt: Some(kill_evt.try_clone().map_err(NetError::CloneKillEventFd)?), kill_evt, worker_thread: None, @@ -545,6 +564,10 @@ where } } + fn read_config(&self, offset: u64, data: &mut [u8]) { + copy_config(data, 0, self.config.as_slice(), offset); + } + fn activate( &mut self, mem: GuestMemory, diff --git a/src/crosvm.rs b/src/crosvm.rs index 81344c32..e69f2dfc 100644 --- a/src/crosvm.rs +++ b/src/crosvm.rs @@ -157,6 +157,10 @@ impl Default for SharedDir { } } +pub struct TapFdOptions { + pub mac: Option<net_util::MacAddress>, +} + /// Aggregate of all configurable options for a running VM. pub struct Config { pub vcpu_count: Option<u32>, @@ -177,7 +181,7 @@ pub struct Config { pub netmask: Option<net::Ipv4Addr>, pub mac_address: Option<net_util::MacAddress>, pub vhost_net: bool, - pub tap_fd: Vec<RawFd>, + pub tap_fd: BTreeMap<RawFd, TapFdOptions>, pub cid: Option<u64>, pub wayland_socket_paths: BTreeMap<String, PathBuf>, pub wayland_dmabuf: bool, @@ -224,7 +228,7 @@ impl Default for Config { netmask: None, mac_address: None, vhost_net: false, - tap_fd: Vec::new(), + tap_fd: BTreeMap::new(), cid: None, #[cfg(feature = "gpu")] gpu_parameters: None, diff --git a/src/linux.rs b/src/linux.rs index 3370c1e1..f7f78ad2 100644 --- a/src/linux.rs +++ b/src/linux.rs @@ -60,7 +60,9 @@ use vm_control::{ VmMemoryRequest, VmMemoryResponse, VmRunMode, }; -use crate::{Config, DiskOption, Executable, SharedDir, SharedDirKind, TouchDeviceOption}; +use crate::{ + Config, DiskOption, Executable, SharedDir, SharedDirKind, TapFdOptions, TouchDeviceOption, +}; use arch::{self, LinuxArch, RunnableLinuxVm, VirtioDeviceStub, VmComponents, VmImage}; #[cfg(any(target_arch = "arm", target_arch = "aarch64"))] @@ -586,14 +588,18 @@ fn create_balloon_device(cfg: &Config, socket: BalloonControlResponseSocket) -> }) } -fn create_tap_net_device(cfg: &Config, tap_fd: RawFd) -> DeviceResult { +fn create_tap_net_device(cfg: &Config, tap_fd: RawFd, options: &TapFdOptions) -> DeviceResult { // Safe because we ensure that we get a unique handle to the fd. let tap = unsafe { Tap::from_raw_fd(validate_raw_fd(tap_fd).map_err(Error::ValidateRawFd)?) .map_err(Error::CreateTapDevice)? }; - let dev = virtio::Net::from(tap).map_err(Error::NetDeviceNew)?; + let net_opts = virtio::NetOptions { + guest_mac: options.mac, + }; + + let dev = virtio::Net::with_tap(tap, net_opts).map_err(Error::NetDeviceNew)?; Ok(VirtioDeviceStub { dev: Box::new(dev), @@ -614,8 +620,8 @@ fn create_net_device( .map_err(Error::VhostNetDeviceNew)?; Box::new(dev) as Box<dyn VirtioDevice> } else { - let dev = - virtio::Net::<Tap>::new(host_ip, netmask, mac_address).map_err(Error::NetDeviceNew)?; + let dev = virtio::Net::<Tap>::new(host_ip, netmask, mac_address, Default::default()) + .map_err(Error::NetDeviceNew)?; Box::new(dev) as Box<dyn VirtioDevice> }; @@ -1006,8 +1012,8 @@ fn create_virtio_devices( devs.push(create_balloon_device(cfg, balloon_device_socket)?); // We checked above that if the IP is defined, then the netmask is, too. - for tap_fd in &cfg.tap_fd { - devs.push(create_tap_net_device(cfg, *tap_fd)?); + for (tap_fd, options) in &cfg.tap_fd { + devs.push(create_tap_net_device(cfg, *tap_fd, options)?); } if let (Some(host_ip), Some(netmask), Some(mac_address)) = diff --git a/src/main.rs b/src/main.rs index 3afca8e0..053af465 100644 --- a/src/main.rs +++ b/src/main.rs @@ -21,7 +21,8 @@ use arch::Pstore; use audio_streams::StreamEffect; use crosvm::{ argument::{self, print_help, set_arguments, Argument}, - linux, BindMount, Config, DiskOption, Executable, GidMap, SharedDir, TouchDeviceOption, + linux, BindMount, Config, DiskOption, Executable, GidMap, SharedDir, TapFdOptions, + TouchDeviceOption, }; #[cfg(feature = "gpu")] use devices::virtio::gpu::{GpuMode, GpuParameters}; @@ -1041,17 +1042,52 @@ fn set_argument(cfg: &mut Config, name: &str, value: Option<&str>) -> argument:: } "vhost-net" => cfg.vhost_net = true, "tap-fd" => { - cfg.tap_fd.push( - value - .unwrap() - .parse() - .map_err(|_| argument::Error::InvalidValue { - value: value.unwrap().to_owned(), - expected: String::from( - "this value for `tap-fd` must be an unsigned integer", - ), - })?, - ); + let mut components = value.unwrap().split(','); + + let fd: RawFd = components + .next() + .and_then(|x| x.parse().ok()) + .ok_or_else(|| argument::Error::InvalidValue { + value: value.unwrap().to_owned(), + expected: String::from("this value for `tap-fd` must be an unsigned integer"), + })?; + + let mut mac = None; + for c in components { + let mut kv = c.splitn(2, '='); + let (kind, value) = match (kv.next(), kv.next()) { + (Some(kind), Some(value)) => (kind, value), + _ => { + return Err(argument::Error::InvalidValue { + value: c.to_owned(), + expected: String::from("option must be of the form `kind=value`"), + }) + } + }; + match kind { + "mac" => { + mac = Some(value.parse().map_err(|_| argument::Error::InvalidValue { + value: value.to_owned(), + expected: String::from( + "`mac` needs to be in the form \"XX:XX:XX:XX:XX:XX\"", + ), + })?) + } + _ => { + return Err(argument::Error::InvalidValue { + value: kind.to_owned(), + expected: String::from("unrecognized option"), + }) + } + } + } + if cfg.tap_fd.contains_key(&fd) { + return Err(argument::Error::TooManyArguments(format!( + "TAP FD already used: '{}'", + name + ))); + } + cfg.tap_fd.insert(fd, TapFdOptions { mac }); } #[cfg(feature = "gpu")] "gpu" => { @@ -1295,8 +1331,8 @@ writeback=BOOL - Indicates whether the VM can use writeback caching (default: fa Argument::value("plugin-gid-map-file", "PATH", "Path to the file listing supplemental GIDs that should be mapped in plugin jail. Can be given more than once."), Argument::flag("vhost-net", "Use vhost for networking."), Argument::value("tap-fd", - "fd", - "File descriptor for configured tap device. A different virtual network card will be added each time this argument is given."), + "FD[,mac=MAC]", + "File descriptor for configured tap device. A different virtual network card will be added each time this argument is given. MAC is the MAC address that will be set in the guest."), #[cfg(feature = "gpu")] Argument::flag_or_value("gpu", "[width=INT,height=INT]", -- 2.27.0

2 3