- Spectrum Development - spectrum-os.org

[PATCH 0/6] Introduce a shared base for application VMs
by Alyssa Ross 19 Sep '22

19 Sep '22

Here are some changes I made months ago to enable work I never finished around building VMs inside the Spectrum system with Nix. I'm coming back to it now because it makes it much easier to add more applications for testing, and with upcoming features like Wayland and a persistent filesystem we're going to want more example applications. The idea here is to reduce duplication between application VMs, both in terms of source code size and output size. After this change, creating a new VM just requires writing a very small Nix file like this: { config ? import ../../../nix/eval-config.nix {} }: import ../../vm-lib/make-vm.nix { inherit config; } { name = "appvm-lynx"; providers.net = [ "netvm" ]; run = config.pkgs.pkgsStatic.callPackage ( { writeScript, lynx }: writeScript "run-lynx" '' #!/bin/execlineb -P ${lynx}/bin/lynx https://spectrum-os.org '' ) { }; } Rather than a whole big source tree as before, most of which was duplicated with every other application VM. When a VM generated this way is started, it gets two disk images. One is the shared base image, which is part of the Spectrum base system, and the other contains only the application-specific stuff: the run script, and any store path dependencies that are not already present in the base image. This means that the amount of storage required for each new application VM is substantially reduced. Of course, this isn't the only way to generate VMs. Monolithic VMs like we had before would still work, with some small adjustments for the new disk layout. I also see this fitting well into making it possible to configure extra VMs at build time. It doesn't directly help with that, but making it so that each VM doesn't need to provide everything itself will make creating external VMs easier when it does happen. In future we might want to apply a similar mechanism to service VMs, like netvm, but since we only have one of those so far, it's not clear which parts exactly would be duplicated, so I'm leaving it for now. Other future work is considering the impacts of the shared base image on guest isolation. Can guests observe whether reads of the shared base image hit the host page cache, or even an internal disk cache? At the moment I suspect that the base image doesn't have enough specialised code in it that there would be any interesting results, but it's worth thinking about if the shared image grows new functionality, whether it would be interesting to another guest to be able to observe whether those resources have previously been loaded or not. If this _does_ turn out to be a concern, it could be mitigated by simply copying the base image to temporary storage before booting a VM, and then booting the VM from the copy. Alyssa Ross (6): host/start-vm: support multiple block devices scripts/make-gpt.sh: add support for labels vm: build GPT images host/start-vm: boot using partition label release: rename from "img" img/app: extract from appvm-{lynx,catgirl} Documentation/creating-vms.adoc | 8 +- Documentation/getting-spectrum.adoc | 2 +- host/initramfs/extfs.nix | 11 +- host/rootfs/default.nix | 11 +- host/start-vm/start-vm.rs | 43 ++++-- {vm/app/lynx => img/app}/Makefile | 57 ++++---- {vm/app/catgirl => img/app}/bin | 0 {vm/app/catgirl => img/app}/default.nix | 22 ++-- img/app/etc/fstab | 8 ++ {vm/app/catgirl => img/app}/etc/init | 0 {vm/app/catgirl => img/app}/etc/mdev.conf | 0 {vm/app/lynx => img/app}/etc/mdev/iface | 2 +- {vm/app/catgirl => img/app}/etc/passwd | 0 .../catgirl => img/app}/etc/passwd.license | 0 {vm/app/catgirl => img/app}/etc/resolv.conf | 0 .../app}/etc/s6-linux-init/scripts/rc.init | 1 + .../s6-rc/lynx => img/app/etc/s6-rc/app}/run | 3 +- .../catgirl => img/app/etc/s6-rc/app}/type | 0 .../app/etc/s6-rc/app}/type.license | 0 .../etc/s6-rc/mdevd-coldplug/dependencies | 0 .../app}/etc/s6-rc/mdevd-coldplug/type | 0 .../etc/s6-rc/mdevd-coldplug/type.license | 0 .../app}/etc/s6-rc/mdevd-coldplug/up | 0 .../app}/etc/s6-rc/mdevd/notification-fd | 0 .../etc/s6-rc/mdevd/notification-fd.license | 0 .../catgirl => img/app}/etc/s6-rc/mdevd/run | 0 .../catgirl => img/app}/etc/s6-rc/mdevd/type | 0 .../app}/etc/s6-rc/mdevd/type.license | 0 .../app}/etc/s6-rc/ok-all/contents | 0 .../catgirl => img/app}/etc/s6-rc/ok-all/type | 0 .../app}/etc/s6-rc/ok-all/type.license | 0 .../app}/etc/ssl/certs/ca-certificates.crt | 0 {vm/app/lynx => img/app}/shell.nix | 11 +- release.nix | 2 +- {img => release}/combined/default.nix | 0 {img => release}/combined/eosimages.nix | 0 {img => release}/combined/grub.cfg.in | 0 {img => release}/combined/run-vm.nix | 0 ...ble-gpt-partition-attribute-55-check.patch | 0 ...pt-disable-partition-table-CRC-check.patch | 0 .../0003-install-remove-Endless-OS-ad.patch | 0 ...4-finished-don-t-run-eos-diagnostics.patch | 0 ...omote-spectrum-not-the-Endless-forum.patch | 0 {img => release}/installer/app/default.nix | 0 .../installer/app/vendor-customer-support.ini | 0 {img => release}/installer/configuration.nix | 0 {img => release}/installer/default.nix | 0 {img => release}/installer/run-vm.nix | 0 {img => release}/installer/seat.rules | 0 {img => release}/live/Makefile | 0 {img => release}/live/default.nix | 0 {img => release}/live/shell.nix | 0 scripts/make-gpt.sh | 4 +- scripts/sfdisk-field.awk | 2 +- vm-lib/make-vm.nix | 46 +++++++ vm/app/catgirl.nix | 18 +++ vm/app/catgirl/Makefile | 123 ------------------ vm/app/catgirl/etc/fstab | 6 - vm/app/catgirl/etc/mdev/iface | 36 ----- .../catgirl/etc/s6-linux-init/scripts/rc.init | 10 -- vm/app/catgirl/etc/s6-rc/catgirl/run | 31 ----- .../data/appvm-catgirl/providers/net/netvm | 0 vm/app/catgirl/shell.nix | 17 --- vm/app/lynx.nix | 16 +++ vm/app/lynx/bin | 1 - vm/app/lynx/default.nix | 92 ------------- vm/app/lynx/etc/fstab | 6 - vm/app/lynx/etc/init | 5 - vm/app/lynx/etc/mdev.conf | 5 - vm/app/lynx/etc/passwd | 1 - vm/app/lynx/etc/passwd.license | 2 - vm/app/lynx/etc/resolv.conf | 4 - vm/app/lynx/etc/s6-rc/lynx/type | 1 - vm/app/lynx/etc/s6-rc/lynx/type.license | 2 - .../etc/s6-rc/mdevd-coldplug/dependencies | 4 - vm/app/lynx/etc/s6-rc/mdevd-coldplug/type | 1 - .../etc/s6-rc/mdevd-coldplug/type.license | 2 - vm/app/lynx/etc/s6-rc/mdevd-coldplug/up | 4 - vm/app/lynx/etc/s6-rc/mdevd/notification-fd | 1 - .../etc/s6-rc/mdevd/notification-fd.license | 2 - vm/app/lynx/etc/s6-rc/mdevd/run | 5 - vm/app/lynx/etc/s6-rc/mdevd/type | 1 - vm/app/lynx/etc/s6-rc/mdevd/type.license | 2 - vm/app/lynx/etc/s6-rc/ok-all/contents | 4 - vm/app/lynx/etc/s6-rc/ok-all/type | 1 - vm/app/lynx/etc/s6-rc/ok-all/type.license | 2 - vm/app/lynx/etc/ssl/certs/ca-certificates.crt | 1 - .../host/data/appvm-lynx/providers/net/netvm | 0 vm/sys/net/Makefile | 23 ++-- vm/sys/net/default.nix | 10 +- 90 files changed, 221 insertions(+), 451 deletions(-) rename {vm/app/lynx => img/app}/Makefile (66%) rename {vm/app/catgirl => img/app}/bin (100%) rename {vm/app/catgirl => img/app}/default.nix (77%) create mode 100644 img/app/etc/fstab rename {vm/app/catgirl => img/app}/etc/init (100%) rename {vm/app/catgirl => img/app}/etc/mdev.conf (100%) rename {vm/app/lynx => img/app}/etc/mdev/iface (98%) rename {vm/app/catgirl => img/app}/etc/passwd (100%) rename {vm/app/catgirl => img/app}/etc/passwd.license (100%) rename {vm/app/catgirl => img/app}/etc/resolv.conf (100%) rename {vm/app/lynx => img/app}/etc/s6-linux-init/scripts/rc.init (90%) rename {vm/app/lynx/etc/s6-rc/lynx => img/app/etc/s6-rc/app}/run (80%) rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type (100%) rename {vm/app/catgirl/etc/s6-rc/catgirl => img/app/etc/s6-rc/app}/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/dependencies (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd-coldplug/up (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/notification-fd.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/run (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/mdevd/type.license (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/contents (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type (100%) rename {vm/app/catgirl => img/app}/etc/s6-rc/ok-all/type.license (100%) rename {vm/app/catgirl => img/app}/etc/ssl/certs/ca-certificates.crt (100%) rename {vm/app/lynx => img/app}/shell.nix (50%) rename {img => release}/combined/default.nix (100%) rename {img => release}/combined/eosimages.nix (100%) rename {img => release}/combined/grub.cfg.in (100%) rename {img => release}/combined/run-vm.nix (100%) rename {img => release}/installer/app/0001-gpt-disable-gpt-partition-attribute-55-check.patch (100%) rename {img => release}/installer/app/0002-gpt-disable-partition-table-CRC-check.patch (100%) rename {img => release}/installer/app/0003-install-remove-Endless-OS-ad.patch (100%) rename {img => release}/installer/app/0004-finished-don-t-run-eos-diagnostics.patch (100%) rename {img => release}/installer/app/0005-finished-promote-spectrum-not-the-Endless-forum.patch (100%) rename {img => release}/installer/app/default.nix (100%) rename {img => release}/installer/app/vendor-customer-support.ini (100%) rename {img => release}/installer/configuration.nix (100%) rename {img => release}/installer/default.nix (100%) rename {img => release}/installer/run-vm.nix (100%) rename {img => release}/installer/seat.rules (100%) rename {img => release}/live/Makefile (100%) rename {img => release}/live/default.nix (100%) rename {img => release}/live/shell.nix (100%) create mode 100644 vm-lib/make-vm.nix create mode 100644 vm/app/catgirl.nix delete mode 100644 vm/app/catgirl/Makefile delete mode 100644 vm/app/catgirl/etc/fstab delete mode 100755 vm/app/catgirl/etc/mdev/iface delete mode 100755 vm/app/catgirl/etc/s6-linux-init/scripts/rc.init delete mode 100755 vm/app/catgirl/etc/s6-rc/catgirl/run delete mode 100644 vm/app/catgirl/host/data/appvm-catgirl/providers/net/netvm delete mode 100644 vm/app/catgirl/shell.nix create mode 100644 vm/app/lynx.nix delete mode 120000 vm/app/lynx/bin delete mode 100644 vm/app/lynx/default.nix delete mode 100644 vm/app/lynx/etc/fstab delete mode 100755 vm/app/lynx/etc/init delete mode 100644 vm/app/lynx/etc/mdev.conf delete mode 100644 vm/app/lynx/etc/passwd delete mode 100644 vm/app/lynx/etc/passwd.license delete mode 100644 vm/app/lynx/etc/resolv.conf delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type delete mode 100644 vm/app/lynx/etc/s6-rc/lynx/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/dependencies delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd-coldplug/up delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/notification-fd.license delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/run delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type delete mode 100644 vm/app/lynx/etc/s6-rc/mdevd/type.license delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/contents delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type delete mode 100644 vm/app/lynx/etc/s6-rc/ok-all/type.license delete mode 120000 vm/app/lynx/etc/ssl/certs/ca-certificates.crt delete mode 100644 vm/app/lynx/host/data/appvm-lynx/providers/net/netvm base-commit: 2cc5bf637f944ec950f7a8162018f421e6671315 -- 2.37.1

1 6

[PATCH] Add image configuration option
by José Pekkarinen 16 Sep '22

16 Sep '22

The following patch proposes to host nix configuration files under nix folder that offers default configuration for an image, defaulting to a release image, which would be plain spectrum. A hardened default configuration will be proposed in the near future. In case of configuration collision between the default configuration and config.nix, the latter will be taken into account. Signed-off-by: José Pekkarinen <jose.pekkarinen(a)unikie.com> --- nix/eval-config.nix | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/nix/eval-config.nix b/nix/eval-config.nix index 467f877..09a7ada 100644 --- a/nix/eval-config.nix +++ b/nix/eval-config.nix @@ -10,4 +10,5 @@ ({ pkgs ? import <nixpkgs> {} }: { inherit pkgs; -}) config +}) (if config ? image && builtins.pathExists ./${config.image}.nix then + import ./${config.image}.nix else {}) // config -- 2.25.1

3 12

[PATCH 1/4] Introduce a build configuration file
by Alyssa Ross 14 Sep '22

14 Sep '22

By default, a file called "config.nix" in the root of the Spectrum repository will be read if it exists. That file should contain an attribute set. Currently, only a "pkgs" key is supported, which allows specifying a custom package set that will be used throughout the Spectrum Nix files. This will allow us to provide configuartion options for people who want to build Spectrum in ways that are probably not suitable for upstreaming. For example, using the "pkgs" config option I'm introducing here, it would be possible to use an overlay to patch individual components, like so: { pkgs = import <nixpkgs> { overlays = [ (final: super: { weston = super.weston.overrideAttrs ({ patches ? [], ... }: { patches = patches ++ [ path/to/weston.patch ]; }); }) ]; }; } Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- José, Ivan, if you'd like to review these patches, there's some documentation on how to do so here: :) https://spectrum-os.org/doc/reviewing-patches.html .gitignore | 1 + Documentation/default.nix | 4 ++-- Documentation/jekyll.nix | 2 +- host/initramfs/default.nix | 7 +++---- host/initramfs/extfs.nix | 8 ++++---- host/initramfs/shell.nix | 9 +++++---- host/rootfs/default.nix | 7 +++++-- host/rootfs/shell.nix | 8 ++++---- host/start-vm/default.nix | 2 +- host/start-vm/shell.nix | 6 +++--- img/combined/default.nix | 6 +++--- img/combined/eosimages.nix | 4 ++-- img/combined/run-vm.nix | 4 ++-- img/installer/default.nix | 3 ++- img/installer/run-vm.nix | 7 ++++--- img/live/default.nix | 11 ++++++----- img/live/shell.nix | 15 +++++++++------ nix/eval-config.nix | 10 ++++++++++ release.nix | 10 +++++++--- scripts/default.nix | 4 ++-- shell.nix | 2 +- vm/app/catgirl/default.nix | 6 +++--- vm/app/catgirl/shell.nix | 6 +++--- vm/app/lynx/default.nix | 6 +++--- vm/app/lynx/shell.nix | 6 +++--- vm/sys/net/default.nix | 6 +++--- vm/sys/net/shell.nix | 6 +++--- 27 files changed, 95 insertions(+), 71 deletions(-) create mode 100644 nix/eval-config.nix diff --git a/.gitignore b/.gitignore index 8b965e2..a97f309 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> +/config.nix build/ result result-* diff --git a/Documentation/default.nix b/Documentation/default.nix index d94a8cd..9edfbe8 100644 --- a/Documentation/default.nix +++ b/Documentation/default.nix @@ -2,7 +2,7 @@ # SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( +{ config ? import ../nix/eval-config.nix {} }: config.pkgs.callPackage ( { lib, stdenvNoCC, jekyll, drawio-headless }: @@ -31,5 +31,5 @@ stdenvNoCC.mkDerivation { passthru = { inherit jekyll; }; } ) { - jekyll = import ./jekyll.nix { inherit pkgs; }; + jekyll = import ./jekyll.nix { inherit config; }; } diff --git a/Documentation/jekyll.nix b/Documentation/jekyll.nix index bc804b1..6f2866f 100644 --- a/Documentation/jekyll.nix +++ b/Documentation/jekyll.nix @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( +{ config ? import ../nix/eval-config.nix {} }: config.pkgs.callPackage ( { bundlerApp }: diff --git a/host/initramfs/default.nix b/host/initramfs/default.nix index a4f7330..68deb5e 100644 --- a/host/initramfs/default.nix +++ b/host/initramfs/default.nix @@ -1,11 +1,10 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} -, rootfs ? import ../rootfs { inherit pkgs; } +{ config ? import ../../nix/eval-config.nix {} +, rootfs ? import ../rootfs { inherit config; } }: - -pkgs.callPackage ( +let inherit (config) pkgs; in pkgs.callPackage ( { lib, stdenvNoCC, makeModulesClosure, runCommand, writeReferencesToFile , pkgsStatic, busybox, cpio, cryptsetup, lvm2, microcodeAmd, microcodeIntel }: diff --git a/host/initramfs/extfs.nix b/host/initramfs/extfs.nix index 9fdbd9a..63f436a 100644 --- a/host/initramfs/extfs.nix +++ b/host/initramfs/extfs.nix @@ -1,21 +1,21 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs, runCommand, tar2ext4 }: +{ config, runCommand, tar2ext4 }: let netvm = import ../../vm/sys/net { - inherit pkgs; + inherit config; # inherit (foot) terminfo; }; appvm-catgirl = import ../../vm/app/catgirl { - inherit pkgs; + inherit config; # inherit (foot) terminfo; }; appvm-lynx = import ../../vm/app/lynx { - inherit pkgs; + inherit config; # inherit (foot) terminfo; }; in diff --git a/host/initramfs/shell.nix b/host/initramfs/shell.nix index 42da6a4..cbd2c60 100644 --- a/host/initramfs/shell.nix +++ b/host/initramfs/shell.nix @@ -1,16 +1,17 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: let + inherit (config) pkgs; inherit (pkgs.lib) cleanSource cleanSourceWith; extfs = pkgs.pkgsStatic.callPackage ./extfs.nix { - inherit pkgs; + inherit config; }; - rootfs = import ../rootfs { inherit pkgs; }; - initramfs = import ./. { inherit pkgs rootfs; }; + rootfs = import ../rootfs { inherit config; }; + initramfs = import ./. { inherit config rootfs; }; in with pkgs; diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index e5f316f..44e910b 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -2,7 +2,8 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-FileCopyrightText: 2022 Unikie -{ pkgs ? import <nixpkgs> {} }: pkgs.pkgsStatic.callPackage ( +{ config ? import ../../nix/eval-config.nix {} }: let inherit (config) pkgs; in +pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod @@ -13,7 +14,9 @@ let inherit (lib) cleanSource cleanSourceWith concatMapStringsSep hasSuffix; inherit (nixosAllHardware.config.hardware) firmware; - start-vm = import ../start-vm { pkgs = pkgs.pkgsStatic; }; + start-vm = import ../start-vm { + config = config // { pkgs = pkgs.pkgsStatic; }; + }; pkgsGui = pkgs.pkgsMusl.extend (final: super: { systemd = final.libudev-zero; diff --git a/host/rootfs/shell.nix b/host/rootfs/shell.nix index 3b2310f..8c30f68 100644 --- a/host/rootfs/shell.nix +++ b/host/rootfs/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { @@ -13,6 +13,6 @@ with pkgs; jq netcat qemu_kvm reuse util-linux ]; - EXT_FS = pkgsStatic.callPackage ../initramfs/extfs.nix { inherit pkgs; }; + EXT_FS = pkgsStatic.callPackage ../initramfs/extfs.nix { inherit config; }; KERNEL = "${passthru.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; }) diff --git a/host/start-vm/default.nix b/host/start-vm/default.nix index 56be882..fcce495 100644 --- a/host/start-vm/default.nix +++ b/host/start-vm/default.nix @@ -1,7 +1,7 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( +{ config ? import ../../nix/eval-config.nix {} }: config.pkgs.callPackage ( { lib, stdenv, fetchpatch, meson, ninja, rustc }: let diff --git a/host/start-vm/shell.nix b/host/start-vm/shell.nix index c7f6365..5192b76 100644 --- a/host/start-vm/shell.nix +++ b/host/start-vm/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { nativeBuildInputs ? [], ... }: { diff --git a/img/combined/default.nix b/img/combined/default.nix index 16cd506..3989d55 100644 --- a/img/combined/default.nix +++ b/img/combined/default.nix @@ -2,17 +2,17 @@ # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-FileCopyrightText: 2021 Yureka <yuka(a)yuka.dev> -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {} }: with config.pkgs; let inherit (builtins) storeDir; inherit (pkgs.lib) removePrefix; - eosimages = import ./eosimages.nix { inherit pkgs; }; + eosimages = import ./eosimages.nix { inherit config; }; installerPartUuid = "6e23b026-9f1e-479d-8a58-a0cda382e1ce"; installer = import ../installer { - inherit pkgs; + inherit config; extraConfig = { boot.initrd.availableKernelModules = [ "squashfs" ]; diff --git a/img/combined/eosimages.nix b/img/combined/eosimages.nix index 4ec28e5..9f2ab10 100644 --- a/img/combined/eosimages.nix +++ b/img/combined/eosimages.nix @@ -1,12 +1,12 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {} }: with config.pkgs; runCommand "eosimages.img" { nativeBuildInputs = [ e2fsprogs tar2ext4 ]; imageName = "Spectrum-0.0-x86_64-generic.0.Live.img"; - image = import ../live { inherit pkgs; }; + image = import ../live { inherit config; }; } '' mkdir dir cd dir diff --git a/img/combined/run-vm.nix b/img/combined/run-vm.nix index 893bc7d..40eacc4 100644 --- a/img/combined/run-vm.nix +++ b/img/combined/run-vm.nix @@ -1,10 +1,10 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {} }: with config.pkgs; let - image = import ./. { inherit pkgs; }; + image = import ./. { inherit config; }; in writeShellScript "run-spectrum-installer-vm.sh" '' diff --git a/img/installer/default.nix b/img/installer/default.nix index ba97a53..0c57704 100644 --- a/img/installer/default.nix +++ b/img/installer/default.nix @@ -1,7 +1,8 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {}, extraConfig ? {} }: with pkgs; +{ config ? import ../../nix/eval-config.nix {}, extraConfig ? {} }: +with config.pkgs; let inherit (builtins) head match storeDir; diff --git a/img/installer/run-vm.nix b/img/installer/run-vm.nix index 4a6c849..4efbf1a 100644 --- a/img/installer/run-vm.nix +++ b/img/installer/run-vm.nix @@ -1,17 +1,18 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: let inherit (builtins) storeDir; + inherit (config) pkgs; inherit (pkgs) coreutils qemu_kvm stdenv writeShellScript; inherit (pkgs.lib) makeBinPath escapeShellArg; - eosimages = import ../combined/eosimages.nix { inherit pkgs; }; + eosimages = import ../combined/eosimages.nix { inherit config; }; installer = import ./. { - inherit pkgs; + inherit config; extraConfig = { boot.initrd.availableKernelModules = [ "9p" "9pnet_virtio" ]; diff --git a/img/live/default.nix b/img/live/default.nix index 88f0ee4..65ad058 100644 --- a/img/live/default.nix +++ b/img/live/default.nix @@ -1,17 +1,18 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: let + inherit (config) pkgs; inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix; extfs = pkgs.pkgsStatic.callPackage ../../host/initramfs/extfs.nix { - inherit pkgs; + inherit config; }; - rootfs = import ../../host/rootfs { inherit pkgs; }; - scripts = import ../../scripts { inherit pkgs; }; - initramfs = import ../../host/initramfs { inherit pkgs rootfs; }; + rootfs = import ../../host/rootfs { inherit config; }; + scripts = import ../../scripts { inherit config; }; + initramfs = import ../../host/initramfs { inherit config rootfs; }; in with pkgs; diff --git a/img/live/shell.nix b/img/live/shell.nix index b9f0246..dcf2059 100644 --- a/img/live/shell.nix +++ b/img/live/shell.nix @@ -1,12 +1,15 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ({ nativeBuildInputs ? [], ... }: { - nativeBuildInputs = nativeBuildInputs ++ [ qemu_kvm ]; +(import ./. { inherit config; }).overrideAttrs ( + { nativeBuildInputs ? [], ... }: + { + nativeBuildInputs = nativeBuildInputs ++ [ qemu_kvm ]; - OVMF_CODE = "${qemu_kvm}/share/qemu/edk2-${stdenv.hostPlatform.qemuArch}-code.fd"; -}) + OVMF_CODE = "${qemu_kvm}/share/qemu/edk2-${stdenv.hostPlatform.qemuArch}-code.fd"; + } +) diff --git a/nix/eval-config.nix b/nix/eval-config.nix new file mode 100644 index 0000000..9265df7 --- /dev/null +++ b/nix/eval-config.nix @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2022 Unikie + +{ config ? + if builtins.pathExists ../config.nix then import ../config.nix else {} +}: + +({ pkgs ? import <nixpkgs> {} }: { + inherit pkgs; +}) config diff --git a/release.nix b/release.nix index da7123f..91a843b 100644 --- a/release.nix +++ b/release.nix @@ -3,10 +3,14 @@ # This file is built to populate the binary cache. -{ pkgs ? import <nixpkgs> {} }: +# Set config = {} to disable implicitly reading config.nix, since +# we'll want the result to be the same as on the binary cache. If it +# turns out there is a compelling reason to read the default config +# here, we can reconsider this. +{ config ? import nix/eval-config.nix { config = {}; } }: { - doc = import ./Documentation { inherit pkgs; }; + doc = import ./Documentation { inherit config; }; - combined = import img/combined/run-vm.nix { inherit pkgs; }; + combined = import img/combined/run-vm.nix { inherit config; }; } diff --git a/scripts/default.nix b/scripts/default.nix index 7995723..2237cb5 100644 --- a/scripts/default.nix +++ b/scripts/default.nix @@ -1,10 +1,10 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../nix/eval-config.nix {} }: let - inherit (pkgs.lib) cleanSource cleanSourceWith hasSuffix; + inherit (config.pkgs.lib) cleanSource cleanSourceWith hasSuffix; in cleanSourceWith { diff --git a/shell.nix b/shell.nix index 41d0865..4dc8558 100644 --- a/shell.nix +++ b/shell.nix @@ -1,7 +1,7 @@ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi(a)alyssa.is> # SPDX-License-Identifier: MIT -{ pkgs ? import <nixpkgs> {} }: with pkgs; +{ config ? import nix/eval-config.nix {} }: with config.pkgs; mkShell { nativeBuildInputs = [ reuse rustfmt ]; diff --git a/vm/app/catgirl/default.nix b/vm/app/catgirl/default.nix index 738a603..61f1462 100644 --- a/vm/app/catgirl/default.nix +++ b/vm/app/catgirl/default.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} -, terminfo ? pkgs.foot.terminfo +{ config ? import ../../../nix/eval-config.nix {} +, terminfo ? config.pkgs.foot.terminfo }: -pkgs.pkgsStatic.callPackage ( +config.pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4 diff --git a/vm/app/catgirl/shell.nix b/vm/app/catgirl/shell.nix index 87c4f6e..852b246 100644 --- a/vm/app/catgirl/shell.nix +++ b/vm/app/catgirl/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { diff --git a/vm/app/lynx/default.nix b/vm/app/lynx/default.nix index 50fd760..ba715ec 100644 --- a/vm/app/lynx/default.nix +++ b/vm/app/lynx/default.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} -, terminfo ? pkgs.foot.terminfo +{ config ? import ../../../nix/eval-config.nix {} +, terminfo ? config.pkgs.foot.terminfo }: -pkgs.pkgsStatic.callPackage ( +config.pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4 diff --git a/vm/app/lynx/shell.nix b/vm/app/lynx/shell.nix index 87c4f6e..852b246 100644 --- a/vm/app/lynx/shell.nix +++ b/vm/app/lynx/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index 0ce72fc..dfc7c35 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} -, terminfo ? pkgs.foot.terminfo +{ config ? import ../../../nix/eval-config.nix {} +, terminfo ? config.pkgs.foot.terminfo }: -pkgs.pkgsStatic.callPackage ( +config.pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, runCommand, writeReferencesToFile, buildPackages , s6-rc, tar2ext4, xorg diff --git a/vm/sys/net/shell.nix b/vm/sys/net/shell.nix index bc4de67..849920d 100644 --- a/vm/sys/net/shell.nix +++ b/vm/sys/net/shell.nix @@ -1,11 +1,11 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> -{ pkgs ? import <nixpkgs> {} }: +{ config ? import ../../../nix/eval-config.nix {} }: -with pkgs; +with config.pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +(import ./. { inherit config; }).overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { -- 2.37.1

4 11

[PATCH 1/2] host/rootfs: use initramfs in "make run"
by Alyssa Ross 08 Sep '22

08 Sep '22

This will allow us to stop compiling e.g. the virtio-blk module into the kernel, because it will be loaded by the initramfs. This introduces some duplication between the rootfs and initramfs's Makefiles. I don't think it's worth the effort at the moment to try to reduce that, because it would come at the expense of additional complexity in the Makefiles. We can revisit this later if we want to. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- host/rootfs/Makefile | 32 ++++++++++++++++++++++++++++---- host/rootfs/shell.nix | 10 ++++++++-- 2 files changed, 36 insertions(+), 6 deletions(-) diff --git a/host/rootfs/Makefile b/host/rootfs/Makefile index 41cf87c..31f76d2 100644 --- a/host/rootfs/Makefile +++ b/host/rootfs/Makefile @@ -6,6 +6,9 @@ # QEMU_KVM = qemu-system-x86_64 -enable-kvm. QEMU_KVM = qemu-kvm +SCRIPTS = ../../scripts +VERITYSETUP = veritysetup + # tar2ext4 will leave half a filesystem behind if it's interrupted # half way through. build/rootfs.ext4: build/rootfs.tar @@ -116,16 +119,37 @@ clean: rm -rf build .PHONY: clean -run: build/rootfs.ext4 $(EXT_FS) +# veritysetup format produces two files, but Make only (portably) +# supports one output per rule, so we combine the two outputs then +# define two more rules to separate them again. +build/rootfs.verity: build/rootfs.ext4 + $(VERITYSETUP) format build/rootfs.ext4 build/rootfs.verity.superblock.tmp \ + | awk -F ':[[:blank:]]*' '$$1 == "Root hash" {print $$2; exit}' \ + > build/rootfs.verity.roothash.tmp + cat build/rootfs.verity.roothash.tmp build/rootfs.verity.superblock.tmp \ + > $@ + rm build/rootfs.verity.roothash.tmp build/rootfs.verity.superblock.tmp +build/rootfs.verity.roothash: build/rootfs.verity + head -n 1 build/rootfs.verity > $@ +build/rootfs.verity.superblock: build/rootfs.verity + tail -n +2 build/rootfs.verity > $@ + +build/live.img: $(SCRIPTS)/format-uuid.sh $(SCRIPTS)/make-gpt.sh build/rootfs.verity.superblock build/rootfs.verity.roothash build/rootfs.ext4 + $(SCRIPTS)/make-gpt.sh $@.tmp \ + build/rootfs.verity.superblock:2c7357ed-ebd2-46d9-aec1-23d437ec2bf5:$$($(SCRIPTS)/format-uuid.sh "$$(dd if=build/rootfs.verity.roothash bs=32 skip=1 count=1 status=none)") \ + build/rootfs.ext4:4f68bce3-e8cd-4db1-96e7-fbcaf984b709:$$($(SCRIPTS)/format-uuid.sh "$$(head -c 32 build/rootfs.verity.roothash)") + mv $@.tmp $@ + +run: build/live.img $(EXT_FS) build/rootfs.verity.roothash $(QEMU_KVM) -cpu host -m 2G \ - -machine q35,kernel=$(KERNEL),kernel-irqchip=split \ + -machine q35,kernel=$(KERNEL),kernel-irqchip=split,initrd=$(INITRAMFS) \ -display gtk,gl=on \ -qmp unix:vmm.sock,server,nowait \ -monitor vc \ -parallel none \ - -drive file=build/rootfs.ext4,if=virtio,format=raw,readonly=on \ + -drive file=build/live.img,if=virtio,format=raw,readonly=on \ -drive file=$(EXT_FS),if=virtio,format=raw,readonly=on \ - -append "console=ttyS0 root=/dev/vda ext=/dev/vdb intel_iommu=on" \ + -append "console=ttyS0 roothash=$$(< build/rootfs.verity.roothash) ext=/dev/vdb intel_iommu=on" \ -device intel-iommu,intremap=on \ -device virtio-vga-gl \ -device vhost-vsock-pci,guest-cid=3 diff --git a/host/rootfs/shell.nix b/host/rootfs/shell.nix index 3b2310f..fe9df1b 100644 --- a/host/rootfs/shell.nix +++ b/host/rootfs/shell.nix @@ -1,18 +1,24 @@ # SPDX-License-Identifier: MIT # SPDX-FileCopyrightText: 2021 Alyssa Ross <hi(a)alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie { pkgs ? import <nixpkgs> {} }: +let + rootfs = import ./. { inherit pkgs; }; +in + with pkgs; -(import ./. { inherit pkgs; }).overrideAttrs ( +rootfs.overrideAttrs ( { passthru ? {}, nativeBuildInputs ? [], ... }: { nativeBuildInputs = nativeBuildInputs ++ [ - jq netcat qemu_kvm reuse util-linux + cryptsetup jq netcat qemu_kvm reuse util-linux ]; EXT_FS = pkgsStatic.callPackage ../initramfs/extfs.nix { inherit pkgs; }; + INITRAMFS = import ../initramfs { inherit pkgs rootfs; }; KERNEL = "${passthru.kernel}/${stdenv.hostPlatform.linux-kernel.target}"; }) -- 2.37.1

3 9

[PATCH v2] host/rootfs: remove kernel override
by José Pekkarinen 08 Sep '22

08 Sep '22

From: Alyssa Ross <hi(a)alyssa.is> These drivers should be loaded by the initramfs if required — most Spectrum installs won't need the virtio drivers on the host, and overriding the kernel means more stuff we can't reuse from cache.nixos.org. We'll probably want to build in the driver for whatever filesystem we end up using for the root file system eventually, since it will always be required, but that should be done as part of a more systematic effort to optimise our kernel configuration. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> Signed-off-by: José Pekkarinen <jose.pekkarinen(a)unikie.com> --- [ v1 -> v2 ] Request mdev to load modules on boot host/rootfs/default.nix | 9 +-------- host/rootfs/etc/modules-autoload.d/modules.conf | 4 ++++ 2 files changed, 5 insertions(+), 8 deletions(-) create mode 100644 host/rootfs/etc/modules-autoload.d/modules.conf diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index e5f316f..a651a20 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -65,14 +65,7 @@ let imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; }); - kernel = pkgs.linux_latest.override { - structuredExtraConfig = with lib.kernel; { - VIRTIO = yes; - VIRTIO_PCI = yes; - VIRTIO_BLK = yes; - EXT4_FS = yes; - }; - }; + kernel = pkgs.linux_latest; packagesSysroot = runCommand "packages-sysroot" { nativeBuildInputs = [ xorg.lndir ]; diff --git a/host/rootfs/etc/modules-autoload.d/modules.conf b/host/rootfs/etc/modules-autoload.d/modules.conf new file mode 100644 index 0000000..dd9529a --- /dev/null +++ b/host/rootfs/etc/modules-autoload.d/modules.conf @@ -0,0 +1,4 @@ +virtio +virtio_pci +virtio_blk +ext4 -- 2.25.1

1 0

[PATCH] Documentation: add "Sending your First Patch"
by Alyssa Ross 11 Aug '22

11 Aug '22

I think having this written down in tutorial style is important, so people can follow through some step by step instructions. We already have some similar documentation on <https://spectrum-os.org/contributing.html>, but it doesn't really go into any detail, and I think this belongs in the main documentation. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- Ville, I'm CCing you in case you'd like to review, since we talked about documenting the patch workflow a bit better before. We also have this small document about reviewing patches on the list now, at <https://spectrum-os.org/doc/reviewing-patches.html>. :) Documentation/first-patch.adoc | 96 ++++++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 Documentation/first-patch.adoc diff --git a/Documentation/first-patch.adoc b/Documentation/first-patch.adoc new file mode 100644 index 0000000..22140e4 --- /dev/null +++ b/Documentation/first-patch.adoc @@ -0,0 +1,96 @@ += Sending Your First Patch +:page-parent: Tutorials + +// SPDX-FileCopyrightText: 2022 Unikie +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +== Prerequisites + +This tutorial assumes that you already have basic +https://git-scm.com/[git] experience. + +Before beginning, you'll need to have a local git checkout of the +https://spectrum-os.org/git/[Spectrum source tree]. You'll also need +to have configured `git send-email` — a guide for this can be found at +https://git-send-email.io/. + +== Making your changes + +If you've worked on any git repository before, the process for making +your changes will probably be very familiar. + +1. Create a branch for your changes: ++ +---- +git checkout -b fix-docs # for example +---- +2. Make changes in your editor. +3. Stage and commit your changes: ++ +---- +git add Documentation/first-patch.adoc # for example +git commit -s +---- ++ +TIP: The `-s` option adds a `Signed-off-by` line to the commit. This +indicates your acceptance of the +https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's +Certificate of Origin], which is mandatory for Spectrum patches. + +== Submitting changes + +Once you're happy with how the commits on your branch look, run: + +---- +git send-email --to devel(a)spectrum-os.org --confirm=always origin/main +---- + +For each message, you will be prompted for confirmation, and then the +change will be submitted to the Spectrum development mailing list. +They will appear shortly on the +https://spectrum-os.org/lists/hyperkitty/list/devel@spectrum-os.org/[web +interface]. + +[#cover-letter] +TIP: If you're submitting multiple changes, and you'd like to provide +an overall description for the whole series, you can add the +`--compose` option, which will prompt you to compose a "cover letter" +message that will be sent before all of your patches. + +== Feedback + +Once your patch has been submitted, wait for it to be reviewed. +Feedback, if any, will be sent as email replies to your submitted +patch. You can respond to feedback in your mail client. Please use +the Reply All button to ensure that your messages are sent to the +mailing list as well as to the person who sent the feedback. + +If you need to make changes to your patch, and submit a new version, +use https://git-rebase.io/[`git rebase`] to create a new version of +your patch(es), and submit it like this: + +---- +git send-email -v2 --to devel(a)spectrum-os.org --confirm=always origin/main +---- + +The added `-v2` flag indicates that this is version two of your +patch set. If your patches require more rounds of changes, submit +subsequent rounds with `-v3`, `-v4`, etc. as appropriate. + +If you'd like to describe what has changed from the previous version +of your patches, you can do so in a xref:cover-letter[cover letter] +as described above. + +== Acceptance + +Once your patch has been accepted, it will be applied to the upstream +Spectrum git repository. You will receive an automated email +confirmation when this happens. + +== Questions + +If you have any questions about the patch submission process, you can +ask them either on the +https://spectrum-os.org/participating.html#spectrum-devel[development +mailing list], or in the +https://spectrum-os.org/participating.html#irc[IRC/Matrix channel]. -- 2.35.1

2 1

Cross-compilation support - call for test / nix-help
by Ville Ilvonen 13 Jul '22

13 Jul '22

Hi all, As Ivan (@jonne) updated earlier this week on #spectrum, we've got initial cross-compilation support for aarch64 available in Spectrum development fork. It still needs some TLC before Spectrum PRs but is a great step forward. Our current ref. board is NXP i.MX8 but rootfs build should be fairly easy to modify for RPi4 configs already in nixpkgs for anyone who cares to try. Quick how-to for non-nixos host like Ubuntu: git clone -b aarch64-imx8-crosscompile https://github.com/tiiuae/nixpkgs-spectrum.git git clone -b aarch64-imx8-crosscompile https://github.com/tiiuae/spectrum.git cd spectrum/host/rootfs nix-build -I nixpkgs=../../../nixpkgs-spectrum Know issue: - crossSystem from https://github.com/tiiuae/spectrum/commit/aad5f90cf2ec931f2af5c8d5111a9fac1… is ignored on NixOS host but not on Ubuntu - maybe someone can point us why. Thanks, -Ville

2 3

[PATCH] doc: architecture view with drawio to svg generation
by Ville Ilvonen 15 Jun '22

15 Jun '22

* Initial architecture document view a view to high level stack of Spectrum. * Stack view is generated to svg from drawio using Alyssa's drawio-headless in nixpkgs-upstream. Cherry-picked to nixpkgs-spectrum for testing. * An example to analyze the details of Spectrum dependencies interactively is also provided. Signed-off-by: Ville Ilvonen <ville.ilvonen(a)unikie.com> --- Documentation/architecture.adoc | 39 +++++++++++++++++++++++++++++ Documentation/default.nix | 8 +++--- Documentation/diagrams/stack.drawio | 1 + 3 files changed, 45 insertions(+), 3 deletions(-) create mode 100644 Documentation/architecture.adoc create mode 100644 Documentation/diagrams/stack.drawio diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc new file mode 100644 index 0000000..2f89e68 --- /dev/null +++ b/Documentation/architecture.adoc @@ -0,0 +1,39 @@ += Architecture +// SPDX-FileCopyrightText: 2022 Ville Ilvonen <ville.ilvonen(a)unikie.com> +// SPDX-License-Identifier: GFDL-1.3-no-invariants-or-later OR CC-BY-SA-4.0 + +== Introduction + +Spectrum operating system stack is based on the principle of security by compartmentalization. The high level system stack is illustrated in the following diagram. + +image::diagrams/stack.svg[] + +=== Kernel space + +In the stack, kernel space security by compartmentalization is supported with linux kernel that includes kernel-based virtual machine (KVM) module enabling the kernel to work as virtual-machine manager, hypervisor. Kernel side hypervisor supports virtualization of hardware resources - computational cores, memory and devices - securely. Userspace virtual machine guests are managed with cloud-hypervisor. Linux with KVM also supports portability to several hardware architectures. Currently Spectrum is supported only on x86_64 but ARM64 is under works. In addition, hardened kernel is to be enabled. + +=== Host user space + +This section provides high level overview of host user space tools and libraries. + +User space stack is build on musl standard C library with added safety on resource exhaustion and security hardening on memory allocation. + +https://skarnet.org/software/s6-rc/overview.html[s6-rc] service manager is used for services. kmod, util-linux and busybox are provided for essential system administration. + +https://github.com/cloud-hypervisor/cloud-hypervisor[cloud-hypervisor] is a host tooling for virtual machine management, written in Rust with a strong focus on security. + +Wayland refers to whole display stack providing communication with compositor (weston) for desktop services, including libraries and drivers for direct rendering and event devices. Clients are implemented as application virtual machines (see next section). Minimal host provides only Wayland terminal client, foot. Wayland, a simpler and more secure, protocol for compositor could provide support for legacy X applications as well but as of now none are provided. https://wayland.freedesktop.org/architecture.html[Wayland architecture] is well documented here. + +=== Application and system virtual machines + +Security by compartmentalization in Spectrum is implemented with virtual machines. Virtual machines currently launch using terminal and support only wayland-console. Wayland graphics support for appvms is under work. Please refer to https://spectrum-os.org/doc/running-vms.html[running VMs] for more information. + +Reference set of virtual machines includes system machine, netvm, and application VMs, appvm-catgirl and appvm-elinks. Please refer to https://spectrum-os.org/doc/creating-vms.html[creating VMs] for more information. + +== Details of Spectrum dependency tree + +High level overview of Spectrum stack is limited view to the system. For detailed, interactive view to dependencies please use `nix-tree` under the spectrum repository: + +`nix-build img/live -I nixpkgs=https://spectrum-os.org/git/nixpkgs/snapshot/nixpkgs-rootfs.tar.gz --no-out-link | xargs -o nix-tree` + +https://diode.zone/w/8DBDQ6HQUe5UUdLkpDuL35[See video of Spectrum live image interactive analysis with nix-tree] \ No newline at end of file diff --git a/Documentation/default.nix b/Documentation/default.nix index 02b3c31..8b969d4 100644 --- a/Documentation/default.nix +++ b/Documentation/default.nix @@ -3,7 +3,7 @@ { pkgs ? import <nixpkgs> {} }: pkgs.callPackage ( -{ lib, runCommand, jekyll }: +{ lib, runCommand, jekyll, drawio-headless }: runCommand "spectrum-docs" { src = with lib; cleanSourceWith { @@ -14,11 +14,13 @@ runCommand "spectrum-docs" { !(hasSuffix ".nix" name); }; - nativeBuildInputs = [ jekyll ]; + nativeBuildInputs = [ jekyll drawio-headless ]; passthru = { inherit jekyll; }; -} '' +} + '' jekyll build --disable-disk-cache -b /doc -s $src -d $out + drawio --recursive $out/diagrams/ --export -f svg $out/assets/images/ '' ) { jekyll = import ./jekyll.nix { inherit pkgs; }; diff --git a/Documentation/diagrams/stack.drawio b/Documentation/diagrams/stack.drawio new file mode 100644 index 0000000..23feae7 --- /dev/null +++ b/Documentation/diagrams/stack.drawio @@ -0,0 +1 @@ +<mxfile host="Electron" modified="2022-05-24T12:19:30.186Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) draw.io/15.7.3 Chrome/91.0.4472.164 Electron/13.6.1 Safari/537.36" etag="8D9FBMK4C5lWSHGzQpql" version="15.7.3" type="device"><diagram id="C5RBs43oDa-KdzZeNtuy" name="Page-1">7VrbcuI4EP0aqiYPTPkOPBIyJJtkdneSnUnmaUtggVWRLUeSueTrV7JljC8ZHArjhZoXbLdal+5z1FJLdMyRv7qmIPS+EhfijqG5q4551TEMXdcM8ZCStZJohp1I5hS5SpYJHtEbTBWVNEIuZDlFTgjmKMwLpyQI4JTnZIBSssyrzQjO9xqCOSwJHqcAl6VPyOVeIu0bvUx+A9HcS3vWnUFS4oNUWVnCPOCS5ZbI/NIxR5QQnrz5qxHE0nupX57+WD/h+xfn+vYbewXfL+/++fNHN2ls/JEqGxMoDPjeTU+e7/r61e34/vvPt8Xz8hX2//qmqmgLgCPlr47hYNHJ5YyIvoS/wTQpcF4jaejlA5kQTrJv6Ri+xnklWbnLYi4MhYJhhKvtGs5cPm8AdZfCJqHwadV3LtKehQlJ54macv6mD4OSKHChtEoTxUsPcfgYJsNcChILmcd9LL70Te2drlMuXkDK4WqLOMqV15D4kNO1UFGlG36reeFY6nuZkcx2lMzbIlgqA4rX803TGXbiRcH3ASj1FqG8R0G0io0Xthva3Y+vJw2m3W8bTKNFMKeYRG7XW4eQLhAj9KSgNPt5KC2tDKWhHRNKswTlHaRBvMyy2D1FPwpLed5ZjFPyAkcECzDMq4AEUGKJMC6IAEbzQHxOhVOhkF9KvyGxFg5VgY9cV3ZTiU4ePwn1GPgIS0cqciVStcZLFJJvEtG4HY9zsagbtuSWLXwmf6QC+zwnZI4hCBH7PCV+XDBlsep4lvQhXrNeGiNHYZoP7BI39KppbjbFDavEjRvC5PSOmMDvN0FaJojVa5sgdovrgB8xMT4H+BL7YMLkoyu0R0XhuwtEIhYkDt4fhYI3Hsem4Y5hmlbMOGVMXixV499adjAOAlfsNX8x7GSI+WE372CMJhRIkpzQ+lraKhltb5V6FVOk4DkYuEOZSmaBcMtTtaPawaNYMVa6gHnxQD4OIHRzWXAZvm14KtBJZRRiwNEinztXQaZ6+JugeJak7HhnH522wGIfqkrbuWqhHX2woyEO6BzyUkMxgTZW1+LUM354ewrGV7fL4XDQ5Xr47+T+IGnxR4LgzhDmdOl0R7xqIHyUwkJdQtYOH5ZTDh+DhqJHJdLl3Xm7SC/BGovl6hywtmukYrp+TLDL2+12wZ5EbD0hq3MA29F2T2y9d0ywD7F1PiTYM3lQHO9x48YmNC34JFIx/+IsSDDYPeP7x+SA8z/jwItPziK0m3Vmu3VMpAcHQHpfVCOOcBcnR+Clye2jAPkAX5SLTgpxy6qBeFN5XyXiepsXHkwkILy78E8Kw9Lmu2pDdtQ1Wm/zniOAPANwa8ayNVv4pz5fi1hvMvxtrKsyLd1uDOyqXOtsTmoKJzD7A3ikkxo7Rw9zMMi3UP+kZkdDDZ/U6IfI6faNIIgyhioiCAjD048gllYjglStFg1GkDavQ6DY4b2wc0Xb7u1Gu/LevEG0y9nbA5xBYaa8E9UWiPII4LLffTD1UABZydG/702bYk/h3tTUayYHe9ybis/sL3fJKpL9c9H88h8=</diagram></mxfile> \ No newline at end of file -- 2.33.3

3 8

[PATCH nixpkgs] stdenv: update aarch64 musl bootstrap tools
by Alyssa Ross 13 Jun '22

13 Jun '22

This will allow removing some workarounds (661dfd83478 and the preceding commits) that I had to do to get aarch64 musl to bootstrap, and hopefully will also fix some of the bootstrapping failures we've been seeing since then on newer aarch64 hardware. This approach unfortunately requires users to have the Spectrum binary cache set up, but it's the best we can do for now. When it's stabilized, we could use builtins.fetchClosure, which would avoid that requirement. But hopefully by that point we can have the bootstrap tools updated upstream anyway. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- .../linux/bootstrap-files/aarch64-musl.nix | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix b/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix index 252783cce47..64c950529e9 100644 --- a/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix +++ b/pkgs/stdenv/linux/bootstrap-files/aarch64-musl.nix @@ -1,11 +1,9 @@ +let + # This path is available on the Spectrum binary cache. + # See <https://spectrum-os.org/doc/binary-cache.html>. + build = builtins.storePath /nix/store/bq9z1m3dvcsk522j9li43gm61caj5ybr-stdenv-bootstrap-tools-aarch64-unknown-linux-musl; +in { - busybox = import <nix/fetchurl.nix> { - url = "https://wdtz.org/files/wjzsj9cmdkc70f78yh072483x8656nci-stdenv-bootstrap-to…"; - sha256 = "01s6bwq84wyrjh3rdsgxni9gkzp7ss8rghg0cmp8zd87l79y8y4g"; - executable = true; - }; - bootstrapTools = import <nix/fetchurl.nix> { - url = "https://wdtz.org/files/wjzsj9cmdkc70f78yh072483x8656nci-stdenv-bootstrap-to…"; - sha256 = "0pbqrw9z4ifkijpfpx15l2dzi00rq8c5zg9ghimz5qgr5dx7f7cl"; - }; + busybox = "${build}/on-server/busybox"; + bootstrapTools = "${build}/on-server/bootstrap-tools.tar.xz"; } -- 2.35.1

1 0

[PATCH] Allow static building of dtc
by Tero Tervala 25 May '22

25 May '22

Signed-off-by: Tero Tervala <tero.tervala(a)unikie.com> --- ...01-Use-libfdt.a-if-building-staticly.patch | 54 +++++++++++++++++++ pkgs/development/compilers/dtc/default.nix | 4 ++ 2 files changed, 58 insertions(+) create mode 100644 pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch diff --git a/pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch b/pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch new file mode 100644 index 00000000000..4f0d6193030 --- /dev/null +++ b/pkgs/development/compilers/dtc/0001-Use-libfdt.a-if-building-staticly.patch @@ -0,0 +1,54 @@ +From 59258caa6e02b5a7d5afc0bfde777b250ef74bbb Mon Sep 17 00:00:00 2001 +From: Tero Tervala <tero.tervala(a)unikie.com> +Date: Wed, 18 May 2022 11:20:40 +0300 +Subject: [PATCH] Use libfdt.a if building staticly + +Signed-off-by: Tero Tervala <tero.tervala(a)unikie.com> +--- + Makefile | 14 ++++++++++---- + 1 file changed, 10 insertions(+), 4 deletions(-) + +diff --git a/Makefile b/Makefile +index ea8c659..9474b63 100644 +--- a/Makefile ++++ b/Makefile +@@ -151,7 +151,7 @@ BIN += fdtoverlay + + SCRIPTS = dtdiff + +-all: $(BIN) libfdt ++all: libfdt $(BIN) + + # We need both Python and swig to build/install pylibfdt. + # This builds the given make ${target} if those deps are found. +@@ -198,6 +198,12 @@ LIBFDT_lib = $(LIBFDT_dir)/$(LIBFDT_LIB) + LIBFDT_include = $(addprefix $(LIBFDT_dir)/,$(LIBFDT_INCLUDES)) + LIBFDT_version = $(addprefix $(LIBFDT_dir)/,$(LIBFDT_VERSION)) + ++ifneq (,$(findstring -static,$(CFLAGS))) ++ LIBFDT_dep = $(LIBFDT_archive) ++else ++ LIBFDT_dep = $(LIBFDT_lib) ++endif ++ + include $(LIBFDT_dir)/Makefile.libfdt + + .PHONY: libfdt +@@ -261,11 +267,11 @@ convert-dtsv0: $(CONVERT_OBJS) + + fdtdump: $(FDTDUMP_OBJS) + +-fdtget: $(FDTGET_OBJS) $(LIBFDT_lib) ++fdtget: $(FDTGET_OBJS) $(LIBFDT_dep) + +-fdtput: $(FDTPUT_OBJS) $(LIBFDT_lib) ++fdtput: $(FDTPUT_OBJS) $(LIBFDT_dep) + +-fdtoverlay: $(FDTOVERLAY_OBJS) $(LIBFDT_lib) ++fdtoverlay: $(FDTOVERLAY_OBJS) $(LIBFDT_dep) + + dist: + git archive --format=tar --prefix=dtc-$(dtc_version)/ HEAD \ +-- +2.25.1 + diff --git a/pkgs/development/compilers/dtc/default.nix b/pkgs/development/compilers/dtc/default.nix index 04dca4772a9..33369fb1fcc 100644 --- a/pkgs/development/compilers/dtc/default.nix +++ b/pkgs/development/compilers/dtc/default.nix @@ -12,6 +12,10 @@ stdenv.mkDerivation rec { sha256 = "sha256-gx9LG3U9etWhPxm7Ox7rOu9X5272qGeHqZtOe68zFs4="; }; + patches = [ + ./0001-Use-libfdt.a-if-building-staticly.patch + ]; + buildInputs = [ libyaml ]; nativeBuildInputs = [ flex bison pkg-config which ] ++ lib.optionals pythonSupport [ python swig ]; -- 2.33.3

2 3