[PATCH] Documentation: Architecture Decision Record
* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc Signed-off-by: Ville Ilvonen <ville.ilvonen@unikie.com> --- Documentation/architecture.adoc | 13 ++++++++++ ...architecture-decision-record-template.adoc | 13 ++++++++++ .../decisions/001-host-update-mechanism.adoc | 19 ++++++++++++++ .../decisions/002-install-options.adoc | 18 +++++++++++++ Documentation/decisions/003-partitioning.adoc | 25 +++++++++++++++++++ .../004-data-at-rest-encryption.adoc | 16 ++++++++++++ .../005-virtual-machine-manager.adoc | 24 ++++++++++++++++++ .../decisions/006-drivers-on-host.adoc | 17 +++++++++++++ .../decisions/007-USB-virtual-machine.adoc | 14 +++++++++++ ...008-Inter-VM-communication-mechanisms.adoc | 18 +++++++++++++ 10 files changed, 177 insertions(+) create mode 100644 Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc create mode 100644 Documentation/decisions/001-host-update-mechanism.adoc create mode 100644 Documentation/decisions/002-install-options.adoc create mode 100644 Documentation/decisions/003-partitioning.adoc create mode 100644 Documentation/decisions/004-data-at-rest-encryption.adoc create mode 100644 Documentation/decisions/005-virtual-machine-manager.adoc create mode 100644 Documentation/decisions/006-drivers-on-host.adoc create mode 100644 Documentation/decisions/007-USB-virtual-machine.adoc create mode 100644 Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc diff --git a/Documentation/architecture.adoc b/Documentation/architecture.adoc index 157907f..185740c 100644 --- a/Documentation/architecture.adoc +++ b/Documentation/architecture.adoc @@ -20,6 +20,19 @@ devices and provides network services to application VMs). Refer to xref:creating-vms.adoc[Creating VMs] and xref:running-vms.adoc[Running VMs] for more information about using VMs in Spectrum. +== Architecture Decision Record (ADR) + +https://adr.github.io/[Architecturally significant decisions] are +recorded as https://github.com/joelparkerhenderson/architecture-decision-record/blob/main/templates/decision-record-template-by-michael-nygard/index.md[light-weight ADRs] + +Status of Spectrum OS ADRs: +Accepted - Implemented and likely not to change. +Proposed - Designed and possibly partially implmented. May change. +Other - Not yet in use. + +Comments and contributions to ADRs are welcome. ADRs can be found at +Documentation/decisions + == The Spectrum host system Compartmentalization is implemented using diff --git a/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc new file mode 100644 index 0000000..087ec44 --- /dev/null +++ b/Documentation/decisions/000-lightweight-architecture-decision-record-template.adoc @@ -0,0 +1,13 @@ +# Title + +## Status +What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.? + +## Context +What is the issue that we're seeing that is motivating this decision or change? + +## Decision +What is the change that we're proposing and/or doing? + +## Consequences +What becomes easier or more difficult to do because of this change? diff --git a/Documentation/decisions/001-host-update-mechanism.adoc b/Documentation/decisions/001-host-update-mechanism.adoc new file mode 100644 index 0000000..03bbae2 --- /dev/null +++ b/Documentation/decisions/001-host-update-mechanism.adoc @@ -0,0 +1,19 @@ +# Host update mechanism + +## Status +Proposed + +## Context +Spectrum OS has no implementation for software update. The host - consisting of +Linux kernel, KVM, cloud-hypervisor and minimal user space tools - software +updates are required to support feature development and security fixes. + +## Decision +A-B partitioning created by Spectrum installer Installer sets up the system on +partition A of the block device A-B update scheme where user (or installer) +writes the update image to partition B Bootloader provides four boot options: +A, A mutable, B, B mutable + +## Consequences +Default boot selection, incremental updates (e.g. overlays), network update +postponed for later. diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc new file mode 100644 index 0000000..f5857c5 --- /dev/null +++ b/Documentation/decisions/002-install-options.adoc @@ -0,0 +1,18 @@ +# Install options + +## Status +Proposed + +## Context +Based on identified different audiences for the Spectrum OS release it is +proposed we support three base configurations to use with Spectrum OS in the +first boot. + +## Decision +* Minimal - Spectrum OS host + system VMs: netvm, guivm, usbvm + home-directory +(optionally encrypted - see 004-disk-encryption.md) +* Common - Minimal + browser app VM + 2-3 selected app VMs +* Power - Common + NixOS VM + +## Consequences +Requires first-boot-vm (like wizard) to support user to get started. diff --git a/Documentation/decisions/003-partitioning.adoc b/Documentation/decisions/003-partitioning.adoc new file mode 100644 index 0000000..345619f --- /dev/null +++ b/Documentation/decisions/003-partitioning.adoc @@ -0,0 +1,25 @@ +# Partitioning + +## Status +Proposed + +## Context +Partitions are required to install the Spectrum OS, VMs and store user data. + +## Decision +---- +<blockdevice> # EFI system partition +<blockdevice> # XBOOTLDR +<blockdevice> # A +<blockdevice> # B +# first 32 GB are reserved for Spectrum system +# rest of the disk is reserved for user data +<blockdevice>n-1 # bootstrap user data +<blockdevice>n to the end of disk # user data +---- + +## Consequences +LVM may support resizing - both increasing and decreasing with some limitation +when there's alreay data on volume(s). Does LVM work with all disk types? We +have to implement XBOOTLDR to support EFI system partition created by Windows - +to support dual boot diff --git a/Documentation/decisions/004-data-at-rest-encryption.adoc b/Documentation/decisions/004-data-at-rest-encryption.adoc new file mode 100644 index 0000000..3ed9abb --- /dev/null +++ b/Documentation/decisions/004-data-at-rest-encryption.adoc @@ -0,0 +1,16 @@ +# Data at rest encryption + +## Status +Proposed + +## Context +To support user data and privacy protection, encryption of data at rest is +required. + +## Decision +User data is encrypted. + +## Consequences +Spectrum OS needs to come with enough SW to get the encryption key via different +methods (password, usb, fido, etc.) Can we use dm-crypt for everything instead +of LUKS? diff --git a/Documentation/decisions/005-virtual-machine-manager.adoc b/Documentation/decisions/005-virtual-machine-manager.adoc new file mode 100644 index 0000000..b4af595 --- /dev/null +++ b/Documentation/decisions/005-virtual-machine-manager.adoc @@ -0,0 +1,24 @@ +# Virtual Machine Manager + +## Status +Accepted + +## Context +rust-vmm-based VMM provides memory and concurrency safe solution. +cloud-hypervisor was chosen because firecrack does not support other +virtio-devices than net or block. crosvm was not chosen because cloud-hypervisor +has more flexible IPC mechanisms, more engaging community as LF-project. +cloud-hypervisor has more core features - such as snapshotting, live migration +and more general hot plugging. crosvm supports more devices we will also need. +It was seen easier to port devices from crosvm to cloud-hypervisor than to port +core features from cloud-hypervisor to crosvm. + +## Decision +Spectrum OS design and implementation decision is to use cloud-hypervisor as the +primary VMM. + +## Consequences +We gotta port some stuff from crosvm to cloud-hypervisor. It's easier for +Spectrum to handle virtualization dynamically with cloud-hypervisor. If the +primary VMM, cloud-hypervisor, is exchanged for trials etc. functionality is +expected to break or not supported. diff --git a/Documentation/decisions/006-drivers-on-host.adoc b/Documentation/decisions/006-drivers-on-host.adoc new file mode 100644 index 0000000..052b596 --- /dev/null +++ b/Documentation/decisions/006-drivers-on-host.adoc @@ -0,0 +1,17 @@ +# Drivers on host + +## Status +Accepted + +## Context +To harden the trusted computing base and make it more minimal, the target is to +minimize the amount of drivers on the Spectrum host kernel. + +## Decision +We are aiming to have as few drivers as possible on the host. + +## Consequences +No networking on the host. Responsibilities of the host are expected to get +smaller over time. More flexible management of devices. We need to decouple +device classes - like net, usb, bluetooth and gui - from host to their +respective VMs. diff --git a/Documentation/decisions/007-USB-virtual-machine.adoc b/Documentation/decisions/007-USB-virtual-machine.adoc new file mode 100644 index 0000000..2072427 --- /dev/null +++ b/Documentation/decisions/007-USB-virtual-machine.adoc @@ -0,0 +1,14 @@ +# USB Virtual Machine + +## Status +Proposed + +## Context +To support specific USB devices on specific VMs + +## Decision +The decision is to pass-through USB controller to a VM with authorization +controls inside the VMs to forward a specific USB device using USBIP. + +## Consequences +We need to modify the upstream USBIP daemon to support authorization. diff --git a/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc new file mode 100644 index 0000000..1007037 --- /dev/null +++ b/Documentation/decisions/008-Inter-VM-communication-mechanisms.adoc @@ -0,0 +1,18 @@ +### Inter VM communication mechanisms + +### Status +Proposed + +### Context +Guest VM software needs to communicate with software in other guest VMs + +### Decision +Spectrum provides two mechanism +- TCP/IP with virtio-net +- Wayland with virtio-gpu (nevermind the semantics) for streamed IPC protocol to + send references to shared memory + + +### Consequences +- Examples required on how to write applications which communicate over + virtio-gpu -- 2.36.0
On Fri, Jul 08, 2022 at 02:02:15PM +0300, Ville Ilvonen wrote:
* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc
Signed-off-by: Ville Ilvonen <ville.ilvonen@unikie.com>
Hi Ville, thanks for taking the lead on these. I've been wondering whether it would be better to only take accepted ADRs into the Spectrum repository, and leave proposed ones for discussion on the list until they're ready to be accepted. I think that would make the distinction between them clearer. What do you think? In future, it would probably also be good for each ADR to be its own commit, to make them easier to discuss one at a time. (I will add this to the documentation around submitting patches, when I get a chance to write it.) Finally, all the files being added need license and copyright headers. I'd suggest using the same license as the rest of the documentation. I'm happy to include at least the accepted ADRs, if the proper headers are included. You can check all required headers are present by running `reuse lint` in the Spectrum repository.
On Tue, Jul 12, 2022 at 9:11 PM Alyssa Ross <hi@alyssa.is> wrote:
On Fri, Jul 08, 2022 at 02:02:15PM +0300, Ville Ilvonen wrote:
* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc
Signed-off-by: Ville Ilvonen <ville.ilvonen@unikie.com>
Hi Ville, thanks for taking the lead on these.
I've been wondering whether it would be better to only take accepted ADRs into the Spectrum repository, and leave proposed ones for discussion on the list until they're ready to be accepted. I think that would make the distinction between them clearer. What do you think?
I was thinking separation by status when writing the PR. Maybe restructuring to folders like Documentation/adr/accepted Documentation/adr/proposed ... would make sense?
In future, it would probably also be good for each ADR to be its own commit, to make them easier to discuss one at a time. (I will add this to the documentation around submitting patches, when I get a chance to write it.)
Agreed.
Finally, all the files being added need license and copyright headers. I'd suggest using the same license as the rest of the documentation.
Sure, this came to my mind right after I had sent the PR but I decided to wait for your review comments before fixing this.
I'm happy to include at least the accepted ADRs, if the proper headers are included. You can check all required headers are present by running `reuse lint` in the Spectrum repository.
Thanks. Best, -Ville
On Wed, Jul 13, 2022 at 12:35:47PM +0300, Ville Ilvonen wrote:
On Tue, Jul 12, 2022 at 9:11 PM Alyssa Ross <hi@alyssa.is> wrote:
On Fri, Jul 08, 2022 at 02:02:15PM +0300, Ville Ilvonen wrote:
* ADRs based on discussions with Alyssa * A note on ADRs to architecture.adoc
Signed-off-by: Ville Ilvonen <ville.ilvonen@unikie.com>
Hi Ville, thanks for taking the lead on these.
I've been wondering whether it would be better to only take accepted ADRs into the Spectrum repository, and leave proposed ones for discussion on the list until they're ready to be accepted. I think that would make the distinction between them clearer. What do you think?
I was thinking separation by status when writing the PR. Maybe restructuring to folders like Documentation/adr/accepted Documentation/adr/proposed ... would make sense?
I've thought about this a bit more, and decided that it's fine to have proposed ADRs in the repo. I was concerned about the in-between state an ADR could find itself in when it's been submitted but I haven't applied the patch yet, but having thought about it more, I don't think having a stage of "being accepted for discussion" is a bad thing.
On Wed, Jul 13, 2022 at 12:53 PM Alyssa Ross <hi@alyssa.is> wrote:
I've thought about this a bit more, and decided that it's fine to have proposed ADRs in the repo. I was concerned about the in-between state an ADR could find itself in when it's been submitted but I haven't applied the patch yet, but having thought about it more, I don't think having a stage of "being accepted for discussion" is a bad thing.
Ok. This patch addresses all your review comments. -Ville
This patch has been committed as 5ad4b02eb9cff90c28d31cb865ca14f81d68a47e, which can be viewed online at https://spectrum-os.org/git/spectrum/commit/?id=5ad4b02eb9cff90c28d31cb865ca.... This is an automated message. Send comments/questions/requests to: Alyssa Ross <hi@alyssa.is>
On Wed, Jul 13, 2022 at 02:04:31PM +0300, Ville Ilvonen wrote:
On Wed, Jul 13, 2022 at 12:53 PM Alyssa Ross <hi@alyssa.is> wrote:
I've thought about this a bit more, and decided that it's fine to have proposed ADRs in the repo. I was concerned about the in-between state an ADR could find itself in when it's been submitted but I haven't applied the patch yet, but having thought about it more, I don't think having a stage of "being accepted for discussion" is a bad thing.
Ok. This patch addresses all your review comments.
Thanks. I've applied the patch. I still have some reservations about the ADRs — I'm still a bit worried about the proposed ADRs being in the repo making people hesitant to suggest changes to them, and I'm not sure yet how they fit into the rest of the documentation — but let's give them a go, as a trial.
On Fri, Jul 08, 2022 at 02:02:15PM +0300, Ville Ilvonen wrote:
diff --git a/Documentation/decisions/002-install-options.adoc b/Documentation/decisions/002-install-options.adoc new file mode 100644 index 0000000..f5857c5 --- /dev/null +++ b/Documentation/decisions/002-install-options.adoc @@ -0,0 +1,18 @@ +# Install options + +## Status +Proposed + +## Context +Based on identified different audiences for the Spectrum OS release it is +proposed we support three base configurations to use with Spectrum OS in the +first boot. + +## Decision +* Minimal - Spectrum OS host + system VMs: netvm, guivm, usbvm + home-directory +(optionally encrypted - see 004-disk-encryption.md)
Optionally? I think it should probably always be encrypted.
+* Common - Minimal + browser app VM + 2-3 selected app VMs +* Power - Common + NixOS VM + +## Consequences +Requires first-boot-vm (like wizard) to support user to get started.
Quoting Ville Ilvonen (2022-07-08 04:02:15)
+++ b/Documentation/decisions/006-drivers-on-host.adoc ... +## Consequences +No networking on the host. Responsibilities of the host are expected to get +smaller over time.
I know you're probably a long, long way from worrying about wifi, but when you get to that point, I suggest taking a look at running `eiwd` in the netvm: https://github.com/illiliti/eiwd/ https://github.com/NixOS/nixpkgs/pull/208844 - It is way more robust than wpa_supplicant. - (unlike iwd) it does not require plumbing a dbus connection from the hostvm to the netvm (nor all the attack surface that would entail). - You control it by simply editing its state files, which it watches with `inotify()`. - It has zero runtime dependencies! - `pkgsStatic.eiwd` builds (caveat: I have not tested it) - You should be able to boot a VM with `init=/nix/store/...-eiwd/bin/eiwd` and run it as PID 1 (note: I have not tried this). I've been running this on my daily driver laptop for the last few weeks and am extremely happy with it. - a
On Mon, Jan 02, 2023 at 11:03:43PM -0800, Adam Joseph wrote:
Quoting Ville Ilvonen (2022-07-08 04:02:15)
+++ b/Documentation/decisions/006-drivers-on-host.adoc ... +## Consequences +No networking on the host. Responsibilities of the host are expected to get +smaller over time.
I know you're probably a long, long way from worrying about wifi, but when you get to that point, I suggest taking a look at running `eiwd` in the netvm:
https://github.com/illiliti/eiwd/ https://github.com/NixOS/nixpkgs/pull/208844
- It is way more robust than wpa_supplicant. - (unlike iwd) it does not require plumbing a dbus connection from the hostvm to the netvm (nor all the attack surface that would entail). - You control it by simply editing its state files, which it watches with `inotify()`. - It has zero runtime dependencies! - `pkgsStatic.eiwd` builds (caveat: I have not tested it) - You should be able to boot a VM with `init=/nix/store/...-eiwd/bin/eiwd` and run it as PID 1 (note: I have not tried this).
I've been running this on my daily driver laptop for the last few weeks and am extremely happy with it.
Thanks for the recommendation! IMO, we're not too far away from worrying about Wi-Fi — it would just involve setting up Wi-Fi modaliases to pass through to the appropriate VM, and setting up the VM to do the right thing if any Wi-Fi devices were connected to it. For embedded use cases, where you could just bake the network configuration in to a custom netvm, that'd be it. But for e.g. a laptop, for Wi-Fi to be useful we'd really need a way to have persistent storage for on-the-fly network configuration. That's the main reason I haven't looked at Wi-Fi yet. It's a very pressing problem, though, so hopefully won't be too long...
participants (4)
-
Adam Joseph -
Alyssa Ross -
Alyssa Ross -
Ville Ilvonen