- Spectrum Development - spectrum-os.org

[PATCH v2] host/start-vmm: replace jq calls with miniserde
by Yureka 25 Aug '24

25 Aug '24

Signed-off-by: Yureka <yuka(a)yuka.dev> --- host/start-vmm/ch.rs | 39 ++++++++++++++++----------------------- 1 file changed, 16 insertions(+), 23 deletions(-) diff --git a/host/start-vmm/ch.rs b/host/start-vmm/ch.rs index cf164c6..46b198a 100644 --- a/host/start-vmm/ch.rs +++ b/host/start-vmm/ch.rs @@ -1,5 +1,6 @@ // SPDX-License-Identifier: EUPL-1.2+ // SPDX-FileCopyrightText: 2022-2024 Alyssa Ross <hi(a)alyssa.is> +// SPDX-FileCopyrightText: 2024 Yureka <yuka(a)yuka.dev> use std::ffi::{CStr, OsStr, OsString}; use std::io::Write; @@ -9,7 +10,7 @@ use std::os::raw::{c_char, c_int}; use std::os::unix::prelude::*; use std::process::{Command, Stdio}; -use miniserde::{json, Serialize}; +use miniserde::{json, Deserialize, Serialize}; use crate::net::MacAddress; @@ -126,34 +127,26 @@ pub fn create_vm(vm_name: &str, mut config: VmConfig) -> Result<(), String> { } pub fn add_net(vm_name: &str, net: &NetConfig) -> Result<OsString, NonZeroI32> { - let mut ch_remote = command(vm_name, "add-net") + let ch_output = command(vm_name, "add-net") .arg(format!("fd={},mac={}", net.fd, net.mac)) - .stdout(Stdio::piped()) .spawn() - .or(Err(EPERM))?; + .or(Err(EPERM))? + .wait_with_output() + .or(Err(EPROTO))?; - let jq_out = match Command::new("jq") - .args(["-j", ".id"]) - .stdin(ch_remote.stdout.take().unwrap()) - .stderr(Stdio::inherit()) - .output() - { - Ok(o) => o, - Err(_) => { - // Try not to leave a zombie. - let _ = ch_remote.kill(); - let _ = ch_remote.wait(); - return Err(EPERM); - } - }; + if !ch_output.status.success() { + return Err(EPROTO); + } - if let Ok(ch_remote_status) = ch_remote.wait() { - if ch_remote_status.success() && jq_out.status.success() { - return Ok(OsString::from_vec(jq_out.stdout)); - } + #[derive(Deserialize)] + struct AddNetOutput { + id: String, } - Err(EPROTO) + let output_str = std::str::from_utf8(&ch_output.stdout).map_err(|_| EPROTO)?; + let output_parsed: AddNetOutput = json::from_str(output_str).map_err(|_| EPROTO)?; + + Ok(OsString::from_vec(output_parsed.id.into())) } pub fn remove_device(vm_name: &str, device_id: &OsStr) -> Result<(), NonZeroI32> { -- 2.45.2

1 0

[PATCH] host/start-vmm: replace jq calls with miniserde
by Yureka 25 Aug '24

25 Aug '24

--- host/start-vmm/ch.rs | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/host/start-vmm/ch.rs b/host/start-vmm/ch.rs index cf164c6..abc6a00 100644 --- a/host/start-vmm/ch.rs +++ b/host/start-vmm/ch.rs @@ -9,7 +9,7 @@ use std::os::raw::{c_char, c_int}; use std::os::unix::prelude::*; use std::process::{Command, Stdio}; -use miniserde::{json, Serialize}; +use miniserde::{json, Deserialize, Serialize}; use crate::net::MacAddress; @@ -126,34 +126,26 @@ pub fn create_vm(vm_name: &str, mut config: VmConfig) -> Result<(), String> { } pub fn add_net(vm_name: &str, net: &NetConfig) -> Result<OsString, NonZeroI32> { - let mut ch_remote = command(vm_name, "add-net") + let ch_output = command(vm_name, "add-net") .arg(format!("fd={},mac={}", net.fd, net.mac)) - .stdout(Stdio::piped()) .spawn() - .or(Err(EPERM))?; + .or(Err(EPERM))? + .wait_with_output() + .or(Err(EPROTO))?; - let jq_out = match Command::new("jq") - .args(["-j", ".id"]) - .stdin(ch_remote.stdout.take().unwrap()) - .stderr(Stdio::inherit()) - .output() - { - Ok(o) => o, - Err(_) => { - // Try not to leave a zombie. - let _ = ch_remote.kill(); - let _ = ch_remote.wait(); - return Err(EPERM); - } - }; + if !ch_output.status.success() { + return Err(EPROTO); + } - if let Ok(ch_remote_status) = ch_remote.wait() { - if ch_remote_status.success() && jq_out.status.success() { - return Ok(OsString::from_vec(jq_out.stdout)); - } + #[derive(Deserialize)] + struct AddNetOutput { + id: String, } - Err(EPROTO) + let output_str = std::str::from_utf8(&ch_output.stdout).map_err(|_| EPROTO)?; + let output_parsed: AddNetOutput = json::from_str(output_str).map_err(|_| EPROTO)?; + + Ok(OsString::from_vec(output_parsed.id.into())) } pub fn remove_device(vm_name: &str, device_id: &OsStr) -> Result<(), NonZeroI32> { -- 2.45.2

2 1

Binary cache key rotation
by Alyssa Ross 16 Aug '24

16 Aug '24

Some of you may know that a few months ago, the Spectrum binary cache builder broke, because a change to the TLS on github.com (where the builder downloaded NixOS netboot images from) meant that Vultr's iPXE was no longer able to connect to it, and Vultr told me they couldn't give a timeline on updating iPXE to include the fix[1]. In the time since then, I've been working on a new binary cache builder with no need for iPXE — more information in the commit message[2]. One of the nice things about the new design is that it allows for being a little bit more careful with the key — it's stored in Scaleway's Secret Manager[3] (which stores it encrypted), and it's never written to disk unencrypted by the builder. Given this slightly higher level of security for the key, it makes sense to transition to a new key that has never been stored outside of this arrangement. (I generated it from Tails.) So, if you have Nix configured to trust the old binary cache key, spectrum-os.org-1:rnnSumz3+Dbs5uewPlwZSTP0k3g/5SRG4hD7Wbr9YuQ=, you should replace that in your configuration with the new key, spectrum-os.org-2:foQk3r7t2VpRx92CaXb5ROyy/NBdRJQG2uX2XJMYZfU=. During the Tails session where I generated the new key, I also generated and uploaded signatures for all store paths in the binary cache that had a valid signature from the old key, so it's possible to distrust the old key without losing the ability to substitute old paths from the binary cache. In future, all store paths built by the builder will only be signed with the new key. I've updated the binary cache documentation to describe the new binary cache[4]. The rendered documentation on the website will update once the new builder has completed its first build. This is a small evolutionary step for builder security — I'd probably still want to do more before using it to build non-development images, for example having the key on an HSM rather than being on the builder's filesystem. [1]: https://github.com/ipxe/ipxe/commit/1d1cf74a5e58811822bee4b3da3cff7282fcdfca [2]: https://spectrum-os.org/git/infra/commit/?id=ef9717440ff4e000cb50009bb68ba3… [3]: https://www.scaleway.com/en/secret-manager/ [4]: https://spectrum-os.org/git/spectrum/commit/?id=f5a75c9739d9ab23323734ccdfd…

1 0

ComputingProfessionals
by info 14 Apr '24

14 Apr '24

Hello. I am currently looking to sell my domain name ComputingProfessionals (.com). My asking price for this domain name is 479 USD. The domain is registered at GoDaddy. Please let me know if you might be interested in it. Thank you!

1 0

Upcoming Wayland projects in Spectrum
by Alyssa Ross 20 Sep '23

20 Sep '23

Hi, I was asked to give an overview of what Wayland-related stuff is on the radar for the near future, so here we go: crosvm security contexts ------------------------ The Wayland security context protocol[1] is how the compositor will know which VM is responsible for which Wayland client, which will allow us to make per-VM policy decisions. crosvm is the part of Spectrum's stack that can give this information to the compositor, so we need it to implement this protocol. Puck previously created such an implementation[2], but it was for an earlier draft of the protocol. It needs to be updated and submitted upstream. [1]: https://wayland.app/protocols/security-context-v1 [2]: https://puck.moe/git/crosvm/commit/?h=wayland-security-context&id=dbdba0bf6… Customisable COSMIC ------------------- I plan to use the COSMIC[3] the compositor for Spectrum. As far as I know, it's the only serious Rust Wayland compositor, and it looks like it's targeting a good balance between wide usability and configurability. But Spectrum's compositor has some requirements that wouldn't make sense for upstream COSMIC. For example, we'll want to implement some sort of clipboard permissions system, and we're likely to want to do some sort of custom window decorations, maybe like Qubes does. Maintaining a fork of cosmic-comp wouldn't be sustainable, so what we need is some way to customize the compositor's behaviour without needing out-of-tree patches. The way I imagine this working is having cosmic-comp export a library that allows inserting hooks for things we might want to customise. Then we could write a very small program that used that library, set up whatever hooks we wanted, and then just told the library to run the compositor. Quick sketch: use cosmic_comp::{Hook, add_hook, run}; fn main() { add_hook(Hook::Paste, |ctx| { ... }); run(); } I've talked to upstream about this, and they're on board with the general concept. Doing it this way would mean the only code we'd have to maintain would be Spectrum-specific, and we wouldn't have to worry about running into merge conflicts forever. Once this is done, we can do things like implement security contexts in cosmic-comp. It wouldn't make much sense to do that first, because it wouldn't be used anywhere. [3]: https://github.com/pop-os/cosmic-epoch#cosmic-comp COSMIC title bars for wayland-proxy-virtwl Xwayland clients ----------------------------------------------------------- wayland-proxy-virtwl[4] is the program used to relay Wayland connections from the guest to the host. It implements its own unpriveleged X11 forwarding. (Normal Xwayland requires the compositor give special priveleges not given to normal Wayland clients.) For some reason, cosmic-comp does not draw title bars for these clients. It's unclear so far whether this is a cosmic-comp program or a wayland-proxy-virtwl problem. [4]: https://github.com/talex5/wayland-proxy-virtwl

1 0

[PATCH v3 0/6] Initial support for VM Wayland
by Alyssa Ross 28 Jul '23

28 Jul '23

Changes since v2: - Disable virgl for virtio-gpu devices - Ensure wayland-proxy-virtwl isn't started before card0 appears - Build everything with musl - Support testing netvm with crosvm in addition to appvms v1: https://spectrum-os.org/lists/archives/spectrum-devel/20220928170128.158379… v2: https://spectrum-os.org/lists/archives/spectrum-devel/20220930213533.171061… This series adds the Spectrum-side support for running VMs that can display Wayland windows on the host compositor, using virtio-gpu. There are various small things still to be resolved, but it's ready for other people to try out and test. The easiest way to test this out is to run "vm-start hello-wayland" on the Spectrum host. If everything goes well, a small window with a picture of a cat (hello-wayland) should appear. This window is running in a VM. Currently, Wayland clients more complex than hello-wayland are unlikely to work, due to a couple of upstream bugs: - https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4584252 - https://chromium-review.googlesource.com/c/crosvm/crosvm/+/4684048 With those two fixes applied, I've been able to run much more complex applications, e.g. Krita. But I don't want to keep this waiting any longer, so let's get the basics in, and not block on fixing the remaining upstream problems. This has felt so close for so long, so I'm not going to miss this opportunity. As for what took this so long, mostly it's been stuff that isn't part of this series. Fixing crosvm, refining my cloud-hypervisor vhost-user-gpu implementation, etc. The previous submissions of this series were marked as RFC because there were bits I just wasn't quite happy enough with to merge them at the time — having to use Glibc for certain components, having virgl always enabled, etc. Those things have all been resolved. Alyssa Ross (6): img/app: allow waiting for GPU to appear vm-lib: add mesa drivers to VM img/app: add support for testing virtio-gpu vm: add support for testing in crosvm host: add support for Wayland in VMs vm/app: add hello-wayland demo VM .../using-spectrum/creating-vms.adoc | 5 ++++ host/initramfs/extfs.nix | 4 ++- host/rootfs/Makefile | 4 +++ host/rootfs/default.nix | 6 ++-- host/rootfs/etc/s6-rc/ext-rc-init/up | 7 +++++ host/rootfs/etc/template/gpu/data/check | 5 ++++ host/rootfs/etc/template/gpu/notification-fd | 1 + .../etc/template/gpu/notification-fd.license | 2 ++ host/rootfs/etc/template/gpu/run | 9 ++++++ host/rootfs/etc/template/gpu/type | 1 + host/rootfs/etc/template/gpu/type.license | 2 ++ host/start-vm/lib.rs | 10 +++++++ img/app/Makefile | 25 +++++++++++++++- img/app/etc/mdev.conf | 3 +- img/app/shell.nix | 2 +- lib/common.mk | 3 ++ vm-lib/make-vm.nix | 12 ++++++-- vm/app/hello-wayland.nix | 29 +++++++++++++++++++ vm/sys/net/Makefile | 11 +++++++ vm/sys/net/shell.nix | 2 +- 20 files changed, 133 insertions(+), 10 deletions(-) create mode 100755 host/rootfs/etc/template/gpu/data/check create mode 100644 host/rootfs/etc/template/gpu/notification-fd create mode 100644 host/rootfs/etc/template/gpu/notification-fd.license create mode 100755 host/rootfs/etc/template/gpu/run create mode 100644 host/rootfs/etc/template/gpu/type create mode 100644 host/rootfs/etc/template/gpu/type.license create mode 100644 vm/app/hello-wayland.nix base-commit: 13bb6394205346d161fa6f1467b7c6ba81196fb0 -- 2.40.1

2 12

[PATCH v2] Documentation: fix accidental anchor links
by Alyssa Ross 29 May '23

29 May '23

The xref macro is for cross references to internal links within the documentation, not external links. xref:https://… results in an anchor link. The existing doc-links test didn't catch this, because it operates at the HTTP level. But the problem is easy to spot in the sources, so I've added a new check that just greps for this pattern. Reported-by: Ryan Lahfa <ryan(a)lahfa.xyz> Fixes: 97cceff ("Documentation: add contributing information") Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- v1: https://spectrum-os.org/lists/archives/spectrum-devel/20230527151106.201590… doc-anchors.nix didn't actually ever succeed in v1, oops. Documentation/contributing/communication.adoc | 26 +++++++++---------- Documentation/contributing/index.adoc | 2 +- .../contributing/writing_documentation.adoc | 18 ++++++------- release/checks/default.nix | 2 ++ release/checks/doc-anchors.nix | 12 +++++++++ 5 files changed, 37 insertions(+), 23 deletions(-) create mode 100644 release/checks/doc-anchors.nix diff --git a/Documentation/contributing/communication.adoc b/Documentation/contributing/communication.adoc index 506fa28..8288c99 100644 --- a/Documentation/contributing/communication.adoc +++ b/Documentation/contributing/communication.adoc @@ -27,17 +27,17 @@ Spectrum's developers, subscribe to *announce@spectrum-os.org*. You can discuss Spectrum and related topics in real time on IRC, or more long-form on spectrum-announce (_discuss@spectrum-os.org_). Finally, development work happens in Spectrum's -xref:https://spectrum-os.org/git/[source repositories], and is -discussed on spectrum-devel (_devel@spectrum-os.org_). +https://spectrum-os.org/git/[source repositories], and is discussed on +spectrum-devel (_devel@spectrum-os.org_). == Internet Relay Chat There is a single Internet Relay Chat (IRC) channel, -xref:ircs://irc.libera.chat:6697/spectrum[#spectrum] -on xref:https://libera.chat/[Libera.Chat], for all kinds of Spectrum -discussions. You can also join through xref:https://matrix.org/[Matrix] as -xref:https://matrix.to/#/#spectrum:libera.chat[#spectrum:libera.chat]. +link:ircs://irc.libera.chat:6697/spectrum[#spectrum] on +https://libera.chat/[Libera.Chat], for all kinds of Spectrum +discussions. You can also join through https://matrix.org/[Matrix] as +https://matrix.to/#/#spectrum:libera.chat[#spectrum:libera.chat]. If you are new to IRC, you might find Matrix the easiest way to get started. The channel is *logged*, but the logs are not available online yet. @@ -49,11 +49,11 @@ following along with it, the best thing to do is subscribe to one or more of the Spectrum mailing lists. If mailing lists are not your thing, that is okay! Try the -xref:https://spectrum-os.org/lists/hyperkitty/[Web UI], which allows -you to interact with the mailing lists much like a modern web forum. +https://spectrum-os.org/lists/hyperkitty/[Web UI], which allows you to +interact with the mailing lists much like a modern web forum. -Each list has xref:https://spectrum-os.org/lists/archives/[archives] -that can also be accessed via the web, NNTP or Atom. You are strongly +Each list has https://spectrum-os.org/lists/archives/[archives] that +can also be accessed via the web, NNTP or Atom. You are strongly encouraged to mirror them. [cols="1,1"] @@ -64,14 +64,14 @@ encouraged to mirror them. from Spectrum's developers will be posted. You can subscribe -xref:https://spectrum-os.org/lists/mailman3/lists/announce.spectrum-os.org/[on the web], +https://spectrum-os.org/lists/mailman3/lists/announce.spectrum-os.org/[on the web], or by sending mail to *announce-subscribe@spectrum-os.org*. %7Cdiscuss(a)spectrum-os.org |This list contains high-level discussion about the project. You can subscribe -xref:https://spectrum-os.org/lists/mailman3/lists/discuss.spectrum-os.org/[on the web], +https://spectrum-os.org/lists/mailman3/lists/discuss.spectrum-os.org/[on the web], or by sending mail to *discuss-subscribe@spectrum-os.org*. %7Cdevel(a)spectrum-os.org @@ -84,6 +84,6 @@ For more information, see xref:working-with-patches.adoc[Working with Patches]. You can subscribe -xref:https://spectrum-os.org/lists/mailman3/lists/devel.spectrum-os.org/[on the web], or by sending mail to *devel-subscribe@spectrum-os.org*. +https://spectrum-os.org/lists/mailman3/lists/devel.spectrum-os.org/[on the web], or by sending mail to *devel-subscribe@spectrum-os.org*. |=== diff --git a/Documentation/contributing/index.adoc b/Documentation/contributing/index.adoc index 48255d8..03eac68 100644 --- a/Documentation/contributing/index.adoc +++ b/Documentation/contributing/index.adoc @@ -48,7 +48,7 @@ There is also a web interface https://spectrum-os.org/lists/hyperkitty/ that you can use to browse or even post to the mailing list. Add a Signed-off-by line to each patch you submit, to indicate your -certification of the xref:https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin] for that patch. +certification of the https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin] for that patch. The easiest way to do this is with `git commit's -s` flag. Do not be too afraid of getting it wrong the first couple of times. diff --git a/Documentation/contributing/writing_documentation.adoc b/Documentation/contributing/writing_documentation.adoc index f8a69ef..cb09241 100644 --- a/Documentation/contributing/writing_documentation.adoc +++ b/Documentation/contributing/writing_documentation.adoc @@ -11,11 +11,11 @@ formatting, and organizing the Spectrum documentation. Please follow these guidelines and conventions when editing the documentation. -We use xref:https://nixos.org/manual/nix/stable/introduction.html[Nix] -and xref:https://github.com/just-the-docs/just-the-docs[Just the Docs] -for building the documentation. Sources are written in AsciiDoc -If you are new with it, -see xref:https://docs.asciidoctor.org/asciidoc/latest/[AsciiDoc Language Documentation]. +We use https://nixos.org/manual/nix/stable/introduction.html[Nix] and +https://github.com/just-the-docs/just-the-docs[Just the Docs] for +building the documentation. Sources are written in AsciiDoc If you are +new with it, see +https://docs.asciidoctor.org/asciidoc/latest/[AsciiDoc Language Documentation]. If you want to test your changes, see xref:../contributing/building-documentation.adoc[Building Documentation]. @@ -38,7 +38,7 @@ to suit your changing needs. == Document Layout -* xref:https://docs.asciidoctor.org/asciidoc/latest/document/header/[Document header]: +* https://docs.asciidoctor.org/asciidoc/latest/document/header/[Document header]: include your name and a revision line following the author line. Example: = Document Title @@ -64,15 +64,15 @@ abbreviations are used. ** Keep text hard-wrapped at 70-80 characters. (Most editors should be able to do this automatically.) This makes it easier to pick out specific parts in review, and also makes it easier to read in some editors. (It does not make a difference to the result as rendered in the web browser.) -** Keep the text in the description attribute hard-wrapped as well. For more information, see the xref:https://docs.asciidoctor.org/asciidoc/latest/document/metadata/#descri… Metadata] section of the Asciidoctor Docs. +** Keep the text in the description attribute hard-wrapped as well. For more information, see the https://docs.asciidoctor.org/asciidoc/latest/document/metadata/#description… Metadata] section of the Asciidoctor Docs. -* Put one sentence on each line. This makes it easy to move content around, and also easy to spot (too) long sentences. For more information, see xref:https://asciidoctor.org/docs/asciidoc-recommended-practices/#one-sente… Recommended Practices]. +* Put one sentence on each line. This makes it easy to move content around, and also easy to spot (too) long sentences. For more information, see https://asciidoctor.org/docs/asciidoc-recommended-practices/#one-sentence-p… Recommended Practices]. == Structure * Use subheading to organize information. * One topic per article. -* To make accents, use xref:https://docs.asciidoctor.org/asciidoc/latest/blocks/admonitions/[admon…. +* To make accents, use https://docs.asciidoctor.org/asciidoc/latest/blocks/admonitions/[admonition…. * Use numbered lists for actions that happen in sequence. * Do not use parenthesis for additional information, make a separate sentence instead. diff --git a/release/checks/default.nix b/release/checks/default.nix index f8a3f9b..100de9a 100644 --- a/release/checks/default.nix +++ b/release/checks/default.nix @@ -8,6 +8,8 @@ import ../../lib/eval-config.nix ({ ... } @ args: doc-links = import ./doc-links.nix args; + doc-anchors = import ./doc-anchors.nix args; + pkg-tests = import ./pkg-tests.nix args; reuse = import ./reuse.nix args; diff --git a/release/checks/doc-anchors.nix b/release/checks/doc-anchors.nix new file mode 100644 index 0000000..1fae7e3 --- /dev/null +++ b/release/checks/doc-anchors.nix @@ -0,0 +1,12 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi(a)alyssa.is> + +import ../../lib/eval-config.nix ({ config, ... }: +config.pkgs.callPackage ({ runCommand }: + +runCommand "spectrum-doc-anchors" {} '' + cd ${(import ../../Documentation { inherit config; }).src} + ! grep --color=always -r xref:http + touch $out +'' +) { }) base-commit: 1c6893145bdf023cd03b7675a18749ce8ed3803f -- 2.37.1

2 1

[PATCH] release/checks/try.nix: init
by Alyssa Ross 27 May '23

27 May '23

This is a regression test for c7f87f3 ("host/rootfs: allow growing ext partition to fail"). It's the first time we're actually doing automated tests of a Spectrum boot. For now, I'm using the NixOS test framework, but because we're not using NixOS and not setting any NixOS options, it feels to me like it doesn't actually buy us very much, so if it doesn't start adding more value as we add more (or more complex) tests, it might be simpler to just use a shell/execline script for tests. Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- Documentation for reviewing patches: https://spectrum-os.org/doc/contributing/reviewing-patches.html release/checks/default.nix | 2 ++ release/checks/try.nix | 50 ++++++++++++++++++++++++++++++++++++++ release/live/default.nix | 2 +- 3 files changed, 53 insertions(+), 1 deletion(-) create mode 100644 release/checks/try.nix diff --git a/release/checks/default.nix b/release/checks/default.nix index f8a3f9b..bf26b9a 100644 --- a/release/checks/default.nix +++ b/release/checks/default.nix @@ -15,4 +15,6 @@ import ../../lib/eval-config.nix ({ ... } @ args: rustfmt = import ./rustfmt.nix args; shellcheck = import ./shellcheck.nix args; + + try = import ./try.nix args; }) diff --git a/release/checks/try.nix b/release/checks/try.nix new file mode 100644 index 0000000..73539ef --- /dev/null +++ b/release/checks/try.nix @@ -0,0 +1,50 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi(a)alyssa.is> + +import ../../lib/eval-config.nix ({ config, ... }: + +let + live = import ../live { inherit config; }; +in + +config.pkgs.nixosTest ({ pkgs, ... }: { + name = "try-spectrum-test"; + nodes = {}; + + testScript = '' + import shlex + import subprocess + + conf = subprocess.run([ + "${pkgs.mtools}/bin/mcopy", + "-i", + "${live}@@1M", + "::loader/entries/spectrum.conf", + "-", + ], stdout=subprocess.PIPE) + conf.check_returncode() + + cmdline = None + for line in conf.stdout.decode('utf-8').splitlines(): + key, value = line.split(' ', 1) + if key == 'options': + cmdline = value + + flags = " ".join(map(shlex.quote, [ + "qemu-kvm", + "-m", "512", + "-kernel", "${live.rootfs.kernel}/${pkgs.stdenv.hostPlatform.linux-kernel.target}", + "-initrd", "${live.initramfs}", + "-device", "qemu-xhci", + "-device", "usb-storage,drive=drive1,removable=true", + "-drive", "file=${live},id=drive1,format=raw,if=none,readonly=on", + "-append", f"panic=-1 {cmdline}", + ])) + + machine = create_machine({"startCommand": flags}) + + machine.start() + machine.wait_for_console_text("EXT4-fs $sda4$: mounted filesystem") + machine.crash() + ''; +})) diff --git a/release/live/default.nix b/release/live/default.nix index b7ee036..6df73d4 100644 --- a/release/live/default.nix +++ b/release/live/default.nix @@ -48,6 +48,6 @@ stdenvNoCC.mkDerivation { enableParallelBuilding = true; - passthru = { inherit rootfs; }; + passthru = { inherit initramfs rootfs; }; } ) {}) -- 2.37.1

3 7

[PATCH] Documentation: fix accidental anchor links
by Alyssa Ross 27 May '23

27 May '23

The xref macro is for cross references to internal links within the documentation, not external links. xref:https://… results in an anchor link. The existing doc-links test didn't catch this, because it operates at the HTTP level. But the problem is easy to spot in the sources, so I've added a new check that just greps for this pattern. Reported-by: Ryan Lahfa <ryan(a)lahfa.xyz> Fixes: 97cceff ("Documentation: add contributing information") Signed-off-by: Alyssa Ross <hi(a)alyssa.is> --- Documentation/contributing/communication.adoc | 26 +++++++++---------- Documentation/contributing/index.adoc | 2 +- .../contributing/writing_documentation.adoc | 18 ++++++------- release/checks/default.nix | 2 ++ release/checks/doc-anchors.nix | 11 ++++++++ 5 files changed, 36 insertions(+), 23 deletions(-) create mode 100644 release/checks/doc-anchors.nix diff --git a/Documentation/contributing/communication.adoc b/Documentation/contributing/communication.adoc index 506fa28..8288c99 100644 --- a/Documentation/contributing/communication.adoc +++ b/Documentation/contributing/communication.adoc @@ -27,17 +27,17 @@ Spectrum's developers, subscribe to *announce@spectrum-os.org*. You can discuss Spectrum and related topics in real time on IRC, or more long-form on spectrum-announce (_discuss@spectrum-os.org_). Finally, development work happens in Spectrum's -xref:https://spectrum-os.org/git/[source repositories], and is -discussed on spectrum-devel (_devel@spectrum-os.org_). +https://spectrum-os.org/git/[source repositories], and is discussed on +spectrum-devel (_devel@spectrum-os.org_). == Internet Relay Chat There is a single Internet Relay Chat (IRC) channel, -xref:ircs://irc.libera.chat:6697/spectrum[#spectrum] -on xref:https://libera.chat/[Libera.Chat], for all kinds of Spectrum -discussions. You can also join through xref:https://matrix.org/[Matrix] as -xref:https://matrix.to/#/#spectrum:libera.chat[#spectrum:libera.chat]. +link:ircs://irc.libera.chat:6697/spectrum[#spectrum] on +https://libera.chat/[Libera.Chat], for all kinds of Spectrum +discussions. You can also join through https://matrix.org/[Matrix] as +https://matrix.to/#/#spectrum:libera.chat[#spectrum:libera.chat]. If you are new to IRC, you might find Matrix the easiest way to get started. The channel is *logged*, but the logs are not available online yet. @@ -49,11 +49,11 @@ following along with it, the best thing to do is subscribe to one or more of the Spectrum mailing lists. If mailing lists are not your thing, that is okay! Try the -xref:https://spectrum-os.org/lists/hyperkitty/[Web UI], which allows -you to interact with the mailing lists much like a modern web forum. +https://spectrum-os.org/lists/hyperkitty/[Web UI], which allows you to +interact with the mailing lists much like a modern web forum. -Each list has xref:https://spectrum-os.org/lists/archives/[archives] -that can also be accessed via the web, NNTP or Atom. You are strongly +Each list has https://spectrum-os.org/lists/archives/[archives] that +can also be accessed via the web, NNTP or Atom. You are strongly encouraged to mirror them. [cols="1,1"] @@ -64,14 +64,14 @@ encouraged to mirror them. from Spectrum's developers will be posted. You can subscribe -xref:https://spectrum-os.org/lists/mailman3/lists/announce.spectrum-os.org/[on the web], +https://spectrum-os.org/lists/mailman3/lists/announce.spectrum-os.org/[on the web], or by sending mail to *announce-subscribe@spectrum-os.org*. %7Cdiscuss(a)spectrum-os.org |This list contains high-level discussion about the project. You can subscribe -xref:https://spectrum-os.org/lists/mailman3/lists/discuss.spectrum-os.org/[on the web], +https://spectrum-os.org/lists/mailman3/lists/discuss.spectrum-os.org/[on the web], or by sending mail to *discuss-subscribe@spectrum-os.org*. %7Cdevel(a)spectrum-os.org @@ -84,6 +84,6 @@ For more information, see xref:working-with-patches.adoc[Working with Patches]. You can subscribe -xref:https://spectrum-os.org/lists/mailman3/lists/devel.spectrum-os.org/[on the web], or by sending mail to *devel-subscribe@spectrum-os.org*. +https://spectrum-os.org/lists/mailman3/lists/devel.spectrum-os.org/[on the web], or by sending mail to *devel-subscribe@spectrum-os.org*. |=== diff --git a/Documentation/contributing/index.adoc b/Documentation/contributing/index.adoc index 48255d8..03eac68 100644 --- a/Documentation/contributing/index.adoc +++ b/Documentation/contributing/index.adoc @@ -48,7 +48,7 @@ There is also a web interface https://spectrum-os.org/lists/hyperkitty/ that you can use to browse or even post to the mailing list. Add a Signed-off-by line to each patch you submit, to indicate your -certification of the xref:https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin] for that patch. +certification of the https://spectrum-os.org/git/spectrum/tree/DCO-1.1.txt[Developer's Certificate of Origin] for that patch. The easiest way to do this is with `git commit's -s` flag. Do not be too afraid of getting it wrong the first couple of times. diff --git a/Documentation/contributing/writing_documentation.adoc b/Documentation/contributing/writing_documentation.adoc index f8a69ef..cb09241 100644 --- a/Documentation/contributing/writing_documentation.adoc +++ b/Documentation/contributing/writing_documentation.adoc @@ -11,11 +11,11 @@ formatting, and organizing the Spectrum documentation. Please follow these guidelines and conventions when editing the documentation. -We use xref:https://nixos.org/manual/nix/stable/introduction.html[Nix] -and xref:https://github.com/just-the-docs/just-the-docs[Just the Docs] -for building the documentation. Sources are written in AsciiDoc -If you are new with it, -see xref:https://docs.asciidoctor.org/asciidoc/latest/[AsciiDoc Language Documentation]. +We use https://nixos.org/manual/nix/stable/introduction.html[Nix] and +https://github.com/just-the-docs/just-the-docs[Just the Docs] for +building the documentation. Sources are written in AsciiDoc If you are +new with it, see +https://docs.asciidoctor.org/asciidoc/latest/[AsciiDoc Language Documentation]. If you want to test your changes, see xref:../contributing/building-documentation.adoc[Building Documentation]. @@ -38,7 +38,7 @@ to suit your changing needs. == Document Layout -* xref:https://docs.asciidoctor.org/asciidoc/latest/document/header/[Document header]: +* https://docs.asciidoctor.org/asciidoc/latest/document/header/[Document header]: include your name and a revision line following the author line. Example: = Document Title @@ -64,15 +64,15 @@ abbreviations are used. ** Keep text hard-wrapped at 70-80 characters. (Most editors should be able to do this automatically.) This makes it easier to pick out specific parts in review, and also makes it easier to read in some editors. (It does not make a difference to the result as rendered in the web browser.) -** Keep the text in the description attribute hard-wrapped as well. For more information, see the xref:https://docs.asciidoctor.org/asciidoc/latest/document/metadata/#descri… Metadata] section of the Asciidoctor Docs. +** Keep the text in the description attribute hard-wrapped as well. For more information, see the https://docs.asciidoctor.org/asciidoc/latest/document/metadata/#description… Metadata] section of the Asciidoctor Docs. -* Put one sentence on each line. This makes it easy to move content around, and also easy to spot (too) long sentences. For more information, see xref:https://asciidoctor.org/docs/asciidoc-recommended-practices/#one-sente… Recommended Practices]. +* Put one sentence on each line. This makes it easy to move content around, and also easy to spot (too) long sentences. For more information, see https://asciidoctor.org/docs/asciidoc-recommended-practices/#one-sentence-p… Recommended Practices]. == Structure * Use subheading to organize information. * One topic per article. -* To make accents, use xref:https://docs.asciidoctor.org/asciidoc/latest/blocks/admonitions/[admon…. +* To make accents, use https://docs.asciidoctor.org/asciidoc/latest/blocks/admonitions/[admonition…. * Use numbered lists for actions that happen in sequence. * Do not use parenthesis for additional information, make a separate sentence instead. diff --git a/release/checks/default.nix b/release/checks/default.nix index f8a3f9b..100de9a 100644 --- a/release/checks/default.nix +++ b/release/checks/default.nix @@ -8,6 +8,8 @@ import ../../lib/eval-config.nix ({ ... } @ args: doc-links = import ./doc-links.nix args; + doc-anchors = import ./doc-anchors.nix args; + pkg-tests = import ./pkg-tests.nix args; reuse = import ./reuse.nix args; diff --git a/release/checks/doc-anchors.nix b/release/checks/doc-anchors.nix new file mode 100644 index 0000000..71c2fc9 --- /dev/null +++ b/release/checks/doc-anchors.nix @@ -0,0 +1,11 @@ +# SPDX-License-Identifier: MIT +# SPDX-FileCopyrightText: 2023 Alyssa Ross <hi(a)alyssa.is> + +import ../../lib/eval-config.nix ({ config, ... }: +config.pkgs.callPackage ({ runCommand }: + +runCommand "spectrum-doc-anchors" {} '' + cd ${(import ../../Documentation { inherit config; }).src} + ! grep --color=always -r xref:http +'' +) { }) base-commit: 1c6893145bdf023cd03b7675a18749ce8ed3803f -- 2.37.1

1 0

[PATCH 00/22] Implement managing VMs with Nix
by Alyssa Ross 26 Feb '23

26 Feb '23

IMPORTANT NOTE: this series should be applied on top of v2 of my previous series "Introduce a shared base for application VMs" [1]. I'm much happier with v2 of that series, but I only posted it yesterday so I still want to leave a little more opportunity for comment before applying it. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20221009114036.463071… This series contains the final big chunk of work I had left to do on Spectrum's original NLnet grant. It adds support for managing Spectrum VMs from the Spectrum system itself using Nix. Nix is optional, and can co-exist with VMs provided in some other way. More information is included in the new documentation. Most of this work was done earlier this year, but I got stuck on some implementation details that prevented me from getting over the last hurdle until I came up with a solution. That's explained in more detail in patch 15. Patches 1–10 add support for configuring VMs with read/write access to host directories using virtiofs. Then, in patches 11–14, come various changes that make the default user data partition more suitable as a mutable filesystem, which we haven't actually used it for before. And then the remaining patches actually implement support for a VM that can run Nix and easily build VMs that are available on the host. Alyssa Ross (22): host/start-vm: use MAP_SHARED memory for VMs host/start-vm: implement shared directories host/rootfs: generate virtiofsd services Documentation: explain VM shared directories vm-lib/make-vm.nix: support shared directories img/app: add support for testing virtiofs img/app: don't block app startup on network online img/app: auto-mount virtiofs0 filesystem vm/app/mg.nix: init vm/app/mg.nix: open virtio filesystem in dired host/rootfs: move ext mounting to s6-rc service host/rootfs: automatically grow user partition host/rootfs: use a bigger test ext partition host/initramfs/extfs.nix: tar2ext4 -> mkfs.ext4 -d host/start-vm: resolve VM symlinks with /ext root host/rootfs: resolve VM symlinks with /ext root Documentation: explain /ext symlink resolution host/start-vm: increase memory size to 512M vm/app/nix: add vm-lib/make-vms.nix: add host/initramfs/extfs.nix: add example Nix-built VM Documentation: add how-to guide for Nix-built VMs .gitignore | 5 +- Documentation/_sass/custom/custom.scss | 22 ++++++ Documentation/creating-vms.adoc | 12 ++- Documentation/nix-vms.adoc | 22 ++++++ host/initramfs/extfs.nix | 29 +++++-- host/rootfs/Makefile | 13 +++- host/rootfs/default.nix | 16 +++- host/rootfs/etc/mdev/block/add | 1 - host/rootfs/etc/s6-rc/ext-rc-init/up | 19 ++++- host/rootfs/etc/s6-rc/ext/up | 5 +- host/rootfs/etc/template/fs/notification-fd | 1 + .../etc/template/fs/notification-fd.license | 2 + host/rootfs/etc/template/fs/run | 11 +++ host/rootfs/etc/template/fs/type | 1 + host/rootfs/etc/template/fs/type.license | 2 + host/start-vm/fs.c | 17 +++++ host/start-vm/fs.rs | 68 +++++++++++++++++ host/start-vm/lib.rs | 54 ++++++++++--- host/start-vm/meson.build | 2 +- host/start-vm/start-vm.rs | 15 ++-- host/start-vm/tests/meson.build | 4 + host/start-vm/tests/vm_command-basic.rs | 6 +- .../tests/vm_command-config-symlink.rs | 30 ++++++++ host/start-vm/tests/vm_command-shared-dir.rs | 43 +++++++++++ img/app/Makefile | 24 +++++- img/app/etc/mdev.conf | 3 +- img/app/etc/mdev/iface | 4 +- img/app/etc/mdev/listen | 12 +++ img/app/etc/mdev/virtiofs | 10 +++ img/app/etc/mdev/wait | 15 ++++ img/app/etc/s6-rc/ok-all/contents | 1 + img/app/shell.nix | 6 +- tools/resolve_in_root/default.nix | 23 ++++++ tools/resolve_in_root/meson.build | 10 +++ tools/resolve_in_root/resolve_in_root.c | 76 +++++++++++++++++++ tools/resolve_in_root/test.sh | 11 +++ vm-lib/make-vm.nix | 20 ++++- vm-lib/make-vms.nix | 19 +++++ vm/app/catgirl.nix | 1 + vm/app/lynx.nix | 1 + vm/app/{lynx.nix => mg.nix} | 10 ++- vm/app/nix/bin/vm-rebuild | 25 ++++++ vm/app/nix/default.nix | 43 +++++++++++ vm/app/nix/example.nix | 13 ++++ 44 files changed, 673 insertions(+), 54 deletions(-) create mode 100644 Documentation/_sass/custom/custom.scss create mode 100644 Documentation/nix-vms.adoc create mode 100644 host/rootfs/etc/template/fs/notification-fd create mode 100644 host/rootfs/etc/template/fs/notification-fd.license create mode 100755 host/rootfs/etc/template/fs/run create mode 100644 host/rootfs/etc/template/fs/type create mode 100644 host/rootfs/etc/template/fs/type.license create mode 100644 host/start-vm/fs.c create mode 100644 host/start-vm/fs.rs create mode 100644 host/start-vm/tests/vm_command-config-symlink.rs create mode 100644 host/start-vm/tests/vm_command-shared-dir.rs create mode 100755 img/app/etc/mdev/listen create mode 100755 img/app/etc/mdev/virtiofs create mode 100755 img/app/etc/mdev/wait create mode 100644 tools/resolve_in_root/default.nix create mode 100644 tools/resolve_in_root/meson.build create mode 100644 tools/resolve_in_root/resolve_in_root.c create mode 100755 tools/resolve_in_root/test.sh create mode 100644 vm-lib/make-vms.nix copy vm/app/{lynx.nix => mg.nix} (52%) create mode 100755 vm/app/nix/bin/vm-rebuild create mode 100644 vm/app/nix/default.nix create mode 100644 vm/app/nix/example.nix -- 2.37.1

2 44