- Spectrum Discuss - spectrum-os.org

This Week in Spectrum, 2025-W19
by Alyssa Ross 11 May '25

11 May '25

Sadly this week has mostly been a continuation of the last — I'm still recovering from dental surgery. I spent the early part of the week in too much pain to think about much else, and the later part of the week in less pain but now struggling with side effects from painkillers. From what I understand there's not anything to worry about, but I've hit some bad luck and ended up with much worse and longer lasting pain than would usually be expected from this. So this isn't going to be the type of comprehensive report I'd like to be sending out every week, but I'll list some things I have been able to get done in those moments where I've been able to focus. • Updated Spectrum's pinned Nixpkgs and made some small improvements (mostly not committed yet) as a result of improvements in updated packages. • Got the automated test I was working on to the point where it's actually testing something useful — that networking works. Still want to polish it a bit more when I can focus better before I commit it. • Made a release of the Cloud Hypervisor virtio-gpu patches[1]. • Some long overdue work on the server that among other things hosts the Spectrum website, git repos, etc. • Reported a GTK regression that has made testing Spectrum very annoying for the last few months[2]. • Tested a crosvm update in Nixpkgs. • Worked on arranging continuing funding for Spectrum. [1]: https://spectrum-os.org/lists/archives/spectrum-discuss/877c2owzjl.fsf@alys… [2]: https://gitlab.gnome.org/GNOME/gtk/-/issues/7544 Most of this stuff has been in the last 2-3 days, so hopefully I'm on a good trajectory, but I don't know how long it's going to be before I'm back to normal.

1 0

[ANNOUNCE] cloud-hypervisor-45.0-spectrum0
by Alyssa Ross 10 May '25

10 May '25

I've updated Spectrum's Cloud Hypervisor patches (which add virtio-gpu support) to support Cloud Hypervisor 45.0. More information is available at <https://spectrum-os.org/software/cloud-hypervisor/>.

1 0

This Week in Spectrum, 2025-W18
by Alyssa Ross 04 May '25

04 May '25

This week started off well — I was continuing working on the automated test I mentioned last week — but then I had two wisdom teeth removed and recovery has been hellish. So not much to report. I've fixed a couple of Nixpkgs things where I could but mostly I've been out of action. The other two are going to need to be removed sometime in the near future too… But hey, at least I've achieved my goal of a concise update!

1 0

This Week in Spectrum, 2025-W17
by Alyssa Ross 27 Apr '25

27 Apr '25

Apparently the last time I wrote an update like this was 2021-08-22[1]. Recently I've been missing them a lot, because I don't think it's ever been as easy to keep up with the progress being made as when I wrote weekly updates. Hopefully this time I can avoid going into quite the same level of detail, so they're more sustainable to keep writing. [1]: https://spectrum-os.org/lists/archives/spectrum-discuss/20210822213036.6lx5… So, here's what I did this week: Automated testing ----------------- For a long time I've had a branch on my computer that makes some significant changes to how networking in Spectrum works to try to resolve a longstanding race condition when starting VMs. I've found it difficult to work on, because it's hard for me to be confident when changing things around that I'm not going to subtly break something in a different way. It's been clear to me for a long time that the obvious answer to this is automated tests, but how they should work was a lot less clear. I'd like to avoid having to make a special testable version of the system, because I want to be testing the real thing. The aarch64 port I completed early this year has helped make the system just a bit more flexible, and a couple of weeks ago it clicked for me that perhaps it's time to make a serious attempt at this. So far, I have a program that starts Spectrum in QEMU. This is an unmodified Spectrum image, but the test uses systemd-stub's support for appending to the kernel command line via through SMBIOS Type 11 strings[2] to configure an extra serial console, which it then communicates with over a socket. Because this console was configured on the kernel command line, it has kernel logs printed to it, which could make trying to read command output from it messy. To work around that, the only interaction the test has with this console is to write a command into it to start a shell on a second serial console that isn't a destination for kernel logs, and then further interaction happens via that second console. (I tried lowering the kernel loglevel, but it seems some messages come through even when it's set to zero.) The test program then uses the Spectrum host shell to mount a filesystem on an attached disk image containing a custom VM configuration, import that VM, and run it. The VM just runs netcat to try to connect a server running on the host via the test program. If the host receives that connection, the test will pass, and once I have that working I can expand the test to make sure that it keeps working after restarting the different VMs involved. What's still to do is getting that server bit right — originally I planned to use QEMU's guestfwd[3] feature to forward connections to a specific IP address and port to an AF_UNIX socket, but that been broken for a long time[4], so instead I'll just use an AF_INET socket on the host. Since it has to be a non-loopback address, I also want the test to run itself in a network namespace so I'm not interfering with networking on the machine running it. Nothing that should be especially difficult. An early experiment with this led me to discover that Python's built-in socket library didn't expose the IP_FREEBIND socket option, so I sent a quick PR to do that[5], which was merged. [2]: https://www.freedesktop.org/software/systemd/man/latest/systemd-stub.html#S… [3]: https://www.qemu.org/docs/master/system/invocation.html#hxtool-5 [4]: https://gitlab.com/qemu-project/qemu/-/issues/1835 [5]: https://github.com/python/cpython/pull/132998 User-mode networking for development ------------------------------------ When developing the built-in Spectrum VMs, I run them directly on my NixOS development system. For the router VM in particular, it's important to test that it still connects to the network. Until now, that's only been possible with QEMU, which has built-in slirp-based user-mode networking — other VMMs would require privileged operations on the development system to have working networking, and it's important to me that the Spectrum development environment does not require special privileges. While I was working on the test and investigating QEMU's guestfwd feature, I discovered that there's a new option for user-mode networking: passt[6][7]. passt runs outside the VMM process, and implements a vhost-user interface, meaning that it should be possible to use from any VMM. I've changed the Spectrum development environment to use it when running VMs in Cloud Hypervisor[8], and also to run the router VM in Cloud Hypervisor by default[9], which means it's now easier to test that VM during development in an environment much more similar to the one it will ultimately run in as part of the Spectrum system. I also tried using passt with crosvm, but it didn't work due to crosvm's slightly strange interpretation of the vhost-user protocol. I sent a couple of patches[10][11] to crosvm to fix the incompatibility, which haven't been accepted yet but have prompted things to move in the right direction. I also sent a patch to passt itself to make it easier to use with an inetd/UCSPI-style super-server like s6-ipcserver, but a day later it still hasn't appeared in their list archive so I think it's probably waiting for somebody to find it in the list spam filter. [6]: https://passt.top/ [7]: https://www.qemu.org/docs/master/system/devices/net.html#using-passt-as-the… [8]: https://spectrum-os.org/git/spectrum/commit/?id=aac74f6165740a6b041a7205ec8… [9]: https://spectrum-os.org/git/spectrum/commit/?id=ca10038c080f396a6cf2649b726… [10]: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6491650 [11]: https://chromium-review.googlesource.com/c/crosvm/crosvm/+/6491651 Nixpkgs ------- I contributed a few fixes for build regressions that would get in the way of updating Spectrum's pinned Nixpkgs[12][13][14], and I also added debug symbols to the passt package[15]. [12]: https://github.com/NixOS/nixpkgs/pull/401563 [13]: https://github.com/NixOS/nixpkgs/pull/402027 [14]: https://github.com/NixOS/nixpkgs/pull/402055 [15]: https://github.com/NixOS/nixpkgs/pull/401971 That's it for this week. I'm at just over a thousand words, which is more than I was aiming for but better than I remember a lot of the past weekly updates being. Hopefully I'll be back with another update next week. As a reminder, you can support my work by donating to Spectrum's development through GitHub Sponsors[16] or Liberapay[17]. This helps a lot, especially with ongoing maintenance (usually fixing upstream bugs etc.) that is difficult to fund via a grant. [16]: https://github.com/sponsors/alyssais/ [17]: https://liberapay.com/qyliss

1 0

A few questions about Spectrum OS
by Jason Long 24 Apr '25

24 Apr '25

Hello,Which hypervisor does Spectrum OS use? How do I download and install it? Thank you.

2 1

[ANNOUNCE] cloud-hypervisor-44.0-spectrum0
by Alyssa Ross 18 Feb '25

18 Feb '25

I've updated Spectrum's Cloud Hypervisor patches (which add virtio-gpu support) to support Cloud Hypervisor 44.0. More information is available at <https://spectrum-os.org/software/cloud-hypervisor/>.

1 0

[ANNOUNCE] cloud-hypervisor-43.0-spectrum0
by Alyssa Ross 20 Dec '24

20 Dec '24

I've updated Spectrum's Cloud Hypervisor patches (which add virtio-gpu support) to support the recently released Cloud Hypervisor 43.0. More information is available at <https://spectrum-os.org/software/cloud-hypervisor/>.

1 0

[ANNOUNCE] cloud-hypervisor-42.0-spectrum0
by Alyssa Ross 26 Oct '24

26 Oct '24

I've updated Spectrum's Cloud Hypervisor patches (which add virtio-gpu support) to support the recently released Cloud Hypervisor 42.0. More information is available at <https://spectrum-os.org/software/cloud-hypervisor/>.

1 0

Questions About Virtio-GPU Patchset for Cloud Hypervisor
by projectmikey 25 Aug '24

25 Aug '24

Dear Spectrum Team, I hope this email finds you well. I am reaching out with a question about your patchset for Cloud Hypervisor with support for virtio-gpu [https://spectrum-os.org/software/cloud-hypervisor/] First, I want to say thanks for the work that has been done — it is much appreciated! I have successfully implemented the latest version of your patchset in my current environment. I am now curious if it can be used with multiple L2 guests, each securely utilizing different GPUs, running concurrently on an L1, and requiring that each L2 guest's resources be kept private and isolated from the others. To provide some more context, I am currently trying to achieve GPU acceleration within a nested L2 VM on GCP (L1: KVM on GCP => L2: Cloud Hypervisor). I'm using GCP rather than a bare metal environment because GCP supports nested virtualization on their affordable N1 and G2 series VMs. Since I have limited access to the GCP environment, specifically to the L0 hypervisor and L1 hypervisor layers, I am unable to modify or access BIOS settings or certain underlying configurations at those levels. I am uncertain whether my attempts to configure the environment was entirely correct. However, after extensive online research, I couldn’t find a definitive answer on whether using VFIO is possible in GCP VMs. Despite my efforts, I have not been able to bind to vfio-pci without first enabling No-IOMMU mode on the system. If secure VFIO usage cannot be achieved, I'm open to exploring alternatives like virtio or vfio-user, provided they can securely allocate GPU access within the L2s without memory or resource sharing between the VMs or other potential security issues that I'm not yet aware of. Do you know if this is possible with the current version of your patchset? If not, do you have any suggestions on how to achieve this in a nested setup like this one [https://cloud.google.com/compute/docs/instances/nested-virtualization/overv… Any other insights you could share that might point me in the right direction for accomplishing this securely would be incredibly helpful, as my knowledge in this area is limited. Thanks again. -Mike Calendo Sent with [Proton Mail](https://proton.me/) secure email.

2 1

virtio-gpu patches for Cloud Hypervisor 41.0
by Alyssa Ross 16 Aug '24

16 Aug '24

The cloud-hypervisor-40.0-spectrum0 patches seem to still apply and work, so no new release for Cloud Hypervisor 41 is necessary.

1 0