- Spectrum Discuss - spectrum-os.org

make run fails: make-gpt.sh: 9: set: Illegal option -o pipefail
by Dan Connolly 01 May '26

01 May '26

Hi. Spectrum looks very interesting! I look forward to more progress. I just added it to https://github.com/dckc/awesome-ocap#os I tried kicking the tires... I set up the binary cache, cloned 2026-03-18 17:11 93839a1 , and then tried `nix-shell --run 'make run'`. My fans spun for a little while building rust crates, and then I got: Build completed. ------ Filesystem UUID: d14e3b49-df7d-487c-9d85-8f6f70b51af0 Filesystem total blocks: 2076633 (of 4096-byte blocks) Filesystem total inodes: 86328 Filesystem total metadata blocks: 19765 Filesystem total deduplicated bytes (of source files): 0 mkdir -p build veritysetup format \ --root-hash-file build/rootfs.verity.roothash \ -- build/rootfs build/rootfs.verity.superblock VERITY header information for build/rootfs.verity.superblock. UUID: 8d368df1-69bb-4f4b-9633-9e1555d3d492 Hash type: 1 Data blocks: 2076633 Data block size: 4096 [bytes] Hash blocks: 16352 Hash block size: 4096 [bytes] Hash algorithm: sha256 Salt: 64d5042171257c589206d50ed727edaeba98da7f7be5dfb3d38b5cf8a5c04288 Root hash: 7919b30b902cda65737af583b1b9c6defeff3a60ba668cc1012300dfd9fa9ef9 Hash device size: 66981888 [bytes] echo >> build/rootfs.verity.roothash touch -- build/verity-timestamp ../../scripts/make-gpt.sh build/live.img.tmp \ build/rootfs.verity.superblock:verity:$(../../scripts/format-uuid.sh "$(dd if=build/rootfs.verity.roothash bs=32 skip=1 count=1 status=none)"):Spectrum_'0.0.0.verity' \ build/rootfs:root:$(../../scripts/format-uuid.sh "$(head -c 32 build/rootfs.verity.roothash)"):Spectrum_'0.0.0' ../../scripts/make-gpt.sh: 9: set: Illegal option -o pipefail make: *** [Makefile:116: build/live.img] Error 2 I'm using nix on Ubuntu 24.04.4 LTS, not NixOS. As to what version of nixpkgs I might have installed or something, I still get dizzy every time I try to find out. -- Dan Connolly https://www.madmode.com

2 3

Re: [BUG] ext4: BUG_ON in ext4_write_inline_data (fs/ext4/inline.c:240)
by Demi Marie Obenour 29 Apr '26

29 Apr '26

On 4/29/26 00:40, Theodore Tso wrote: > On Tue, Apr 28, 2026 at 04:50:14PM -0400, Demi Marie Obenour wrote: >> This is an example of a BadUSB attack, which has been known since >> at least 2014. USB sticks *do* have their own microcontrollers to >> run their firmware. At least in the past this firmware has been >> programmable and not been digitally signed. This means that the USB >> stick *can* be reprogrammed to perform a TOCTOU attack on a filesystem, >> or indeed to implement a completely different kind of USB device. > > Honestly, if that's what you are worried about, then best solution is > put epoxy in every single USB port. I've since financial institutions > that have done precisely this, and both Android and Chrome OS supports > enterprise security policies which does the equivalent in software. That works well in enterprises using laptops, tablets, or mobile devices. Enterprises can require that all devices have built-in, non-USB input devices or touchscreens. Furthermore, corporate environments use network-based backup and file sharing. So there is hardly a need for USB except for security tokens, smart card readers, OS installation media, and miscellaneous hardware devices that generally will not be hotplugged. Only the third is a block device, it is trusted, and it can be a USB stick with a physical write-protect switch and signed firmware. >> Protecting against replay attacks requires a Merkle tree. The only >> Linux filesystems that I know have one are ZFS, bcachefs, and BTRFS. >> The first two are out of tree and the third is not shipped in RHEL >> at least. > > FYI, there was a patch for btrfs, but it was never landed. It's > unclear how many people would be willing to pay the performance tax of > using hmac-sha256 for every single data and metadata block write. I thought BTRFS already had one. In any case corrupting an encrypted disk will cause a checksum failure with fairly high probability, as a 128-bit region is completely scrambled. >> Of course, you are free to choose which (if any) of these attacks you >> care about. One can that USB sticks should be mounted in userspace, >> UEFI secure boot with Microsoft keys is irrelevant as long as >> administrator -> kernel isn't a security boundary on Windows, and that >> confidential computing only makes sense for stateless workloads (which >> can use dm-verity) until there is a way to trust storage devices. >> But it's always best to be aware that an attack vector exists, >> whether or not one chooses to address it. > > Sure, but a drive-by comment on a patch review advocating that we slow > down ext4 to protect a single instance where the attacker has > read/write access to a mounted block device, when the file system > doesn't have generalized protections against that whole class of > attacks.... isn't particularly helpful. My main goal is to point out that the attack vector does exist. If nothing else, hopefully this will persuade distro maintainers to switch to using libguestfs + FUSE as the default way to mount USB drives. That isolates the driver in a VM. Qubes OS users can already mount the device in a disposable VM, and presumably many of them already do that. Again, this provides strong isolation and severely limits the impact of an exploit. I do wonder if this could be used against confidential computing workloads. That said, work there would more likely be put into allowing them to attest their storage. > By the way, I'm not aware of *any* company that has been interested in > funding work to protect against this class of attacks. Given that > most file system developers prefer food with their meals, and have > enough *other* unfunded mandates from our user community, it doesn't > seem likely that we're going to see much forward progress towards your > desires/interests. I'm not surprised at all. The people who would benefit the most from this work are consumers who are running desktop Linux on general purpose computers they own. I have spent most of my career providing secure solutions for these people. I worked on Qubes OS for several years, and I now work on Spectrum. Unfortunately, this group has very little budget and so very little market power. Therefore, work that benefits them is perpetually and massively underfunded. Grants do exist, and crowdfunding might also be an option. However, unless one is very passionate about the client space, it is hard to resist much better-paying jobs in the server world. -- Sincerely, Demi Marie Obenour (she/her/hers)

1 0

GPU IOV in spectrum
by No Name 24 Mar '26

24 Mar '26

What's the status of GPU/SR-IOV in spectrum? Looks like Intel & AMD has made it somewhat possible https://www.phoronix.com/news/AMD-GIM-Open-Source (avail to firepro-s cards which are cheap nowadays) https://forum.level1techs.com/t/proxmox-9-0-intel-b50-sr-iov-finally-its-al… (& seems like a lot of Intel iGPUs support sriov) Any plans on implementing support? Blocked by wider hardware mfr support? & likely indefinitely blocked due to legacy systems?

2 1

[ANNOUNCE] cloud-hypervisor-50.0-spectrum0
by Alyssa Ross 20 Dec '25

20 Dec '25

I've updated Spectrum's Cloud Hypervisor patches (which add virtio-gpu support) to support Cloud Hypervisor 50.0. More information is available at <https://spectrum-os.org/software/cloud-hypervisor/>.

1 0

Taking a break from This Week in Spectrum
by Alyssa Ross 19 Nov '25

19 Nov '25

Hi all, We have a funding deadline at the end of the month, so to keep my workload manageable I'm going to take a break from writing This Week in Spectrum until we're done with that. Lots happening though! Userspace router, Flatpak VMs, system updates, persistent storage, less stuff running as root.

1 0

[ANNOUNCE] cloud-hypervisor-49.0-spectrum0
by Alyssa Ross 10 Nov '25

10 Nov '25

I've updated Spectrum's Cloud Hypervisor patches (which add virtio-gpu support) to support Cloud Hypervisor 49.0. More information is available at <https://spectrum-os.org/software/cloud-hypervisor/>.

1 0

This Week in Spectrum, 2025-W44
by Alyssa Ross 04 Nov '25

04 Nov '25

It has been a very busy week. System updates -------------- We finally got the first posting of Demi's work on system updates using systemd-sysupdate[1]! There are still quite a few issues to work through, but it's going ins a promising direction. The most significant of those issues is the following: our current installer, eos-installer[2], works by copying a whole disk image to the target disk. Our combined image allows either booting that image directly, or booting into a system that runs eos-installer with that image. For an updatable system, the partitions that we install need to have room for the system to grow in future, and we also need to have two copies of those partitions for A/B updates. Changing these things causes the image to become unreasonably large. eos-installer supports compressed disk images, but if we did that, we'd lose the ability to directly boot the image to try it out. Demi also points out that eos-installer's approach is incompatible with systems that have e.g. firmware partitions that need to be preserved for the system to boot. Really, then, we're going to need an installer that can work at the partition level rather than the disk image level, so minimally sized partition images can be copied into future-proof bigger partitions in the installed system, B partitions can be created empty without needing a template, and existing partitions on the system can be left alone. This will also mean we get natural dual boot support. I was disappointed to learn that an off the shelf installer that works in this way, analogous to eos-installer, doesn't seem to exist, though. I asked what GNOME OS does, because I knew they previously used eos-installer and must have encountered this problem, and what we were able to figure out on Matrix is that they're in the process of writing a new installer for themselves. I plan to reach out to them to find out about their plans and see if this is something that could also be shared between image-based distros. In the meantime, though, we'll probably just get rid of the combined image, and have one non-updatable live image for testing Spectrum, and an installer image that installs a compressed disk image using eos-installer. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/20251029-updates-v1-0… [2]: https://github.com/endlessm/eos-installer "Try Spectrum" test ------------------- The updates submission, which took the approach of compressing the images and breaking the "Try Spectrum" function of the combined live/installer image, made me realise that there was no test coverage for that, so I wrote an integration test for it[3]. This was shortly before we realised dropping the combined image for the time being is probably inevitable, but even so it led to lots of nice improvements to the integration test infrastructure that will still be useful going forward. [3]: https://spectrum-os.org/git/spectrum/commit/?id=e0f5aad7901d150b1abe381ffd9… udev ---- I changed the script that assigns network devices to VMs to be idempotent[4], which was the last thing that needed to happen before Demi's patches to switch the host system to use udev[5] can be applied, so that should happen very soon. [4]: https://spectrum-os.org/git/spectrum/commit/?id=1d666582892fbdf812662b4490c… [5]: https://spectrum-os.org/lists/archives/spectrum-devel/20251103-udev-v5-0-5c… nixpkgs updates --------------- I finished bisecting the kernel regression we encountered when trying to update Nixpkgs last week. A fix is on the way[6], and I was also able to identify a workaround that could let us keep up to date in the meantime[7]. With this, the update passed all tests and I committed it. A few days later though, it was reported to me that the installer no longer started. I'm bisecting this now, and once we've figured out how to fix it, I'll be able to use all those nice integration test framework improvements to write a regression test for this, too. I used to do more manual testing of Nixpkgs updates, but really to be able to deliver updates at the pace we want, we're going to have to rely a lot less on manual testing and a lot more on automated tests, so going forward I'm going to be trying to get Nixpkgs updated more frequently, with less manual testing, and a focus on adding automated regression tests when something goes wrong. This will likely mean more of this sort of issue in the short term while we build up our test coverage, but a more robust approach to updates in the long term when we've built up our test suite. Yureka has also been continuing to work on identifying Spectrum-related build regressions in Nixpkgs quickly, which will also be important to enable a more rapid pace of updates. [6]: https://lore.kernel.org/netdev/20251030144438.7582-1-minhquangbui99@gmail.c… [7]: https://spectrum-os.org/git/spectrum/commit/?id=9a03e3ea12a40e03735132ed2ed… What a pace, right!?

1 0

This Week in Spectrum, 2025-W43
by Alyssa Ross 27 Oct '25

27 Oct '25

Hi! I'm back after being unwell for a couple of weeks. As such, I'll be covering things that have happened since the last update. Demi fixed the BPF x86_64 build failure mentioned last time by avoiding the use of userspace headers in BPF compilation environments[1]. While she was at it, she also submitted a refactor and possible small performance improvement, which is still awaiting review. [1]: https://spectrum-os.org/git/spectrum/commit/?id=db54efac07deee32f39bdec4e4b… [2]: https://spectrum-os.org/lists/archives/spectrum-devel/20251021-fix-forwarde… Yureka fixed some recently introduced build regressions in Nixpkgs[3], including a fix for iproute2 which was applied upstream[4], and then submitted a patch to update Spectrum's pinned Nixpkgs[5]. I am absolutely delighted with this, because it's the first time anybody else has taken on this regular maintenance work. Unfortunately, when I tested the Nixpkgs update, I found that the networking integration test no longer passed. A still-running bisect indicates this is probably due to the update to Linux 6.17. If so, it's probably been caught by somebody else and fixed already, so which means hopefully the fix will just be to update to an even newer Nixpkgs and cross our fingers no new build regressions have been introduced in the meantime. Yureka has also been working on automatically testing Spectrum builds against newer Nixpkgs versions, which I hope might pave the way for semi-automatic updates at some point, and/or automated notifications of regressions. [3]: https://spectrum-os.org/lists/archives/spectrum-devel/82249ddc-ae1a-4e3a-a6… [4]: https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=25b… [5]: https://spectrum-os.org/lists/archives/spectrum-devel/20251012222756.20947-… I've been working on running Flatpak applications in VMs. This is a bit more complicated than the AppImages we already support, because rather than relying on the system to provide libraries, Flatpaks are installed to a shared OSTree repository, so that they can be hermetic while still not duplicating huge amounts of common libraries between applications. We don't want to just pass through the whole Flatpak repository to application VMs, because then any application would be able to see which other applications were installed, so to do Flatpak VMs we need to determine which paths are actually needed by an application, and pass through just those. I've written a program that sets up bind mounts for a Flatpak application and its runtime to pass through to a VM (but not extensions yet), and also the glue in the Spectrum application VM to actually run a passed through Flatpak. I haven't had a chance to do a full end to end test yet, but I think everything should just about work. Thanks to work from Demi[6], file modes in Spectrum images are now fixed, rather than being affected by the build environment. As well as being an important reproducibility fix, this will be important to start running things as non-root users, where modes actually become relevant. [6]: https://spectrum-os.org/git/spectrum/commit/?id=a972ba0ce2be8fb32f4b44a7080… Demi has been working on implementing OS updates using systemd-sysupdate, and just got to the point where everything works! Hopefully we'll see that on the list soon.

1 0

This Week in Spectrum, 2025-W40
by Alyssa Ross 06 Oct '25

06 Oct '25

It was quite a week. Yureka and I were at XDC from Monday until Wednesday. My talk, "An upstream-first approach to application virtualization", was on Tuesday, and the video is now online[1]. I didn't learn that much at the conference that was directly relevant to Spectrum, but I was interested to learn from a lightning talk that kmscon has been revived and was recently repackaged in Fedora[2]. I'm still getting used to public speaking again, so the talk wasn't as smooth as I'd have liked, but it was okay. After XDC, I stopped by the Systems Meetup Dresden[3] and did a lighting talk. This one went much better, but wasn't recorded, so you'll just have to believe me. While we were away, All Systems Go[4] (AIUI basically the systemd conference) took place in Berlin, and that meant a lot of Linux mobile/desktop people were in town who are not normally. We were able to get together over the following days, and in particular we had some good conversations about what the future of Flatpak/Portals might look like. It's great to be in the room for this sort of thing because it means Spectrum's needs can be considered from the start. The designs being considered would solve our biggest pain points, so I'm optimistic, but there's of course the question of momentum / developer attention to make it happen. Hopefully we can contribute to that. [1]: https://video.tuwien.ac.at/events/xdc/v/Ow9xtIyJeTt?t=01h13m49s [2]: https://github.com/Aetf/kmscon/ [3]: https://ukvly.org/ [4]: https://all-systems-go.io/ In between the talks at XDC (and during some of the talks about graphics drivers I had no hope of understanding), I was pleasantly surprised to find myself in a particularly productive state — it must have been the conference atmosphere. It didn't make sense to try to work on anything big, so I went through my backlog of small Spectrum improvements. Image rebuild times during development have gone down by a further non-trivial two seconds on a fast laptop, more code has been integrated into the single Meson build system I aspire to, and a few other smaller things. I also managed to completely clear my review queue, although it of course started filling up again immediately after. Demi's patch to automate the lists of files to include in images is finally finished and committed, and her udev changes made progress, but they've been blocked on further review from me, since with so much stuff on I haven't had a chance since leaving XDC. She's also made a bit of progress on system updates using systemd-sysupdate, about which we'll hopefully have more to say soon. Finally, we've been discussing for a couple of days now what to do about a recently introduced build failure on x86_64. I'm already late getting this update out, so I'm not going to go into detail, but basically it's very unclear what the right way to build BPF programs that include kernel UAPI headers is, or if there even is one. The common way of using a distro's linux-headers/libc package is not correct, and likely to lead to either loud compilation failures (as we're currently experiencing) or silent hidden bugs[5]. If we can't find a good solution, we can fall back to copying the definitions we need out of the headers, but I'm hoping it won't come to that. Investigation continues. This has also highlighted the need for pre-push testing on all supported architectures — the reason this build failure got past me is that it only happens on x86_64, and I was working on aarch64. I'll try to get that addressed. [5]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… Hopefully next week will be less busy, and I'll be able to get my head down a bit and do some programming.

1 0

This Week in Spectrum, 2025-W39
by Alyssa Ross 29 Sep '25

29 Sep '25

Sent this to the wrong list at first, oops! I've got a lot going on. >.< Alyssa Ross <hi(a)alyssa.is> writes: > The big news this week is that review of Yureka's XDP forwarding program > is complete, and has now been committed to the Spectrum repo[1]. We > also saw the usual couple of small cleanups from Demi, several new > versions of her script to automatically generate file lists for > Makefiles[2], and just a few hours ago, a new version of the udev > switch[3]. > > [1]: https://spectrum-os.org/git/spectrum/commit/?id=535b6bec89a1aae6b0ee9579f9b… > [2]: https://spectrum-os.org/lists/archives/spectrum-devel/20250927-genfiles-v6-… > [3]: https://spectrum-os.org/lists/archives/spectrum-devel/20250924-udev-v2-0-60… > > On Friday I gave a talk at the Qubes OS Summit called "Rethinking the > Compartmentalization Experience"[4]. Tomorrow (Tuesday, 08:30 UTC) I'll > be doing another talk at XDC called "An Upstream-first Approach to > Application Virtualization", which will be streamed[5], and on the way > back home I'll be stopping by Systems Meetup Dresden[6], where I'll do a > lightning talk. > > [4]: https://www.youtube.com/live/HwisqKFEQ0g?feature=shared&t=16880 > [5]: https://live.video.tuwien.ac.at/watch?l=hsJNWfHvxZPnonNLxYNU5Z > [6]: https://ukvly.org/

1 0