May 2025 - Spectrum Development

[PATCH] Add interrupt remapping support
by Demi Marie Obenour 30 May '25

30 May '25

This adds interrupt remapping to the virtio-IOMMU device. This is needed to support deprivileged userspace drivers on x86_64 and other platforms where interrupt remapping is the job of the IOMMU. It is not needed on AArch64 and must not be supported on that platform. --- device-types/iommu/description.tex | 70 +++++++++++++++++++++++++++++- 1 file changed, 69 insertions(+), 1 deletion(-) diff --git a/device-types/iommu/description.tex b/device-types/iommu/description.tex index f8cbe88..0a3bcab 100644 --- a/device-types/iommu/description.tex +++ b/device-types/iommu/description.tex @@ -3,7 +3,8 @@ \section{IOMMU device}\label{sec:Device Types / IOMMU Device} The virtio-iommu device manages Direct Memory Access (DMA) from one or more endpoints. It may act both as a proxy for physical IOMMUs managing devices assigned to the guest, and as virtual IOMMU managing emulated and -paravirtualized devices. +paravirtualized devices. On some platforms, it is also responsible for +interrupt remapping. The driver first discovers endpoints managed by the virtio-iommu device using platform specific mechanisms. It then sends requests to create @@ -74,6 +75,13 @@ \subsection{Feature bits}\label{sec:Device Types / IOMMU Device / Feature bits} whether endpoints that are attached to a domain are in bypass mode. +\item[VIRTIO_IOMMU_F_IRQ_MAP_UNMAP (7)] + Interrupt remapping is available. This is architecture specific + and MUST NOT be negotiated on architectures such as AArch64 where + interrupt remapping is not handled by the IOMMU. If this feature + is negotiated, all interrupts from endpoints that are not in + bypass mode are initially disabled. + \end{description} \drivernormative{\subsubsection}{Feature bits}{Device Types / IOMMU Device / Feature bits} @@ -531,6 +539,66 @@ \subsubsection{MAP request}\label{sec:Device Types / IOMMU Device / Device opera range mapped with VIRTIO_IOMMU_MAP_F_WRITE but not VIRTIO_IOMMU_MAP_F_READ. +\subsubsection{IRQ MAP request}\label{sec:Device Types / IOMMU Device / Device operations / IRQ MAP request} + +\begin{lstlisting} +struct virtio_iommu_req_irq_map { + struct virtio_iommu_req_head head; + le32 domain; + le64 msi_address; + le32 msi_payload; + le32 vector_id; + le32 vcpu_id; + le32 flags; + u8 reserved[4]; + struct virtio_iommu_req_tail tail; +}; +\end{lstlisting} + +Map a host MSI to a guest MSI directed at the given vCPU. vCPUs range from 0 to +$2^{32} - 2$ inclusive. $2^{32} - 1$ is reserved and means ``no vCPU'', +effectively disabling the interrupt. Any previous mapping of the MSI is overwritten. + +This request takes effect immediately. It is the device's job to flush caches if +necessary. However, any interrupts that are pending remain pending. + +The set of permitted values for \field{msi_address} and \field{msi_payload} +is architecture-dependent. Invalid values include, but are not limited to: + +\begin{itemize} +\item Values that are not in the architecture-dependent range of MSI addresses. +\item Values that have bits set that are ignored by the physical IOMMU. +\item Values that have reserved bits set. +\item Values that would cause the MSI to not be subject to interrupt remapping, + and which the host must therefore block for its own security. +\end{itemize} + +This request is only available when VIRTIO_IOMMU_F_IRQ_MAP_UNMAP has been negotiated. + +\drivernormative{\paragraph}{IRQ MAP request}{Device Types / IOMMU Device / Device operations / IRQ MAP request} + +The driver SHOULD set undefined \field{flags} bits to zero. + +The driver SHOULD NOT send IRQ MAP requests on a bypass domain. + +The driver SHOULD NOT choose values for \field{msi_address} or \field{msi_payload} +that are invalid for the CPU architecture. + +The driver SHOULD NOT choose a \field{vcpu_id} that corresponds to a vCPU that does +not exist and is not $2^{32} - 1$. + +\devicenormative{\paragraph}{IRQ MAP request}{Device Types / IOMMU Device / Device operations / IRQ MAP request} + +If \field{msi_address} or \field{msi_payload} is invalid for the CPU architecture, +the device SHOULD reject the request and set \field{status} to VIRTIO_IOMMU_S_INVAL. + +If \field{domain} does not exist, the device SHOULD reject the request and +set \field{status} to VIRTIO_IOMMU_S_NOENT. + +If \field{vcpu_id} does not correspond to a CPU that is running in the guest +and is not $2^{32} - 1$, the device SHOULD reject the request and set +\field{status} to VIRTIO_IOMMU_S_NOENT. + \subsubsection{UNMAP request}\label{sec:Device Types / IOMMU Device / Device operations / UNMAP request} \begin{lstlisting} -- Sincerely, Demi Marie Obenour

1 0

This Week in Spectrum, 2025-W21
by Alyssa Ross 25 May '25

25 May '25

Not too much to report this week. I spent most of the week either continuing with my recovery (this was a substantial fraction), or continuing work on future funding — hopefully I'll have good news to share about this at some point. I did update Cloud Hypervisor in Nixpkgs and release new a version of Spectrum's virtio-gpu patches for it. The good news is I'm now basically fully recovered, so should be back to normal from here on, once I've finished getting through the backlog of email. Until next week!

1 0

This Week in Spectrum, 2025-W20
by Alyssa Ross 18 May '25

18 May '25

I'm still recovering, and it's rough. There are some small signs of progress but it's a bit two steps forward, one step back. But here's what I managed to do this week in spite of things: • Updated[1] the binary cache builder scripts for Scaleway's block storage API migration. (The old one no longer works.) • Finished the networking test[2] I mentioned a couple of updates ago. • Deployed Anubis[3] on https://spectrum-os.org/git/ to stop the server being overloaded by AI scrapers. I'm unhappy it's come to this, but it made me feel better to realise it only activates on things that look like graphical web browsers, so curl, lynx, etc. still get through fine. • Dropped the hack[4] Spectrum had for loading device trees on aarch64 where they were spread across more than a dozen UKI extensions, thanks to systemd having dropped the limit of how many each UKI could include after I brought it up[5]. • Tested and merged a crosvm update in Nixpkgs[6]. • Finished some work to support encrypted user data partitions. This was a big blocker preventing Spectrum being really useful for anything. Currently, partitions have to be manually mounted on the command line, but I could imagine there being a nice GUI for this that runs on boot, a bit like what Tails has, since the OS itself doesn't have to be encrypted. • Discovered that the main blocker I knew about for building Spectrum for RISC-V had been resolved, so did some very early work for having Nixpkgs support RISC-V musl builds[7][8][9]. This one was a bit of a happy accident of my surgery recovery — I only tried it again on a lark because I wasn't well enough at the time to do anything else, and then made the happy discovery. Probably won't pursue this any more for a while, because apart from anything else obtaining RISC-V hardware at the moment isn't easy, but it's nice to know it's now theoretically possible. • Demi Marie Obenour helped me figure out that passthrough of Wi-Fi devices to VMs was not broken for complicated reasons to do with Cloud Hypervisor's VFIO implementation like I thought, but for simpler reasons: passthrough to a Fedora VM works, so it must just be some sort of misconfiguration in the Spectrum VM that should be relatively straightforward to narrow down and fix. I had tried to test passthrough to other guest OSes before, but since then I discovered that my main laptop has a faulty Wi-Fi card, so I suspect that produced confusing results that made me think it was something more complicated. • Continued work on arranging future funding for Spectrum. [1]: https://spectrum-os.org/git/infra/commit/?id=df2824a8e19c16a27adb951f664a53… [2]: https://spectrum-os.org/git/spectrum/tree/release/checks/networking?id=879b… [3]: https://anubis.techaro.lol/ [4]: https://spectrum-os.org/git/spectrum/commit/?id=e06b57f4fefc834f4868540a53d… [5]: https://github.com/systemd/systemd/commit/8c5b359579b0f1029edafb0bd96b5ebfb… [6]: https://github.com/NixOS/nixpkgs/pull/408305 [7]: https://github.com/NixOS/nixpkgs/pull/406797 [8]: https://github.com/NixOS/nixpkgs/pull/406966 [9]: https://github.com/NixOS/nixpkgs/pull/406996 Looking back at that list now, I'm pretty happy! Even though I was struggling for a lot of the week that's not too bad overall. Maybe I'm even happy enough with it that I feel justified including this bit again for the first time since the surgery: As a reminder, you can support my work by donating to Spectrum's development through GitHub Sponsors[10] or Liberapay[11]. This helps a lot, especially with ongoing maintenance (usually fixing upstream bugs etc.) that is difficult to fund via a grant. [10]: https://github.com/sponsors/alyssais/ [11]: https://liberapay.com/qyliss Until next week!

1 0

This Week in Spectrum, 2025-W19
by Alyssa Ross 11 May '25

11 May '25

Sadly this week has mostly been a continuation of the last — I'm still recovering from dental surgery. I spent the early part of the week in too much pain to think about much else, and the later part of the week in less pain but now struggling with side effects from painkillers. From what I understand there's not anything to worry about, but I've hit some bad luck and ended up with much worse and longer lasting pain than would usually be expected from this. So this isn't going to be the type of comprehensive report I'd like to be sending out every week, but I'll list some things I have been able to get done in those moments where I've been able to focus. • Updated Spectrum's pinned Nixpkgs and made some small improvements (mostly not committed yet) as a result of improvements in updated packages. • Got the automated test I was working on to the point where it's actually testing something useful — that networking works. Still want to polish it a bit more when I can focus better before I commit it. • Made a release of the Cloud Hypervisor virtio-gpu patches[1]. • Some long overdue work on the server that among other things hosts the Spectrum website, git repos, etc. • Reported a GTK regression that has made testing Spectrum very annoying for the last few months[2]. • Tested a crosvm update in Nixpkgs. • Worked on arranging continuing funding for Spectrum. [1]: https://spectrum-os.org/lists/archives/spectrum-discuss/877c2owzjl.fsf@alys… [2]: https://gitlab.gnome.org/GNOME/gtk/-/issues/7544 Most of this stuff has been in the last 2-3 days, so hopefully I'm on a good trajectory, but I don't know how long it's going to be before I'm back to normal.

1 0

This Week in Spectrum, 2025-W18
by Alyssa Ross 04 May '25

04 May '25

This week started off well — I was continuing working on the automated test I mentioned last week — but then I had two wisdom teeth removed and recovery has been hellish. So not much to report. I've fixed a couple of Nixpkgs things where I could but mostly I've been out of action. The other two are going to need to be removed sometime in the near future too… But hey, at least I've achieved my goal of a concise update!

1 0