On Tue, Mar 03, 2026 at 12:56:46PM -0500, Demi Marie Obenour wrote:
Spectrum (https://spectrum-os.org) is going to be implementing virtio devices outside of the host. One proposed method of doing this is virtio-vhost-user, which is a virtio device that allows a VM to expose a vhost-user device to another VM. For instance, one could assign a NIC to one VM and have it provide a vhost-user-net device for use by a different VM.
I brought this up on the KVM/QEMU community call today. Alex Bennée recommended using virtio-msg instead. However, I have a few concerns with this:
1. Virtio-msg buses are specific to a given hypervisor or (in the case of FF-A) to a given CPU architecture. None of the current buses support KVM on platforms other than Arm64. Therefore, a brand-new bus would be needed.
Hi Demi, The virtio-msg AMP PCI bus works on KVM both for ARM and x86 and any other arch, it's generic. These are the patches we posted to qemu-devel: https://lore.kernel.org/qemu-devel/20260224155721.612314-1-edgar.iglesias@gm...
2. Virtio-msg requires not-yet-upstream drivers in both the frontend (the VM using the device) and the backend (the VM providing the device). Vhost-user uses any of the existing transports, such as PCI or MMIO. This means that upstream drivers can be used in the frontend, and also enables supports for Windows and other guests that lack support for virtio-msg.
Fair point, a bit of chicken and egg...
3. Vhost-user is already widely deployed, so frontend implementations are quite well tested. A KVM-specific virtio-msg transport would serve only one purpose: driver VMs (with assigned devices) on non-Arm64 platforms. This is a quite niche use-case. Therefore, I'm concerned that the needed frontend code will be poorly tested and bitrot.
Manos Pitsidianakis stated that vhost-user does not make sense in this case. Why is that? Would it make sense to use virtio-msg between VMM and its VM, and expose a vhost-user device to the outside world? What about having the virtio-vhost-user guest driver emulate a virtio-msg transport, so that it can be used with any device implementation supporting virtio-msg?
I would greatly appreciate any and all suggestions here. This is a serious project that is going to be used in production, but I want to ensure that the design is the best possible.
I've not looked in detail at virtio-vhost-user but it seems to be a bit similar to virtio-msg over PCI AMP with some differences. My take is: vhost-user is an interface designed for splitting virtio transport from virtio device backend in user-space on the same OS instance. The virtio-vhost-user device, enables the use of vhost-user across VM's streching vhost-user's intended use. This is a great step but Since vhost-user is not a virtio transport, It is not end-to-end in the sense that you need QEMU to translate virtio-pci to vhost-user and tunnel it over the virtio-vhost-user device to the backend. Virtio-msg is from the start a transport meant to work between heterogenous systems, different OS, OS instances even different SoCs. It's end-to-end in the sense that if you have a front-end driver in the front-end VM with an appropriate virtio-msg bus you can talk directly with the backend without intermediate proxies, potentially without VM exits. Cheers, Edgar