Re: [PATCH v9 0/1] virtio-media: Add device specification
On 7/9/25 05:34, Albert Esteve wrote:
New attempt of including virtio-media device specification.
v8->v9: - Rebased to virtio-1.4
v7->v8: - Merged DQBUF and DQEVENT into the same item for unsupported ioctls - Rewrote/clarified a couple paragraphs based on previous version reviews.
Virtio-media came from a discussion on virtio-dev mailing list, which lead to Alex Courbot presenting virtio-v4l2[1] specification as an alternative to virtio-video.
Later, virtio-v4l2 was renamed to virtio-media[2] and published at:
https://github.com/chromeos/virtio-media
The repository above includes a virtio-media driver able to pass v4l2-compliance when proxying the vivid/vicodec virtual devices or an actual UVC camera using the V4L2 vhost device (available in the repository). It also includes a FFmpeg-based video encoder device. Steps to reproduce are also detailed[3].
Recently, virtio-media has landed in AOSP[4]. Also the driver patch has been sent to the kernel, currently in its v2 [5].
Furthermore, virtio-media got a proposal to reserve device ID 48, which was finally approved for inclusion in v1.4.
There is some overlap with virtio-video in regards to which devices it can handle. However, they take different approaches, potentially making them the preferable choice for different scenarios. Furthermore, virtio-media has matured since it was first presented, and there are public efforts to have upstream driver and devices.
But mainly, given that virtio-media will be the virtualization solution for media devices for ChromeOS, Android, and possibly others, I think is important that it gets standardized and included in the specification, despite the aforementioned overlap.
Full PDF: https://drive.google.com/file/d/1sDZXWVlvPbHKKguhekCXMzkpNZZ0g3hV/view?usp=s... PDF with the media section only: https://drive.google.com/file/d/1JjOIzecnkFFlpFBRjPQEUyAOD3BDHAdw/view?usp=s...
[1] https://mail.google.com/mail/u/0?ui=2&ik=73ebd65ebd&attid=0.1&permmsgid=msg-... [2] https://www.mail-archive.com/virtio-dev@lists.oasis-open.org/msg12665.html [3] https://github.com/chromeos/virtio-media/blob/main/TRY_IT_OUT.md [4] https://cs.android.com/android/platform/superproject/main/+/main:external/vi... [5] https://lore.kernel.org/lkml/20250201-virtio-media-v2-1-ac840681452d@gmail.c...
Albert Esteve (1): virtio-media: Add virtio media device specification
conformance.tex | 6 + content.tex | 1 + device-types/media/description.tex | 639 ++++++++++++++++++++++ device-types/media/device-conformance.tex | 15 + device-types/media/driver-conformance.tex | 11 + 5 files changed, 672 insertions(+) create mode 100644 device-types/media/description.tex create mode 100644 device-types/media/device-conformance.tex create mode 100644 device-types/media/driver-conformance.tex
It's probably too late, but I have some serious concerns about this device. Specifically: 1. I don't see a reasonable way to support untrusted virtio-media devices. v4l2 has too many ioctls and I can't realistically see a way to enforce that the return values from all of them are consistent with each other. It is possible to only allow a fixed format (such as uncompressed ARGB) and only allow the video source and/or sink to provide resolutions and frame rates, but this means losing a lot of performance and abandoning any attempt at zero-copy. The use-case for untrusted virtio-media devices is not confidential computing, but rather disaggregated systems where video sources (like webcams) may be attached to untrusted virtual machines. 2. v4l2 is a chatty protocol and using it implies a lot of guest <=> host round-trips. This is bad for performance. Is this overhead actually significant, and if so, are there plans to reduce it? -- Sincerely, Demi Marie Obenour (she/her/hers)
Hi Demi, Le mercredi 09 juillet 2025 à 13:26 -0400, Demi Marie Obenour a écrit :
On 7/9/25 05:34, Albert Esteve wrote:
[...]
It's probably too late, but I have some serious concerns about this device. Specifically:
1. I don't see a reasonable way to support untrusted virtio-media devices. v4l2 has too many ioctls and I can't realistically see a way to enforce that the return values from all of them are consistent with each other. It is possible to only allow a fixed format (such as uncompressed ARGB) and only allow the video source and/or sink to provide resolutions and frame rates, but this means losing a lot of performance and abandoning any attempt at zero-copy.
The use-case for untrusted virtio-media devices is not confidential computing, but rather disaggregated systems where video sources (like webcams) may be attached to untrusted virtual machines.
Please consider that you are posting broadly and understanding of what you mean by untrusted virtio-media can be difficult for non-virtio devs like me. With the enclosed information, I believe implementer are not forced to map to a host V4L2 driver. You can implement a userspace layer in between, and possibly even implement reclaiming by simulation some hot-unplug of the devices, or by not scheduling any work for M2M. I don't really see the relation between "trust" and the pixel format selection. Nor why preventing zero-copy would help handling untrusted virtual machines. I do believe you should not attempt this if you don't have the required knowledge though, as its unlikely the end-result will be safe. For me the main issue for untrusted VM is the lack of control measure, such as cgroup for resource usage. This is being worked on though, Maxime Ripard have given an update at the Linux Media Summit in Nice.
2. v4l2 is a chatty protocol and using it implies a lot of guest <=> host round-trips. This is bad for performance. Is this overhead actually significant, and if so, are there plans to reduce it?
You should try and demonstrate this. Virtio Video tried to make their own and endup with something kind of similar with about the same level of back and forth. We already had few concerns with ioctl() overhead, and some measure have been implemented, notably s_ctrl_ext. Nicolas
On 7/9/25 13:58, Nicolas Dufresne wrote:
Hi Demi,
Le mercredi 09 juillet 2025 à 13:26 -0400, Demi Marie Obenour a écrit :
On 7/9/25 05:34, Albert Esteve wrote:
[...]
It's probably too late, but I have some serious concerns about this device. Specifically:
1. I don't see a reasonable way to support untrusted virtio-media devices. v4l2 has too many ioctls and I can't realistically see a way to enforce that the return values from all of them are consistent with each other. It is possible to only allow a fixed format (such as uncompressed ARGB) and only allow the video source and/or sink to provide resolutions and frame rates, but this means losing a lot of performance and abandoning any attempt at zero-copy.
The use-case for untrusted virtio-media devices is not confidential computing, but rather disaggregated systems where video sources (like webcams) may be attached to untrusted virtual machines.
Please consider that you are posting broadly and understanding of what you mean by untrusted virtio-media can be difficult for non-virtio devs like me.
My use-case is this: 1. There is a VM that has all USB devices. 2. Some of those USB devices, like untrusted webcams, may be passed through to another VM via usbip. 3. The VM with the usbip device provides a video device to yet another VM. The VM with the webcam and the VM receiving the video stream do not trust each other.
With the enclosed information, I believe implementer are not forced to map to a host V4L2 driver. You can implement a userspace layer in between, and possibly even implement reclaiming by simulation some hot-unplug of the devices, or by not scheduling any work for M2M.
I expect that any implementation that cares about security is going to do this.
I don't really see the relation between "trust" and the pixel format selection. Nor why preventing zero-copy would help handling untrusted virtual machines. I do believe you should not attempt this if you don't have the required knowledge though, as its unlikely the end-result will be safe.
How can I validate the various parameters provided by a backend?
For me the main issue for untrusted VM is the lack of control measure, such as cgroup for resource usage. This is being worked on though, Maxime Ripard have given an update at the Linux Media Summit in Nice.
Denial of service is not a concern in my use-cases.
2. v4l2 is a chatty protocol and using it implies a lot of guest <=> host round-trips. This is bad for performance. Is this overhead actually significant, and if so, are there plans to reduce it?
You should try and demonstrate this. Virtio Video tried to make their own and endup with something kind of similar with about the same level of back and forth. We already had few concerns with ioctl() overhead, and some measure have been implemented, notably s_ctrl_ext. I'll leave this for later. -- Sincerely, Demi Marie Obenour (she/her/hers)
participants (2)
-
Demi Marie Obenour -
Nicolas Dufresne