[PATCH] host/rootfs: Sandbox Cloud Hypervisor
It only needs access to a small number of resources. Unfortunately, it needs access to /dev/vfio right now. This should be fixed by using file descriptor passing instead. Also, Cloud Hypervisor should not run as root. Cloud Hypervisor needs to be able to lock memory. Running in a user namespace prevents it from using CAP_IPC_LOCK. Therefore, it is necessary to increase RLIMIT_MLOCK before running Cloud Hypervisor. Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/image/usr/bin/run-vmm | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index ba8b59c2677408acdd01c2eda3cf2dd60992d881..24c3d607bfcf6fea6196b61d2941141486d33fd6 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -52,5 +52,36 @@ unexport ! fdmove -c 3 0 redirfd -r 0 /dev/null +s6-softlimit -H -l 18446744073709551615 if { udevadm wait /dev/kvm } -cloud-hypervisor --api-socket fd=3 +bwrap + --unshare-all + --unshare-user + --dev /dev + --dev-bind /dev/kvm /dev/kvm + --dev-bind /dev/vfio /dev/vfio + --tmpfs /dev/shm + --tmpfs /tmp + --tmpfs /var/tmp + --ro-bind /etc /etc + --ro-bind /lib /lib + --ro-bind /nix /nix + --ro-bind /usr /usr + --ro-bind /sys /sys + --bind /run /run + --proc /proc + --ro-bind /proc/sys /proc/sys + --tmpfs /proc/scsi + --remount-ro /proc/scsi + --tmpfs /proc/acpi + --remount-ro /proc/acpi + --tmpfs /proc/fs + --remount-ro /proc/fs + --tmpfs /proc/irq + --remount-ro /proc/irq + --ro-bind /dev/null /proc/timer_list + --ro-bind /dev/null /proc/kcore + --ro-bind /dev/null /proc/kallsyms + --ro-bind /dev/null /proc/sysrq-trigger + -- + cloud-hypervisor --api-socket fd=3 --- base-commit: 92e219e7c08c479d216a46d2736ea9d229ff034d change-id: 20251206-b4-sandbox-9be7e5ed9926 -- Sincerely, Demi Marie Obenour (she/her/hers)
Demi Marie Obenour <demiobenour@gmail.com> writes:
It only needs access to a small number of resources. Unfortunately, it needs access to /dev/vfio right now. This should be fixed by using file descriptor passing instead. Also, Cloud Hypervisor should not run as root.
Cloud Hypervisor needs to be able to lock memory. Running in a user namespace prevents it from using CAP_IPC_LOCK. Therefore, it is necessary to increase RLIMIT_MLOCK before running Cloud Hypervisor.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/image/usr/bin/run-vmm | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index ba8b59c2677408acdd01c2eda3cf2dd60992d881..24c3d607bfcf6fea6196b61d2941141486d33fd6 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -52,5 +52,36 @@ unexport ! fdmove -c 3 0 redirfd -r 0 /dev/null
+s6-softlimit -H -l 18446744073709551615
My question about the limit from last time is still waiting for an answer…
On 12/6/25 12:29, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
It only needs access to a small number of resources. Unfortunately, it needs access to /dev/vfio right now. This should be fixed by using file descriptor passing instead. Also, Cloud Hypervisor should not run as root.
Cloud Hypervisor needs to be able to lock memory. Running in a user namespace prevents it from using CAP_IPC_LOCK. Therefore, it is necessary to increase RLIMIT_MLOCK before running Cloud Hypervisor.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/image/usr/bin/run-vmm | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-)
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index ba8b59c2677408acdd01c2eda3cf2dd60992d881..24c3d607bfcf6fea6196b61d2941141486d33fd6 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -52,5 +52,36 @@ unexport ! fdmove -c 3 0 redirfd -r 0 /dev/null
+s6-softlimit -H -l 18446744073709551615
My question about the limit from last time is still waiting for an answer…
Whoops, I saw that all the other patches had been applied and missed that this one hadn't been. -- Sincerely, Demi Marie Obenour (she/her/hers)
Demi Marie Obenour <demiobenour@gmail.com> writes:
It only needs access to a small number of resources. Unfortunately, it needs access to /dev/vfio right now. This should be fixed by using file descriptor passing instead. Also, Cloud Hypervisor should not run as root.
Cloud Hypervisor needs to be able to lock memory. Running in a user namespace prevents it from using CAP_IPC_LOCK. Therefore, it is necessary to increase RLIMIT_MLOCK before running Cloud Hypervisor.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/image/usr/bin/run-vmm | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-)
Looks good, but it seems to only work for VMs run as s6 services, not those run through run-appimage or run-flatpak. (I suppose the appimage integration test is somehow not thorough enough to catch this, which should be easier to fix once we understand the problem.) bwrap: execvp cloud-hypervisor: No such file or directory I suppose this is because etc/login sets PATH to /bin, and we don't share /bin with the sandbox. Changing that to /usr/bin would be a good idea I suppose, but would it also be a good idea to share /bin with the sandbox? What do you think?
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index ba8b59c2677408acdd01c2eda3cf2dd60992d881..24c3d607bfcf6fea6196b61d2941141486d33fd6 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -52,5 +52,36 @@ unexport ! fdmove -c 3 0 redirfd -r 0 /dev/null
+s6-softlimit -H -l 18446744073709551615 if { udevadm wait /dev/kvm } -cloud-hypervisor --api-socket fd=3 +bwrap + --unshare-all + --unshare-user + --dev /dev + --dev-bind /dev/kvm /dev/kvm + --dev-bind /dev/vfio /dev/vfio + --tmpfs /dev/shm + --tmpfs /tmp + --tmpfs /var/tmp + --ro-bind /etc /etc + --ro-bind /lib /lib + --ro-bind /nix /nix + --ro-bind /usr /usr + --ro-bind /sys /sys + --bind /run /run + --proc /proc + --ro-bind /proc/sys /proc/sys + --tmpfs /proc/scsi + --remount-ro /proc/scsi + --tmpfs /proc/acpi + --remount-ro /proc/acpi + --tmpfs /proc/fs + --remount-ro /proc/fs + --tmpfs /proc/irq + --remount-ro /proc/irq + --ro-bind /dev/null /proc/timer_list + --ro-bind /dev/null /proc/kcore + --ro-bind /dev/null /proc/kallsyms + --ro-bind /dev/null /proc/sysrq-trigger + -- + cloud-hypervisor --api-socket fd=3
--- base-commit: 92e219e7c08c479d216a46d2736ea9d229ff034d change-id: 20251206-b4-sandbox-9be7e5ed9926
-- Sincerely, Demi Marie Obenour (she/her/hers)
On 12/9/25 04:35, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
It only needs access to a small number of resources. Unfortunately, it needs access to /dev/vfio right now. This should be fixed by using file descriptor passing instead. Also, Cloud Hypervisor should not run as root.
Cloud Hypervisor needs to be able to lock memory. Running in a user namespace prevents it from using CAP_IPC_LOCK. Therefore, it is necessary to increase RLIMIT_MLOCK before running Cloud Hypervisor.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/image/usr/bin/run-vmm | 33 ++++++++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-)
Looks good, but it seems to only work for VMs run as s6 services, not those run through run-appimage or run-flatpak. (I suppose the appimage integration test is somehow not thorough enough to catch this, which should be easier to fix once we understand the problem.)
bwrap: execvp cloud-hypervisor: No such file or directory
I suppose this is because etc/login sets PATH to /bin, and we don't share /bin with the sandbox. Changing that to /usr/bin would be a good idea I suppose, but would it also be a good idea to share /bin with the sandbox? What do you think?
+1 on sharing /bin and /sbin.
diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index ba8b59c2677408acdd01c2eda3cf2dd60992d881..24c3d607bfcf6fea6196b61d2941141486d33fd6 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -52,5 +52,36 @@ unexport ! fdmove -c 3 0 redirfd -r 0 /dev/null
+s6-softlimit -H -l 18446744073709551615 if { udevadm wait /dev/kvm } -cloud-hypervisor --api-socket fd=3 +bwrap + --unshare-all + --unshare-user + --dev /dev + --dev-bind /dev/kvm /dev/kvm + --dev-bind /dev/vfio /dev/vfio + --tmpfs /dev/shm + --tmpfs /tmp + --tmpfs /var/tmp + --ro-bind /etc /etc + --ro-bind /lib /lib + --ro-bind /nix /nix + --ro-bind /usr /usr + --ro-bind /sys /sys + --bind /run /run + --proc /proc + --ro-bind /proc/sys /proc/sys + --tmpfs /proc/scsi + --remount-ro /proc/scsi + --tmpfs /proc/acpi + --remount-ro /proc/acpi + --tmpfs /proc/fs + --remount-ro /proc/fs + --tmpfs /proc/irq + --remount-ro /proc/irq + --ro-bind /dev/null /proc/timer_list + --ro-bind /dev/null /proc/kcore + --ro-bind /dev/null /proc/kallsyms + --ro-bind /dev/null /proc/sysrq-trigger + -- + cloud-hypervisor --api-socket fd=3
--- base-commit: 92e219e7c08c479d216a46d2736ea9d229ff034d change-id: 20251206-b4-sandbox-9be7e5ed9926
-- Sincerely, Demi Marie Obenour (she/her/hers)
-- Sincerely, Demi Marie Obenour (she/her/hers)
This patch has been committed as ec47d362e3053d30668976f00bfd31b539cfd9a7, which can be viewed online at https://spectrum-os.org/git/spectrum/commit/?id=ec47d362e3053d30668976f00bfd.... This is an automated message. Send comments/questions/requests to: Alyssa Ross <hi@alyssa.is>
participants (3)
-
Alyssa Ross -
Alyssa Ross -
Demi Marie Obenour