[PATCH v1 RFC 0/4] spectrum-router
A first version of the spectrum user-space router & its integration into the spectrum system. Many details of the router itself are not yet finalized, and the main focus of this patch series is getting feedback on the integration into the spectrum system. Yureka Lilian (4): vm/sys/net: remove connman & dbus vm/sys/net: integrate xdp-forwarder tools: add spectrum-router host: integrate router host/rootfs/default.nix | 4 +- host/rootfs/file-list.mk | 2 + .../data/service/spectrum-router/down | 0 .../template/data/service/spectrum-router/run | 13 + host/rootfs/image/usr/bin/run-vmm | 12 - host/rootfs/image/usr/bin/vm-import | 13 - pkgs/default.nix | 2 + pkgs/overlay.nix | 1 + tools/router/Cargo.lock | 785 ++++++++++++++++++ tools/router/Cargo.toml | 18 + tools/router/default.nix | 18 + tools/router/src/main.rs | 235 ++++++ tools/start-vmm/ch.rs | 40 +- tools/start-vmm/lib.rs | 68 +- tools/start-vmm/meson.build | 2 +- tools/start-vmm/net-util.c | 39 - tools/start-vmm/net-util.h | 6 - tools/start-vmm/net.c | 55 -- tools/start-vmm/net.rs | 10 - tools/start-vmm/tests/meson.build | 5 - .../start-vmm/tests/tap_open-name-too-long.c | 20 - tools/start-vmm/tests/tap_open.c | 28 - vm/sys/net/Makefile | 2 +- vm/sys/net/default.nix | 15 +- vm/sys/net/file-list.mk | 13 +- vm/sys/net/image/etc/dbus-1/system.conf | 8 - vm/sys/net/image/etc/fstab | 2 + vm/sys/net/image/etc/mdev/iface | 27 +- vm/sys/net/image/etc/nftables.conf | 16 +- vm/sys/net/image/etc/s6-rc/connman/run | 19 - vm/sys/net/image/etc/s6-rc/connman/type | 1 - .../net/image/etc/s6-rc/connman/type.license | 2 - .../net/image/etc/s6-rc/dbus/notification-fd | 1 - .../etc/s6-rc/dbus/notification-fd.license | 2 - vm/sys/net/image/etc/s6-rc/dbus/run | 10 - vm/sys/net/image/etc/s6-rc/dbus/type | 1 - vm/sys/net/image/etc/s6-rc/dbus/type.license | 2 - .../image/etc/s6-rc/ok-all/contents.d/sysctl | 0 vm/sys/net/image/etc/s6-rc/sysctl/type | 1 - .../net/image/etc/s6-rc/sysctl/type.license | 2 - vm/sys/net/image/etc/s6-rc/sysctl/up | 4 - vm/sys/net/image/etc/sysctl.conf | 4 - 42 files changed, 1157 insertions(+), 351 deletions(-) rename vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus => host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down (100%) create mode 100644 host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run create mode 100644 tools/router/Cargo.lock create mode 100644 tools/router/Cargo.toml create mode 100644 tools/router/default.nix create mode 100644 tools/router/src/main.rs delete mode 100644 tools/start-vmm/net-util.c delete mode 100644 tools/start-vmm/net-util.h delete mode 100644 tools/start-vmm/net.c delete mode 100644 tools/start-vmm/tests/tap_open-name-too-long.c delete mode 100644 tools/start-vmm/tests/tap_open.c delete mode 100644 vm/sys/net/image/etc/dbus-1/system.conf delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/run delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/run delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/up delete mode 100644 vm/sys/net/image/etc/sysctl.conf -- 2.51.2
In preparation to integrating xdp-forwarder, making the net-vm a net-driver VM. Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- vm/sys/net/Makefile | 2 +- vm/sys/net/default.nix | 8 +++----- vm/sys/net/file-list.mk | 13 +------------ vm/sys/net/image/etc/dbus-1/system.conf | 8 -------- .../etc/s6-rc/connman/dependencies.d/dbus | 0 vm/sys/net/image/etc/s6-rc/connman/run | 19 ------------------- vm/sys/net/image/etc/s6-rc/connman/type | 1 - .../net/image/etc/s6-rc/connman/type.license | 2 -- .../net/image/etc/s6-rc/dbus/notification-fd | 1 - .../etc/s6-rc/dbus/notification-fd.license | 2 -- vm/sys/net/image/etc/s6-rc/dbus/run | 10 ---------- vm/sys/net/image/etc/s6-rc/dbus/type | 1 - vm/sys/net/image/etc/s6-rc/dbus/type.license | 2 -- .../image/etc/s6-rc/ok-all/contents.d/sysctl | 0 vm/sys/net/image/etc/s6-rc/sysctl/type | 1 - .../net/image/etc/s6-rc/sysctl/type.license | 2 -- vm/sys/net/image/etc/s6-rc/sysctl/up | 4 ---- vm/sys/net/image/etc/sysctl.conf | 4 ---- 18 files changed, 5 insertions(+), 75 deletions(-) delete mode 100644 vm/sys/net/image/etc/dbus-1/system.conf delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/run delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/run delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/up delete mode 100644 vm/sys/net/image/etc/sysctl.conf diff --git a/vm/sys/net/Makefile b/vm/sys/net/Makefile index d71c232..7ad5e5c 100644 --- a/vm/sys/net/Makefile +++ b/vm/sys/net/Makefile @@ -29,7 +29,7 @@ $(vmdir)/netvm/blk/root.img: ../../../scripts/make-gpt.sh ../../../scripts/sfdis build/rootfs.erofs:root:ea21da27-0391-48da-9235-9d2ab2ca7844:root mv $@.tmp $@ -DIRS = dev etc/s6-linux-init/env proc run sys var/lib/connman +DIRS = dev etc/s6-linux-init/env proc run sys BUILD_FILES = build/etc/s6-rc diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index de273e5..c7ae88e 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -7,7 +7,7 @@ pkgsMusl.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeClosure , erofs-utils, jq, s6-rc, util-linux, xorg -, busybox, connmanMinimal, dbus, execline, kmod, linux_latest, mdevd, nftables +, busybox, execline, kmod, linux_latest, mdevd, nftables , s6, s6-linux-init }: @@ -51,10 +51,8 @@ let ]; }); - connman = connmanMinimal; - packages = [ - connman dbus execline kmod mdevd s6 s6-linux-init s6-rc + execline kmod mdevd s6 s6-linux-init s6-rc (busybox.override { extraConfig = '' @@ -73,7 +71,7 @@ let # Packages that should be fully linked into /usr, # (not just their bin/* files). - usrPackages = [ connman dbus firmware kernel.modules terminfo ]; + usrPackages = [ firmware kernel.modules terminfo ]; packagesSysroot = runCommand "packages-sysroot" { inherit packages; diff --git a/vm/sys/net/file-list.mk b/vm/sys/net/file-list.mk index a6f1a41..7bb5dbf 100644 --- a/vm/sys/net/file-list.mk +++ b/vm/sys/net/file-list.mk @@ -2,7 +2,6 @@ # SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com> FILES = \ - image/etc/dbus-1/system.conf \ image/etc/fstab \ image/etc/init \ image/etc/mdev.conf \ @@ -11,7 +10,6 @@ FILES = \ image/etc/passwd \ image/etc/s6-linux-init/run-image/service/getty-hvc0/run \ image/etc/s6-linux-init/scripts/rc.init \ - image/etc/sysctl.conf LINKS = \ image/bin \ @@ -20,12 +18,6 @@ LINKS = \ image/var/run S6_RC_FILES = \ - image/etc/s6-rc/connman/dependencies.d/dbus \ - image/etc/s6-rc/connman/run \ - image/etc/s6-rc/connman/type \ - image/etc/s6-rc/dbus/notification-fd \ - image/etc/s6-rc/dbus/run \ - image/etc/s6-rc/dbus/type \ image/etc/s6-rc/mdevd-coldplug/dependencies.d/mdevd \ image/etc/s6-rc/mdevd-coldplug/type \ image/etc/s6-rc/mdevd-coldplug/up \ @@ -35,7 +27,4 @@ S6_RC_FILES = \ image/etc/s6-rc/nftables/type \ image/etc/s6-rc/nftables/up \ image/etc/s6-rc/ok-all/contents.d/mdevd-coldplug \ - image/etc/s6-rc/ok-all/contents.d/sysctl \ - image/etc/s6-rc/ok-all/type \ - image/etc/s6-rc/sysctl/type \ - image/etc/s6-rc/sysctl/up + image/etc/s6-rc/ok-all/type diff --git a/vm/sys/net/image/etc/dbus-1/system.conf b/vm/sys/net/image/etc/dbus-1/system.conf deleted file mode 100644 index 9ceda7c..0000000 --- a/vm/sys/net/image/etc/dbus-1/system.conf +++ /dev/null @@ -1,8 +0,0 @@ -<?xml version="1.0"?> -<!-- SPDX-License-Identifier: CC0-1.0 --> -<!-- SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is> --> -<!DOCTYPE busconfig PUBLIC "-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN" - "http://www.freedesktop.org/standards/dbus/1.0/busconfig.dtd"> -<busconfig> - <user>root</user> -</busconfig> diff --git a/vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus b/vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus deleted file mode 100644 index e69de29..0000000 diff --git a/vm/sys/net/image/etc/s6-rc/connman/run b/vm/sys/net/image/etc/s6-rc/connman/run deleted file mode 100644 index 058fc17..0000000 --- a/vm/sys/net/image/etc/s6-rc/connman/run +++ /dev/null @@ -1,19 +0,0 @@ -#!/bin/execlineb -P -# SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> - -if { modprobe af_packet } - -backtick -E HARDWARE_INTERFACES { - pipeline { - find -L /sys/class/net -mindepth 2 -maxdepth 2 -name address -print0 - } - - # Filter out other VMs and the loopback device. - pipeline { xargs -0 grep -iL ^\\(02:01:\\|00:00:00:00:00:00$\\) } - - # Extract the interface names from the address file paths. - awk -F/ "{if (NR > 1) printf \",\"; printf \"%s\", $5}" -} - -connmand -ni $HARDWARE_INTERFACES diff --git a/vm/sys/net/image/etc/s6-rc/connman/type b/vm/sys/net/image/etc/s6-rc/connman/type deleted file mode 100644 index 5883cff..0000000 --- a/vm/sys/net/image/etc/s6-rc/connman/type +++ /dev/null @@ -1 +0,0 @@ -longrun diff --git a/vm/sys/net/image/etc/s6-rc/connman/type.license b/vm/sys/net/image/etc/s6-rc/connman/type.license deleted file mode 100644 index 2b3b032..0000000 --- a/vm/sys/net/image/etc/s6-rc/connman/type.license +++ /dev/null @@ -1,2 +0,0 @@ -SPDX-License-Identifier: CC0-1.0 -SPDX-FileCopyrightText: 2020 Alyssa Ross <hi@alyssa.is> diff --git a/vm/sys/net/image/etc/s6-rc/dbus/notification-fd b/vm/sys/net/image/etc/s6-rc/dbus/notification-fd deleted file mode 100644 index 00750ed..0000000 --- a/vm/sys/net/image/etc/s6-rc/dbus/notification-fd +++ /dev/null @@ -1 +0,0 @@ -3 diff --git a/vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license b/vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license deleted file mode 100644 index c49c11b..0000000 --- a/vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license +++ /dev/null @@ -1,2 +0,0 @@ -SPDX-License-Identifier: CC0-1.0 -SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is> diff --git a/vm/sys/net/image/etc/s6-rc/dbus/run b/vm/sys/net/image/etc/s6-rc/dbus/run deleted file mode 100644 index 26dd403..0000000 --- a/vm/sys/net/image/etc/s6-rc/dbus/run +++ /dev/null @@ -1,10 +0,0 @@ -#!/bin/execlineb -P -# SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> - -foreground { mkdir /run/dbus } - -dbus-daemon - --config-file=/usr/share/dbus-1/system.conf - --nofork - --print-address=3 diff --git a/vm/sys/net/image/etc/s6-rc/dbus/type b/vm/sys/net/image/etc/s6-rc/dbus/type deleted file mode 100644 index 5883cff..0000000 --- a/vm/sys/net/image/etc/s6-rc/dbus/type +++ /dev/null @@ -1 +0,0 @@ -longrun diff --git a/vm/sys/net/image/etc/s6-rc/dbus/type.license b/vm/sys/net/image/etc/s6-rc/dbus/type.license deleted file mode 100644 index 2b3b032..0000000 --- a/vm/sys/net/image/etc/s6-rc/dbus/type.license +++ /dev/null @@ -1,2 +0,0 @@ -SPDX-License-Identifier: CC0-1.0 -SPDX-FileCopyrightText: 2020 Alyssa Ross <hi@alyssa.is> diff --git a/vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl b/vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl deleted file mode 100644 index e69de29..0000000 diff --git a/vm/sys/net/image/etc/s6-rc/sysctl/type b/vm/sys/net/image/etc/s6-rc/sysctl/type deleted file mode 100644 index bdd22a1..0000000 --- a/vm/sys/net/image/etc/s6-rc/sysctl/type +++ /dev/null @@ -1 +0,0 @@ -oneshot diff --git a/vm/sys/net/image/etc/s6-rc/sysctl/type.license b/vm/sys/net/image/etc/s6-rc/sysctl/type.license deleted file mode 100644 index c49c11b..0000000 --- a/vm/sys/net/image/etc/s6-rc/sysctl/type.license +++ /dev/null @@ -1,2 +0,0 @@ -SPDX-License-Identifier: CC0-1.0 -SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is> diff --git a/vm/sys/net/image/etc/s6-rc/sysctl/up b/vm/sys/net/image/etc/s6-rc/sysctl/up deleted file mode 100644 index dafa493..0000000 --- a/vm/sys/net/image/etc/s6-rc/sysctl/up +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is> - -sysctl -pq diff --git a/vm/sys/net/image/etc/sysctl.conf b/vm/sys/net/image/etc/sysctl.conf deleted file mode 100644 index 3991453..0000000 --- a/vm/sys/net/image/etc/sysctl.conf +++ /dev/null @@ -1,4 +0,0 @@ -# SPDX-License-Identifier: CC0-1.0 -# SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> - -net.ipv4.ip_forward = 1 -- 2.51.2
Yureka Lilian <yureka@cyberchaos.dev> writes:
In preparation to integrating xdp-forwarder, making the net-vm a net-driver VM.
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- vm/sys/net/Makefile | 2 +- vm/sys/net/default.nix | 8 +++----- vm/sys/net/file-list.mk | 13 +------------ vm/sys/net/image/etc/dbus-1/system.conf | 8 -------- .../etc/s6-rc/connman/dependencies.d/dbus | 0 vm/sys/net/image/etc/s6-rc/connman/run | 19 ------------------- vm/sys/net/image/etc/s6-rc/connman/type | 1 - .../net/image/etc/s6-rc/connman/type.license | 2 -- .../net/image/etc/s6-rc/dbus/notification-fd | 1 - .../etc/s6-rc/dbus/notification-fd.license | 2 -- vm/sys/net/image/etc/s6-rc/dbus/run | 10 ---------- vm/sys/net/image/etc/s6-rc/dbus/type | 1 - vm/sys/net/image/etc/s6-rc/dbus/type.license | 2 -- .../image/etc/s6-rc/ok-all/contents.d/sysctl | 0 vm/sys/net/image/etc/s6-rc/sysctl/type | 1 - .../net/image/etc/s6-rc/sysctl/type.license | 2 -- vm/sys/net/image/etc/s6-rc/sysctl/up | 4 ---- vm/sys/net/image/etc/sysctl.conf | 4 ---- 18 files changed, 5 insertions(+), 75 deletions(-) delete mode 100644 vm/sys/net/image/etc/dbus-1/system.conf delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/run delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/run delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/up delete mode 100644 vm/sys/net/image/etc/sysctl.conf
Won't we still need connman or NetworkManager or something to configure Wi-Fi?
On 11/25/25 11:15, Alyssa Ross wrote:
Yureka Lilian <yureka@cyberchaos.dev> writes:
In preparation to integrating xdp-forwarder, making the net-vm a net-driver VM.
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- vm/sys/net/Makefile | 2 +- vm/sys/net/default.nix | 8 +++----- vm/sys/net/file-list.mk | 13 +------------ vm/sys/net/image/etc/dbus-1/system.conf | 8 -------- .../etc/s6-rc/connman/dependencies.d/dbus | 0 vm/sys/net/image/etc/s6-rc/connman/run | 19 ------------------- vm/sys/net/image/etc/s6-rc/connman/type | 1 - .../net/image/etc/s6-rc/connman/type.license | 2 -- .../net/image/etc/s6-rc/dbus/notification-fd | 1 - .../etc/s6-rc/dbus/notification-fd.license | 2 -- vm/sys/net/image/etc/s6-rc/dbus/run | 10 ---------- vm/sys/net/image/etc/s6-rc/dbus/type | 1 - vm/sys/net/image/etc/s6-rc/dbus/type.license | 2 -- .../image/etc/s6-rc/ok-all/contents.d/sysctl | 0 vm/sys/net/image/etc/s6-rc/sysctl/type | 1 - .../net/image/etc/s6-rc/sysctl/type.license | 2 -- vm/sys/net/image/etc/s6-rc/sysctl/up | 4 ---- vm/sys/net/image/etc/sysctl.conf | 4 ---- 18 files changed, 5 insertions(+), 75 deletions(-) delete mode 100644 vm/sys/net/image/etc/dbus-1/system.conf delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/run delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/run delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/up delete mode 100644 vm/sys/net/image/etc/sysctl.conf Won't we still need connman or NetworkManager or something to configure Wi-Fi?
Thank you for this excellent question. connman and NetworkManager are all-in-one tools for networking. They mostly provide a unified interface for the underlying stacks (wpa_supplicant/iwd, kernel networking stack, firewall, VPNs). I don't expect we can re-use such an all-in-one tool in the driver VM, as the driver VM is only responsible for one device and shuffling data from and to it. In the Wi-Fi case there is an exception because the device needs special configuration, with user input. In the future, I see wpa_supplicant as a candidate which can maintain Wi-Fi connections in a stateful configuration file, providing both a cli and a GUI which we could forward to the user. For more complex networking configuration, I would expect them to be done via the VM "graph", where an advanced user could build a chain of provider VMs which do 1:n multiplexing (the router I'm currently writing), n:1 multiplexing (something like Android's automatic switching between Wi-Fi, cellular uplink and cabled connections depending on network conditions), and VPN/Tor as a 1:1 provider performing some encapsulation. In this current patch series the spectrum router takes over the 1:n multiplexing that connman was previously used for. For the n:1 multiplexing needed for network devices with multiple interfaces, I expect to just use the next best one which receives router advertisements (which should be similar to the previous user experience with connman), but in this current patch series the n:1 multiplexing is not implemented. From what I can tell Wi-Fi was not a previously working feature, and supporting it in the future via wpa_supplicant would not be much more difficult, as we have tested the router in principle supports multiplexing multiple apps to one Wi-Fi connection.
On 11/25/25 06:37, Yureka wrote:
On 11/25/25 11:15, Alyssa Ross wrote:
Yureka Lilian <yureka@cyberchaos.dev> writes:
In preparation to integrating xdp-forwarder, making the net-vm a net-driver VM.
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- vm/sys/net/Makefile | 2 +- vm/sys/net/default.nix | 8 +++----- vm/sys/net/file-list.mk | 13 +------------ vm/sys/net/image/etc/dbus-1/system.conf | 8 -------- .../etc/s6-rc/connman/dependencies.d/dbus | 0 vm/sys/net/image/etc/s6-rc/connman/run | 19 ------------------- vm/sys/net/image/etc/s6-rc/connman/type | 1 - .../net/image/etc/s6-rc/connman/type.license | 2 -- .../net/image/etc/s6-rc/dbus/notification-fd | 1 - .../etc/s6-rc/dbus/notification-fd.license | 2 -- vm/sys/net/image/etc/s6-rc/dbus/run | 10 ---------- vm/sys/net/image/etc/s6-rc/dbus/type | 1 - vm/sys/net/image/etc/s6-rc/dbus/type.license | 2 -- .../image/etc/s6-rc/ok-all/contents.d/sysctl | 0 vm/sys/net/image/etc/s6-rc/sysctl/type | 1 - .../net/image/etc/s6-rc/sysctl/type.license | 2 -- vm/sys/net/image/etc/s6-rc/sysctl/up | 4 ---- vm/sys/net/image/etc/sysctl.conf | 4 ---- 18 files changed, 5 insertions(+), 75 deletions(-) delete mode 100644 vm/sys/net/image/etc/dbus-1/system.conf delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/dependencies.d/dbus delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/run delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type delete mode 100644 vm/sys/net/image/etc/s6-rc/connman/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/notification-fd.license delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/run delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type delete mode 100644 vm/sys/net/image/etc/s6-rc/dbus/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/ok-all/contents.d/sysctl delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/type.license delete mode 100644 vm/sys/net/image/etc/s6-rc/sysctl/up delete mode 100644 vm/sys/net/image/etc/sysctl.conf Won't we still need connman or NetworkManager or something to configure Wi-Fi?
Thank you for this excellent question.
Me too! I have some comments below, but I will be the first to state that none of them should block merging this patch. The current situation is definitely unsatisfactory.
connman and NetworkManager are all-in-one tools for networking. They mostly provide a unified interface for the underlying stacks (wpa_supplicant/iwd, kernel networking stack, firewall, VPNs). I don't expect we can re-use such an all-in-one tool in the driver VM, as the driver VM is only responsible for one device and shuffling data from and to it. In the Wi-Fi case there is an exception because the device needs special configuration, with user input. In the future, I see wpa_supplicant as a candidate which can maintain Wi-Fi connections in a stateful configuration file, providing both a cli and a GUI which we could forward to the user.
NetworkManager has the advantage that good GUI and CLI tools for it already exist. That said, if we are going to use a daemon directly, I strongly recommend going with iwd over wpa_supplicant. It has much better code quality and can handle network configuration itself.
For more complex networking configuration, I would expect them to be done via the VM "graph", where an advanced user could build a chain of provider VMs which do 1:n multiplexing (the router I'm currently writing), n:1 multiplexing (something like Android's automatic switching between Wi-Fi, cellular uplink and cabled connections depending on network conditions), and VPN/Tor as a 1:1 provider performing some encapsulation.
I don't think that n:1 multiplexing is an advanced configuration. My home laptop and probably many others has Wi-Fi and Ethernet, and others have Wi-Fi and USB/Thunderbolt docking stations. Yet other devices will have cellular data too. It's a mess, and it should work out of the box.
In this current patch series the spectrum router takes over the 1:n multiplexing that connman was previously used for. For the n:1 multiplexing needed for network devices with multiple interfaces, I expect to just use the next best one which receives router advertisements (which should be similar to the previous user experience with connman), but in this current patch series the n:1 multiplexing is not implemented.
NetworkManager might have more complex policies. If so, it might be best to reuse it. I don't think this is an area where it is worth diverging too much from most Linux distros.
From what I can tell Wi-Fi was not a previously working feature, and supporting it in the future via wpa_supplicant would not be much more difficult, as we have tested the router in principle supports multiplexing multiple apps to one Wi-Fi connection. Getting Wi-Fi working should be quite simple, but I expect that choosing between different networks won't be, at least for all of the types of networks in the wild. I would love to be prove wrong on this. -- Sincerely, Demi Marie Obenour (she/her/hers)
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- vm/sys/net/default.nix | 13 +++++++++---- vm/sys/net/image/etc/fstab | 2 ++ vm/sys/net/image/etc/mdev/iface | 27 ++++++++------------------- vm/sys/net/image/etc/nftables.conf | 16 ++++++++++++---- 4 files changed, 31 insertions(+), 27 deletions(-) diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index c7ae88e..fd5bf08 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -2,12 +2,12 @@ # SPDX-FileCopyrightText: 2021-2023 Alyssa Ross <hi@alyssa.is> import ../../../lib/call-package.nix ( -{ spectrum-build-tools, src, terminfo, pkgsMusl }: +{ spectrum-build-tools, spectrum-driver-tools, src, terminfo, pkgsMusl }: pkgsMusl.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeClosure , erofs-utils, jq, s6-rc, util-linux, xorg -, busybox, execline, kmod, linux_latest, mdevd, nftables +, busybox, execline, kmod, linux_latest, mdevd, nftables, xdp-tools , s6, s6-linux-init }: @@ -52,7 +52,7 @@ let }); packages = [ - execline kmod mdevd s6 s6-linux-init s6-rc + execline kmod mdevd s6 s6-linux-init s6-rc xdp-tools (busybox.override { extraConfig = '' @@ -71,7 +71,12 @@ let # Packages that should be fully linked into /usr, # (not just their bin/* files). - usrPackages = [ firmware kernel.modules terminfo ]; + usrPackages = [ + firmware kernel.modules terminfo + + # for xdp-forwarder + spectrum-driver-tools + ]; packagesSysroot = runCommand "packages-sysroot" { inherit packages; diff --git a/vm/sys/net/image/etc/fstab b/vm/sys/net/image/etc/fstab index 6a82ecc..5a1bbf4 100644 --- a/vm/sys/net/image/etc/fstab +++ b/vm/sys/net/image/etc/fstab @@ -1,6 +1,8 @@ # SPDX-License-Identifier: CC0-1.0 # SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> proc /proc proc defaults 0 0 devpts /dev/pts devpts defaults,gid=4,mode=620 0 0 tmpfs /dev/shm tmpfs defaults 0 0 sysfs /sys sysfs defaults 0 0 +bpffs /sys/fs/bpf bpf defaults 0 0 diff --git a/vm/sys/net/image/etc/mdev/iface b/vm/sys/net/image/etc/mdev/iface index 2306575..ff4bf53 100755 --- a/vm/sys/net/image/etc/mdev/iface +++ b/vm/sys/net/image/etc/mdev/iface @@ -1,36 +1,25 @@ #!/bin/execlineb -P # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> importas -Si INTERFACE ifte { - # This interface is connected to another VM. - - # The other VM's IP is encoded in the NIC-specific portion of the - # interface's MAC address. - backtick -E CLIENT_IP { - awk -F: "{printf \"100.64.%d.%d\\n\", \"0x\" $5, \"0x\" $6}" - /sys/class/net/${INTERFACE}/address - } - - if { ip address add 169.254.0.1/32 dev $INTERFACE } - if { ip link set $INTERFACE up } - ip route add $CLIENT_IP dev $INTERFACE + # This interface is connected to the router + if { xdp-loader load $INTERFACE /usr/lib/xdp/prog_router.o -m skb -p /sys/fs/bpf } + if { ip link set $INTERFACE promisc on } + if { set-router-iface $INTERFACE } + ip link set $INTERFACE up } { if { test $INTERFACE != lo } # This is a physical connection to a network device. - background { s6-rc -bu change connman } - if { s6-rc -bu change nftables } - if { - forx -pE module { nft_counter nft_masq } - modprobe $module - } - nft add rule ip nat postrouting oifname $INTERFACE counter masquerade + if { xdp-loader load $INTERFACE /usr/lib/xdp/prog_physical.o -m skb -p /sys/fs/bpf } + ip link set $INTERFACE up } grep -iq ^02:01: /sys/class/net/${INTERFACE}/address diff --git a/vm/sys/net/image/etc/nftables.conf b/vm/sys/net/image/etc/nftables.conf index 296d92c..cc8e462 100644 --- a/vm/sys/net/image/etc/nftables.conf +++ b/vm/sys/net/image/etc/nftables.conf @@ -1,8 +1,16 @@ # SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> -table nat { - chain postrouting { - type nat hook postrouting priority 100; +table driver-fw { + chain input { + type filter hook input priority filter; policy drop; + } + + chain output { + type filter hook output priority filter; policy drop; + } + + chain forward { + type filter hook forward priority filter; policy drop; } } -- 2.51.2
Yureka Lilian <yureka@cyberchaos.dev> writes:
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- vm/sys/net/default.nix | 13 +++++++++---- vm/sys/net/image/etc/fstab | 2 ++ vm/sys/net/image/etc/mdev/iface | 27 ++++++++------------------- vm/sys/net/image/etc/nftables.conf | 16 ++++++++++++---- 4 files changed, 31 insertions(+), 27 deletions(-)
Looks good. Just a couple of small questions.
diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index c7ae88e..fd5bf08 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -2,12 +2,12 @@ # SPDX-FileCopyrightText: 2021-2023 Alyssa Ross <hi@alyssa.is>
import ../../../lib/call-package.nix ( -{ spectrum-build-tools, src, terminfo, pkgsMusl }: +{ spectrum-build-tools, spectrum-driver-tools, src, terminfo, pkgsMusl }:
We're taking this from the default package set, where it's built with Glibc — presumably it should be built with musl like everything else in the VM?
diff --git a/vm/sys/net/image/etc/nftables.conf b/vm/sys/net/image/etc/nftables.conf index 296d92c..cc8e462 100644 --- a/vm/sys/net/image/etc/nftables.conf +++ b/vm/sys/net/image/etc/nftables.conf @@ -1,8 +1,16 @@ # SPDX-License-Identifier: EUPL-1.2+ -# SPDX-FileCopyrightText: 2021 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev>
-table nat { - chain postrouting { - type nat hook postrouting priority 100; +table driver-fw { + chain input { + type filter hook input priority filter; policy drop; + } + + chain output { + type filter hook output priority filter; policy drop; + } + + chain forward { + type filter hook forward priority filter; policy drop; } }
As someone with basically no netfilter experience, I'm surprised to not see a newline after a semicolon. Is that idiomatic for netfilter?
The tokio-vhost & vhost-device-net crates which we also wrote and depend on are left external in the outlook of becoming a rust-vmm project soon. Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- pkgs/default.nix | 2 + tools/router/Cargo.lock | 785 +++++++++++++++++++++++++++++++++++++++ tools/router/Cargo.toml | 18 + tools/router/default.nix | 18 + tools/router/src/main.rs | 235 ++++++++++++ 5 files changed, 1058 insertions(+) create mode 100644 tools/router/Cargo.lock create mode 100644 tools/router/Cargo.toml create mode 100644 tools/router/default.nix create mode 100644 tools/router/src/main.rs diff --git a/pkgs/default.nix b/pkgs/default.nix index cc60228..7c1c9c3 100644 --- a/pkgs/default.nix +++ b/pkgs/default.nix @@ -52,6 +52,8 @@ let xdg-desktop-portal-spectrum-host = self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {}; + spectrum-router = self.callSpectrumPackage ../tools/router {}; + # Packages from the overlay, so it's possible to build them from # the CLI easily. inherit (pkgs) cloud-hypervisor dbus; diff --git a/tools/router/Cargo.lock b/tools/router/Cargo.lock new file mode 100644 index 0000000..04179af --- /dev/null +++ b/tools/router/Cargo.lock @@ -0,0 +1,785 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "anstream" +version = "0.6.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43d5b281e737544384e969a5ccad3f1cdd24b48086a0fc1b2a5262a26b8f4f4a" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5192cca8006f1fd4f7237516f40fa183bb07f8fbdfedaa0036de5ea9b0b45e78" + +[[package]] +name = "anstyle-parse" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7644824f0aa2c7b9384579234ef10eb7efb6a0deb83f9630a49594dd9c15c2" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + +[[package]] +name = "anyhow" +version = "1.0.100" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a23eb6b1614318a8071c9b2521f36b424b2c83db5eb3a0fead4a6c0809af6e61" + +[[package]] +name = "async-stream" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b5a71a6f37880a80d1d7f19efd781e4b5de42c88f0722cc13bcb6cc2cfe8476" +dependencies = [ + "async-stream-impl", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-stream-impl" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + +[[package]] +name = "bitvec" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bc2832c24239b0141d5674bb9174f9d68a8b5b3f2753311927c172ca46f7e9c" +dependencies = [ + "funty", + "radium", + "tap", + "wyz", +] + +[[package]] +name = "bytes" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b35204fbdc0b3f4446b89fc1ac2cf84a8a68971995d0bf2e925ec7cd960f9cb3" + +[[package]] +name = "clap" +version = "4.5.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9e340e012a1bf4935f5282ed1436d1489548e8f72308207ea5df0e23d2d03f8" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.5.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d76b5d13eaa18c901fd2f7fca939fefe3a0727a953561fefdf3b2922b8569d00" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.5.49" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a0b5487afeab2deb2ff4e03a807ad1a03ac532ff5a2cee5d86884440c7f7671" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "clap_lex" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1d728cc89cf3aee9ff92b05e62b19ee65a02b5702cff7d5a377e32c6ae29d8d" + +[[package]] +name = "colorchoice" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b05b61dc5112cbb17e4b6cd61790d9845d13888356391624cbe7e41efeac1e75" + +[[package]] +name = "env_filter" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bf3c259d255ca70051b30e2e95b5446cdb8949ac4cd22c0d7fd634d89f568e2" +dependencies = [ + "log", + "regex", +] + +[[package]] +name = "env_logger" +version = "0.11.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c863f0904021b108aa8b2f55046443e6b1ebde8fd4a15c399893aae4fa069f" +dependencies = [ + "anstream", + "anstyle", + "env_filter", + "jiff", + "log", +] + +[[package]] +name = "fastrand" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be" + +[[package]] +name = "funty" +version = "2.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6d5a32815ae3f33302d95fdcb2ce17862f8c65363dcfd29360480ba1001fc9c" + +[[package]] +name = "futures-core" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e" + +[[package]] +name = "futures-io" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6" + +[[package]] +name = "futures-lite" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +dependencies = [ + "fastrand", + "futures-core", + "futures-io", + "parking", + "pin-project-lite", +] + +[[package]] +name = "futures-macro" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "futures-sink" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7" + +[[package]] +name = "futures-task" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988" + +[[package]] +name = "futures-util" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81" +dependencies = [ + "futures-core", + "futures-macro", + "futures-sink", + "futures-task", + "pin-project-lite", + "pin-utils", + "slab", +] + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "jiff" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49cce2b81f2098e7e3efc35bc2e0a6b7abec9d34128283d7a26fa8f32a6dbb35" +dependencies = [ + "jiff-static", + "log", + "portable-atomic", + "portable-atomic-util", + "serde_core", +] + +[[package]] +name = "jiff-static" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "980af8b43c3ad5d8d349ace167ec8170839f753a42d233ba19e08afe1850fa69" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "libc" +version = "0.2.177" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2874a2af47a2325c2001a6e6fad9b16a53b802102b528163885171cf92b15976" + +[[package]] +name = "log" +version = "0.4.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34080505efa8e45a4b816c349525ebe327ceaa8559756f0356cba97ef3bf7432" + +[[package]] +name = "memchr" +version = "2.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f52b00d39961fc5b2736ea853c9cc86238e165017a493d1d5c8eac6bdc4cc273" + +[[package]] +name = "mio" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69d83b0086dc8ecf3ce9ae2874b2d1290252e2a30720bea58a5c6639b0092873" +dependencies = [ + "libc", + "wasi", + "windows-sys 0.61.2", +] + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "parking" +version = "2.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" + +[[package]] +name = "pin-project-lite" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b" + +[[package]] +name = "pin-utils" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" + +[[package]] +name = "portable-atomic" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483" + +[[package]] +name = "portable-atomic-util" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8a2f0d8d040d7848a709caf78912debcc3f33ee4b3cac47d73d1e1069e83507" +dependencies = [ + "portable-atomic", +] + +[[package]] +name = "proc-macro2" +version = "1.0.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ee95bc4ef87b8d5ba32e8b7714ccc834865276eab0aed5c9958d00ec45f49e8" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.42" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a338cc41d27e6cc6dce6cefc13a0729dfbb81c262b1f519331575dd80ef3067f" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "radium" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc33ff2d4973d518d823d61aa239014831e521c75da58e3df4840d3f47749d09" + +[[package]] +name = "regex" +version = "1.12.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58" + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "slab" +version = "0.4.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2ae44ef20feb57a68b23d846850f861394c2e02dc425a50098ae8c90267589" + +[[package]] +name = "socket2" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "17129e116933cf371d018bb80ae557e889637989d8638274fb25622827b03881" +dependencies = [ + "libc", + "windows-sys 0.60.2", +] + +[[package]] +name = "spectrum-router" +version = "0.1.0" +dependencies = [ + "anyhow", + "clap", + "env_logger", + "futures-util", + "log", + "tokio", + "tokio-stream", + "vhost-device-net", + "zerocopy", +] + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "syn" +version = "2.0.110" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a99801b5bd34ede4cf3fc688c5919368fea4e4814a4664359503e6015b280aea" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tap" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369" + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tokio" +version = "1.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff360e02eab121e0bc37a2d3b4d4dc622e6eda3a8e5253d5435ecf5bd4c68408" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-eventfd" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e66bd133670ac39baa1aca5c3a86709f4595c08ca4464a1e1400b83d62c0639" +dependencies = [ + "futures-lite", + "libc", + "tokio", +] + +[[package]] +name = "tokio-macros" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af407857209536a95c8e56f8231ef2c2e2aff839b22e07a1ffcbc617e9db9fa5" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tokio-stream" +version = "0.1.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eca58d7bba4a75707817a2c44174253f9236b2d5fbd055602e9d5c07c139a047" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tokio-vhost" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "21d64b3e4d573da90b2bb040d69a9c2d754e8a3ab9d9ecf04a268748c99f1cd3" +dependencies = [ + "async-stream", + "bitvec", + "futures-util", + "libc", + "log", + "tokio", + "tokio-eventfd", + "virtio-queue", + "vm-memory", + "zerocopy", +] + +[[package]] +name = "unicode-ident" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9312f7c4f6ff9069b165498234ce8be658059c6728633667c526e27dc2cf1df5" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "vhost-device-net" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac05caccd6d484f672551a187f7110ff9d32edd6a39bb16bb04f53017b1e6fd0" +dependencies = [ + "futures-util", + "log", + "tokio", + "tokio-vhost", + "vm-memory", +] + +[[package]] +name = "virtio-bindings" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "804f498a26d5a63be7bbb8bdcd3869c3f286c4c4a17108905276454da0caf8cb" + +[[package]] +name = "virtio-queue" +version = "0.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb0479158f863e59323771a1f684d843962f76960b86fecfec2bfa9c8f0f9180" +dependencies = [ + "log", + "virtio-bindings", + "vm-memory", + "vmm-sys-util", +] + +[[package]] +name = "vm-memory" +version = "0.16.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd5e56d48353c5f54ef50bd158a0452fc82f5383da840f7b8efc31695dd3b9d" +dependencies = [ + "libc", + "thiserror", + "winapi", +] + +[[package]] +name = "vmm-sys-util" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d21f366bf22bfba3e868349978766a965cbe628c323d58e026be80b8357ab789" +dependencies = [ + "bitflags", + "libc", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb" +dependencies = [ + "windows-targets", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-targets" +version = "0.53.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3" +dependencies = [ + "windows-link", + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_gnullvm", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" + +[[package]] +name = "windows_i686_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" + +[[package]] +name = "windows_i686_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" + +[[package]] +name = "wyz" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05f360fc0b24296329c78fda852a1e9ae82de9cf7b27dae4b7f62f118f77b9ed" +dependencies = [ + "tap", +] + +[[package]] +name = "zerocopy" +version = "0.8.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43fa6694ed34d6e57407afbccdeecfa268c470a7d2a5b0cf49ce9fcc345afb90" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c640b22cd9817fae95be82f0d2f90b11f7605f6c319d16705c459b27ac2cbc26" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] diff --git a/tools/router/Cargo.toml b/tools/router/Cargo.toml new file mode 100644 index 0000000..f9d8138 --- /dev/null +++ b/tools/router/Cargo.toml @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> + +[package] +name = "spectrum-router" +version = "0.1.0" +edition = "2024" + +[dependencies] +anyhow = "1.0.100" +clap = { version = "4.5.45", features = ["derive"] } +env_logger = "0.11.8" +log = { version = "0.4.27", features = ["release_max_level_debug"] } +vhost-device-net = "0.1.0" +tokio = { version = "1.48.0", features = ["macros", "rt"] } +futures-util = "0.3.31" +zerocopy = "0.8.27" +tokio-stream = "0.1.17" diff --git a/tools/router/default.nix b/tools/router/default.nix new file mode 100644 index 0000000..e70f9ec --- /dev/null +++ b/tools/router/default.nix @@ -0,0 +1,18 @@ +# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> +# SPDX-License-Identifier: MIT + +import ../../lib/call-package.nix ( +{ src, lib, rustPlatform }: + +rustPlatform.buildRustPackage { + name = "spectrum-router"; + + src = lib.fileset.toSource { + root = ../..; + fileset = lib.fileset.intersection src ./.; + }; + sourceRoot = "source/tools/router"; + + cargoLock.lockFile = ./Cargo.lock; +}) (_: {}) diff --git a/tools/router/src/main.rs b/tools/router/src/main.rs new file mode 100644 index 0000000..42fb8f7 --- /dev/null +++ b/tools/router/src/main.rs @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: EUPL-1.2+ +// SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> + +use std::collections::HashMap; +use std::io; +use std::net::Ipv6Addr; +use std::path::PathBuf; + +use clap::Parser; +use futures_util::SinkExt; +use futures_util::StreamExt; +use log::{debug, error, info, trace, warn}; +use tokio::net::UnixListener; +use tokio_stream::StreamMap; +use vhost_device_net::VhostDeviceNet; + +use zerocopy::byteorder::network_endian::{U16, U32}; +use zerocopy::*; + +type MacAddr = [u8; 6]; + +#[derive(Debug, PartialEq, Eq, FromBytes, IntoBytes, KnownLayout, Immutable, Unaligned)] +#[repr(C)] +struct EtherFrame { + //preamble_sfd: U64, + dst_addr: MacAddr, + src_addr: MacAddr, + ether_type: U16, + //data: [u8] +} + +impl AsRef<[u8]> for EtherFrame { + fn as_ref(&self) -> &[u8] { + self.as_bytes() + } +} + +#[derive(Debug, PartialEq, Eq, FromBytes, IntoBytes, KnownLayout, Immutable, Unaligned)] +#[repr(C)] +struct VlanTag { + tag_control_information: U16, + ether_type: U16, +} + +impl AsRef<[u8]> for VlanTag { + fn as_ref(&self) -> &[u8] { + self.as_bytes() + } +} + +#[derive(Debug, PartialEq, Eq, FromBytes, IntoBytes, KnownLayout, Immutable, Unaligned)] +#[repr(C)] +struct Ipv6Header { + version_traffic_class_flow_label: U32, + payload_length: U16, + hext_header: u8, + hop_limit: u8, + src_addr: [u8; 16], + dst_addr: [u8; 16], + //data: [u8] +} + +impl AsRef<[u8]> for Ipv6Header { + fn as_ref(&self) -> &[u8] { + self.as_bytes() + } +} + +#[derive(Parser, Debug)] +#[command()] //version = None, about = None, long_about = None)] +struct Args { + #[arg(long)] + driver_listen_path: PathBuf, + #[arg(long)] + app_listen_path: PathBuf, + #[arg(long)] + upstream_interface: u16, +} + +#[tokio::main(flavor = "current_thread")] +async fn main() -> anyhow::Result<()> { + env_logger::init(); + let args = Args::parse(); + + run_router(args).await +} + +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +enum InterfaceId { + Driver, + App(usize), +} + +async fn run_router(args: Args) -> anyhow::Result<()> { + let driver_listener = UnixListener::bind(&args.driver_listen_path)?; + let app_listener = UnixListener::bind(&args.app_listen_path)?; + + let fib: HashMap<Ipv6Addr, (MacAddr, InterfaceId)> = Default::default(); + + let mut fib = fib.clone(); + let mut phys_mac = None; + + let mut streams = StreamMap::new(); + let mut sinks = HashMap::<InterfaceId, _>::new(); + + let mut app_num = 0; + + loop { + tokio::select! { + driver_conn = driver_listener.accept() => { + info!("driver connected"); + match driver_conn { + Ok((stream, _addr)) => { + let device = VhostDeviceNet::from_unix_stream(stream).await?; + streams.insert(InterfaceId::Driver, Box::pin(device.tx().await?)); + sinks.insert(InterfaceId::Driver, Box::pin(device.rx::<Box<dyn io::Read + Send + Sync + 'static>>().await?)); + phys_mac = None; + } + Err(e) => error!("driver connection failed: {}", e), + } + } + app_conn = app_listener.accept() => { + info!("app connected"); + match app_conn { + Ok((stream, _addr)) => { + let device = VhostDeviceNet::from_unix_stream(stream).await?; + streams.insert(InterfaceId::App(app_num), Box::pin(device.tx().await?)); + sinks.insert(InterfaceId::App(app_num), Box::pin(device.rx().await?)); + app_num = app_num.checked_add(1).unwrap(); + } + Err(e) => error!("app connection failed: {}", e), + } + } + next_res = streams.next(), if !streams.is_empty() => { + let Some((key, Ok(buf))) = next_res else { + info!("incoming other"); + continue; + }; + match &key { + InterfaceId::Driver => { + use io::Read; + let mut ether_bytes = buf.take(size_of::<EtherFrame>() as u64); + let mut ether_frame = EtherFrame::read_from_io(&mut ether_bytes)?; + let remaining_buf = ether_bytes.into_inner(); + + if ether_frame.ether_type != 0x8100 { + warn!("untagged packet from driver"); + //eprintln!("{:x?}", ether_frame); + //let mut ipv6_bytes = remaining_buf.take(size_of::<Ipv6Header>() as u64); + //let ipv6_hdr = Ipv6Header::read_from_io(&mut ipv6_bytes)?; + //let remaining_buf = ipv6_bytes.into_inner(); + //eprintln!("{:x?}", ipv6_hdr); + + continue; + } + + let mut vlan_bytes = remaining_buf.take(size_of::<VlanTag>() as u64); + let vlan_tag = VlanTag::read_from_io(&mut vlan_bytes)?; + let remaining_buf = vlan_bytes.into_inner(); + + let vlan_id = u16::from(vlan_tag.tag_control_information) & 0xfff; + if vlan_id != args.upstream_interface { + debug!("dropping packet with vlan {}", vlan_id); + continue; + } + if !phys_mac.is_some() { + debug!("set phys mac"); + phys_mac = Some(ether_frame.dst_addr); + } + + if vlan_tag.ether_type != 0x86dd { + continue; + } + let mut ipv6_bytes = remaining_buf.take(size_of::<Ipv6Header>() as u64); + let ipv6_hdr = Ipv6Header::read_from_io(&mut ipv6_bytes)?; + let remaining_buf = ipv6_bytes.into_inner(); + trace!("{:x?}", ipv6_hdr); + let dst_addr = Ipv6Addr::from(ipv6_hdr.dst_addr); + + let Some((dst_mac, if_idx)) = fib.get(&dst_addr) else { + warn!("dropped incoming message to {} because no fib match", dst_addr); + continue; + }; + ether_frame.dst_addr = *dst_mac; + ether_frame.ether_type = vlan_tag.ether_type; + + sinks.get_mut(if_idx).unwrap().send(Box::new(io::Cursor::new(ether_frame).chain(io::Cursor::new(ipv6_hdr)).chain(remaining_buf))).await.unwrap(); + } + InterfaceId::App(_) => { + use io::Read; + let mut ether_bytes = buf.take(size_of::<EtherFrame>() as u64); + let mut ether_frame = EtherFrame::read_from_io(&mut ether_bytes)?; + let remaining_buf = ether_bytes.into_inner(); + trace!("{:x?}", ether_frame); + if ether_frame.ether_type != 0x86dd { + continue; + } + let mut ipv6_bytes = remaining_buf.take(size_of::<Ipv6Header>() as u64); + let ipv6_hdr = Ipv6Header::read_from_io(&mut ipv6_bytes)?; + let remaining_buf = ipv6_bytes.into_inner(); + trace!("{:x?}", ipv6_hdr); + let src_addr = Ipv6Addr::from(ipv6_hdr.src_addr); + trace!("{:x?}", src_addr); + if !src_addr.is_unspecified() && !src_addr.is_multicast() && !fib.contains_key(&src_addr) { + debug!("adding fib entry for {}", src_addr); + fib.insert(src_addr, (ether_frame.src_addr.clone(), key)); + } + + let Some(phys_mac) = &phys_mac else { + warn!("dropped message because phys mac is unknown"); + continue; + }; + let vlan_tag = VlanTag { + ether_type: ether_frame.ether_type, + tag_control_information: args.upstream_interface.into(), + }; + ether_frame.src_addr = phys_mac.clone(); + ether_frame.ether_type = 0x8100.into(); + + let Some(sink) = sinks.get_mut(&InterfaceId::Driver) else { + warn!("dropped message because driver is not ready"); + continue; + }; + + sink.send(Box::new(io::Cursor::new(ether_frame) + .chain(io::Cursor::new(vlan_tag)) + .chain(io::Cursor::new(ipv6_hdr)) + .chain(remaining_buf))) + .await?; + } + } + } + } + } +} -- 2.51.2
On 11/24/25 11:35, Yureka Lilian wrote:
The tokio-vhost & vhost-device-net crates which we also wrote and depend on are left external in the outlook of becoming a rust-vmm project soon.
Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- pkgs/default.nix | 2 + tools/router/Cargo.lock | 785 +++++++++++++++++++++++++++++++++++++++ tools/router/Cargo.toml | 18 + tools/router/default.nix | 18 + tools/router/src/main.rs | 235 ++++++++++++ 5 files changed, 1058 insertions(+) create mode 100644 tools/router/Cargo.lock create mode 100644 tools/router/Cargo.toml create mode 100644 tools/router/default.nix create mode 100644 tools/router/src/main.rs
diff --git a/pkgs/default.nix b/pkgs/default.nix index cc60228..7c1c9c3 100644 --- a/pkgs/default.nix +++ b/pkgs/default.nix @@ -52,6 +52,8 @@ let xdg-desktop-portal-spectrum-host = self.callSpectrumPackage ../tools/xdg-desktop-portal-spectrum-host {};
+ spectrum-router = self.callSpectrumPackage ../tools/router {}; + # Packages from the overlay, so it's possible to build them from # the CLI easily. inherit (pkgs) cloud-hypervisor dbus; diff --git a/tools/router/Cargo.lock b/tools/router/Cargo.lock new file mode 100644 index 0000000..04179af --- /dev/null +++ b/tools/router/Cargo.lock @@ -0,0 +1,785 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "anstream" +version = "0.6.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43d5b281e737544384e969a5ccad3f1cdd24b48086a0fc1b2a5262a26b8f4f4a" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5192cca8006f1fd4f7237516f40fa183bb07f8fbdfedaa0036de5ea9b0b45e78" + +[[package]] +name = "anstyle-parse" +version = "0.2.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4e7644824f0aa2c7b9384579234ef10eb7efb6a0deb83f9630a49594dd9c15c2" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys 0.61.2", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys 0.61.2", +] + +[[package]] +name = "anyhow" +version = "1.0.100" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a23eb6b1614318a8071c9b2521f36b424b2c83db5eb3a0fead4a6c0809af6e61" + +[[package]] +name = "async-stream" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0b5a71a6f37880a80d1d7f19efd781e4b5de42c88f0722cc13bcb6cc2cfe8476" +dependencies = [ + "async-stream-impl", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-stream-impl" +version = "0.3.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c7c24de15d275a1ecfd47a380fb4d5ec9bfe0933f309ed5e705b775596a3574d" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + +[[package]] +name = "bitvec" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bc2832c24239b0141d5674bb9174f9d68a8b5b3f2753311927c172ca46f7e9c" +dependencies = [ + "funty", + "radium", + "tap", + "wyz", +] + +[[package]] +name = "bytes" +version = "1.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b35204fbdc0b3f4446b89fc1ac2cf84a8a68971995d0bf2e925ec7cd960f9cb3" + +[[package]] +name = "clap" +version = "4.5.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c9e340e012a1bf4935f5282ed1436d1489548e8f72308207ea5df0e23d2d03f8" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.5.53" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d76b5d13eaa18c901fd2f7fca939fefe3a0727a953561fefdf3b2922b8569d00" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.5.49" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2a0b5487afeab2deb2ff4e03a807ad1a03ac532ff5a2cee5d86884440c7f7671" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "clap_lex" +version = "0.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a1d728cc89cf3aee9ff92b05e62b19ee65a02b5702cff7d5a377e32c6ae29d8d" + +[[package]] +name = "colorchoice" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b05b61dc5112cbb17e4b6cd61790d9845d13888356391624cbe7e41efeac1e75" + +[[package]] +name = "env_filter" +version = "0.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1bf3c259d255ca70051b30e2e95b5446cdb8949ac4cd22c0d7fd634d89f568e2" +dependencies = [ + "log", + "regex", +] + +[[package]] +name = "env_logger" +version = "0.11.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "13c863f0904021b108aa8b2f55046443e6b1ebde8fd4a15c399893aae4fa069f" +dependencies = [ + "anstream", + "anstyle", + "env_filter", + "jiff", + "log", +] + +[[package]] +name = "fastrand" +version = "2.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "37909eebbb50d72f9059c3b6d82c0463f2ff062c9e95845c43a6c9c0355411be" + +[[package]] +name = "funty" +version = "2.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6d5a32815ae3f33302d95fdcb2ce17862f8c65363dcfd29360480ba1001fc9c" + +[[package]] +name = "futures-core" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05f29059c0c2090612e8d742178b0580d2dc940c837851ad723096f87af6663e" + +[[package]] +name = "futures-io" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9e5c1b78ca4aae1ac06c48a526a655760685149f0d465d21f37abfe57ce075c6" + +[[package]] +name = "futures-lite" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +dependencies = [ + "fastrand", + "futures-core", + "futures-io", + "parking", + "pin-project-lite", +] + +[[package]] +name = "futures-macro" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "162ee34ebcb7c64a8abebc059ce0fee27c2262618d7b60ed8faf72fef13c3650" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "futures-sink" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e575fab7d1e0dcb8d0c7bcf9a63ee213816ab51902e6d244a95819acacf1d4f7" + +[[package]] +name = "futures-task" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f90f7dce0722e95104fcb095585910c0977252f286e354b5e3bd38902cd99988" + +[[package]] +name = "futures-util" +version = "0.3.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fa08315bb612088cc391249efdc3bc77536f16c91f6cf495e6fbe85b20a4a81" +dependencies = [ + "futures-core", + "futures-macro", + "futures-sink", + "futures-task", + "pin-project-lite", + "pin-utils", + "slab", +] + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "jiff" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "49cce2b81f2098e7e3efc35bc2e0a6b7abec9d34128283d7a26fa8f32a6dbb35" +dependencies = [ + "jiff-static", + "log", + "portable-atomic", + "portable-atomic-util", + "serde_core", +] + +[[package]] +name = "jiff-static" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "980af8b43c3ad5d8d349ace167ec8170839f753a42d233ba19e08afe1850fa69" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "libc" +version = "0.2.177" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2874a2af47a2325c2001a6e6fad9b16a53b802102b528163885171cf92b15976" + +[[package]] +name = "log" +version = "0.4.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34080505efa8e45a4b816c349525ebe327ceaa8559756f0356cba97ef3bf7432" + +[[package]] +name = "memchr" +version = "2.7.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f52b00d39961fc5b2736ea853c9cc86238e165017a493d1d5c8eac6bdc4cc273" + +[[package]] +name = "mio" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "69d83b0086dc8ecf3ce9ae2874b2d1290252e2a30720bea58a5c6639b0092873" +dependencies = [ + "libc", + "wasi", + "windows-sys 0.61.2", +] + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "parking" +version = "2.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" + +[[package]] +name = "pin-project-lite" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b3cff922bd51709b605d9ead9aa71031d81447142d828eb4a6eba76fe619f9b" + +[[package]] +name = "pin-utils" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184" + +[[package]] +name = "portable-atomic" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483" + +[[package]] +name = "portable-atomic-util" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8a2f0d8d040d7848a709caf78912debcc3f33ee4b3cac47d73d1e1069e83507" +dependencies = [ + "portable-atomic", +] + +[[package]] +name = "proc-macro2" +version = "1.0.103" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5ee95bc4ef87b8d5ba32e8b7714ccc834865276eab0aed5c9958d00ec45f49e8" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.42" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a338cc41d27e6cc6dce6cefc13a0729dfbb81c262b1f519331575dd80ef3067f" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "radium" +version = "0.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc33ff2d4973d518d823d61aa239014831e521c75da58e3df4840d3f47749d09" + +[[package]] +name = "regex" +version = "1.12.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "843bc0191f75f3e22651ae5f1e72939ab2f72a4bc30fa80a066bd66edefc24d4" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5276caf25ac86c8d810222b3dbb938e512c55c6831a10f3e6ed1c93b84041f1c" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2d987857b319362043e95f5353c0535c1f58eec5336fdfcf626430af7def58" + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "slab" +version = "0.4.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7a2ae44ef20feb57a68b23d846850f861394c2e02dc425a50098ae8c90267589" + +[[package]] +name = "socket2" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "17129e116933cf371d018bb80ae557e889637989d8638274fb25622827b03881" +dependencies = [ + "libc", + "windows-sys 0.60.2", +] + +[[package]] +name = "spectrum-router" +version = "0.1.0" +dependencies = [ + "anyhow", + "clap", + "env_logger", + "futures-util", + "log", + "tokio", + "tokio-stream", + "vhost-device-net", + "zerocopy", +] + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "syn" +version = "2.0.110" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a99801b5bd34ede4cf3fc688c5919368fea4e4814a4664359503e6015b280aea" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tap" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369" + +[[package]] +name = "thiserror" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6aaf5339b578ea85b50e080feb250a3e8ae8cfcdff9a461c9ec2904bc923f52" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "1.0.69" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4fee6c4efc90059e10f81e6d42c60a18f76588c3d74cb83a0b242a2b6c7504c1" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tokio" +version = "1.48.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff360e02eab121e0bc37a2d3b4d4dc622e6eda3a8e5253d5435ecf5bd4c68408" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "socket2", + "tokio-macros", + "windows-sys 0.61.2", +] + +[[package]] +name = "tokio-eventfd" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7e66bd133670ac39baa1aca5c3a86709f4595c08ca4464a1e1400b83d62c0639" +dependencies = [ + "futures-lite", + "libc", + "tokio", +] + +[[package]] +name = "tokio-macros" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af407857209536a95c8e56f8231ef2c2e2aff839b22e07a1ffcbc617e9db9fa5" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "tokio-stream" +version = "0.1.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "eca58d7bba4a75707817a2c44174253f9236b2d5fbd055602e9d5c07c139a047" +dependencies = [ + "futures-core", + "pin-project-lite", + "tokio", +] + +[[package]] +name = "tokio-vhost" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "21d64b3e4d573da90b2bb040d69a9c2d754e8a3ab9d9ecf04a268748c99f1cd3" +dependencies = [ + "async-stream", + "bitvec", + "futures-util", + "libc", + "log", + "tokio", + "tokio-eventfd", + "virtio-queue", + "vm-memory", + "zerocopy", +] + +[[package]] +name = "unicode-ident" +version = "1.0.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9312f7c4f6ff9069b165498234ce8be658059c6728633667c526e27dc2cf1df5" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "vhost-device-net" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac05caccd6d484f672551a187f7110ff9d32edd6a39bb16bb04f53017b1e6fd0" +dependencies = [ + "futures-util", + "log", + "tokio", + "tokio-vhost", + "vm-memory", +] + +[[package]] +name = "virtio-bindings" +version = "0.2.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "804f498a26d5a63be7bbb8bdcd3869c3f286c4c4a17108905276454da0caf8cb" + +[[package]] +name = "virtio-queue" +version = "0.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb0479158f863e59323771a1f684d843962f76960b86fecfec2bfa9c8f0f9180" +dependencies = [ + "log", + "virtio-bindings", + "vm-memory", + "vmm-sys-util", +] + +[[package]] +name = "vm-memory" +version = "0.16.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd5e56d48353c5f54ef50bd158a0452fc82f5383da840f7b8efc31695dd3b9d" +dependencies = [ + "libc", + "thiserror", + "winapi", +] + +[[package]] +name = "vmm-sys-util" +version = "0.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d21f366bf22bfba3e868349978766a965cbe628c323d58e026be80b8357ab789" +dependencies = [ + "bitflags", + "libc", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "winapi" +version = "0.3.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5c839a674fcd7a98952e593242ea400abe93992746761e38641405d28b00f419" +dependencies = [ + "winapi-i686-pc-windows-gnu", + "winapi-x86_64-pc-windows-gnu", +] + +[[package]] +name = "winapi-i686-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" + +[[package]] +name = "winapi-x86_64-pc-windows-gnu" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.60.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f500e4d28234f72040990ec9d39e3a6b950f9f22d3dba18416c35882612bcb" +dependencies = [ + "windows-targets", +] + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "windows-targets" +version = "0.53.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4945f9f551b88e0d65f3db0bc25c33b8acea4d9e41163edf90dcd0b19f9069f3" +dependencies = [ + "windows-link", + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_gnullvm", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a9d8416fa8b42f5c947f8482c43e7d89e73a173cead56d044f6a56104a6d1b53" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9d782e804c2f632e395708e99a94275910eb9100b2114651e04744e9b125006" + +[[package]] +name = "windows_i686_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "960e6da069d81e09becb0ca57a65220ddff016ff2d6af6a223cf372a506593a3" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fa7359d10048f68ab8b09fa71c3daccfb0e9b559aed648a8f95469c27057180c" + +[[package]] +name = "windows_i686_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1e7ac75179f18232fe9c285163565a57ef8d3c89254a30685b57d83a38d326c2" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9c3842cdd74a865a8066ab39c8a7a473c0778a3f29370b5fd6b4b9aa7df4a499" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ffa179e2d07eee8ad8f57493436566c7cc30ac536a3379fdf008f47f6bb7ae1" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6bbff5f0aada427a1e5a6da5f1f98158182f26556f345ac9e04d36d0ebed650" + +[[package]] +name = "wyz" +version = "0.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "05f360fc0b24296329c78fda852a1e9ae82de9cf7b27dae4b7f62f118f77b9ed" +dependencies = [ + "tap", +] + +[[package]] +name = "zerocopy" +version = "0.8.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43fa6694ed34d6e57407afbccdeecfa268c470a7d2a5b0cf49ce9fcc345afb90" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c640b22cd9817fae95be82f0d2f90b11f7605f6c319d16705c459b27ac2cbc26" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] diff --git a/tools/router/Cargo.toml b/tools/router/Cargo.toml new file mode 100644 index 0000000..f9d8138 --- /dev/null +++ b/tools/router/Cargo.toml @@ -0,0 +1,18 @@ +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> + +[package] +name = "spectrum-router" +version = "0.1.0" +edition = "2024" + +[dependencies] +anyhow = "1.0.100" +clap = { version = "4.5.45", features = ["derive"] } +env_logger = "0.11.8" +log = { version = "0.4.27", features = ["release_max_level_debug"] } +vhost-device-net = "0.1.0" +tokio = { version = "1.48.0", features = ["macros", "rt"] } +futures-util = "0.3.31" +zerocopy = "0.8.27" +tokio-stream = "0.1.17" diff --git a/tools/router/default.nix b/tools/router/default.nix new file mode 100644 index 0000000..e70f9ec --- /dev/null +++ b/tools/router/default.nix @@ -0,0 +1,18 @@ +# SPDX-FileCopyrightText: 2024 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> +# SPDX-License-Identifier: MIT + +import ../../lib/call-package.nix ( +{ src, lib, rustPlatform }: + +rustPlatform.buildRustPackage { + name = "spectrum-router"; + + src = lib.fileset.toSource { + root = ../..; + fileset = lib.fileset.intersection src ./.; + }; + sourceRoot = "source/tools/router"; + + cargoLock.lockFile = ./Cargo.lock; +}) (_: {}) diff --git a/tools/router/src/main.rs b/tools/router/src/main.rs new file mode 100644 index 0000000..42fb8f7 --- /dev/null +++ b/tools/router/src/main.rs @@ -0,0 +1,235 @@ +// SPDX-License-Identifier: EUPL-1.2+ +// SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> + +use std::collections::HashMap; +use std::io; +use std::net::Ipv6Addr; +use std::path::PathBuf; + +use clap::Parser;
Nit: Would it make sense to use a lighter option parser, possibly even C getopt? This isn't going to be used interactively, so completions aren't a concern. Reducing the number of dependencies would be a good idea.
+use futures_util::SinkExt; +use futures_util::StreamExt; +use log::{debug, error, info, trace, warn}; +use tokio::net::UnixListener; +use tokio_stream::StreamMap; +use vhost_device_net::VhostDeviceNet; + +use zerocopy::byteorder::network_endian::{U16, U32}; +use zerocopy::*; + +type MacAddr = [u8; 6]; + +#[derive(Debug, PartialEq, Eq, FromBytes, IntoBytes, KnownLayout, Immutable, Unaligned)] +#[repr(C)] +struct EtherFrame { + //preamble_sfd: U64, + dst_addr: MacAddr, + src_addr: MacAddr, + ether_type: U16, + //data: [u8] +} + +impl AsRef<[u8]> for EtherFrame { + fn as_ref(&self) -> &[u8] { + self.as_bytes() + } +} + +#[derive(Debug, PartialEq, Eq, FromBytes, IntoBytes, KnownLayout, Immutable, Unaligned)] +#[repr(C)] +struct VlanTag { + tag_control_information: U16, + ether_type: U16, +} + +impl AsRef<[u8]> for VlanTag { + fn as_ref(&self) -> &[u8] { + self.as_bytes() + } +} + +#[derive(Debug, PartialEq, Eq, FromBytes, IntoBytes, KnownLayout, Immutable, Unaligned)] +#[repr(C)] +struct Ipv6Header { + version_traffic_class_flow_label: U32, + payload_length: U16, + hext_header: u8, + hop_limit: u8, + src_addr: [u8; 16], + dst_addr: [u8; 16], + //data: [u8] +} + +impl AsRef<[u8]> for Ipv6Header { + fn as_ref(&self) -> &[u8] { + self.as_bytes() + } +} + +#[derive(Parser, Debug)] +#[command()] //version = None, about = None, long_about = None)] +struct Args { + #[arg(long)] + driver_listen_path: PathBuf, + #[arg(long)] + app_listen_path: PathBuf, + #[arg(long)] + upstream_interface: u16, +} + +#[tokio::main(flavor = "current_thread")] +async fn main() -> anyhow::Result<()> { + env_logger::init(); + let args = Args::parse(); + + run_router(args).await +} + +#[derive(Debug, Clone, PartialEq, Eq, Hash)] +enum InterfaceId { + Driver, + App(usize), +} + +async fn run_router(args: Args) -> anyhow::Result<()> { + let driver_listener = UnixListener::bind(&args.driver_listen_path)?; + let app_listener = UnixListener::bind(&args.app_listen_path)?; + + let fib: HashMap<Ipv6Addr, (MacAddr, InterfaceId)> = Default::default(); + + let mut fib = fib.clone(); + let mut phys_mac = None; + + let mut streams = StreamMap::new();
This won't scale to large numbers of streams. See the documentation:
In general, StreamMap works best with a “smallish” number of streams as all entries are scanned on insert, remove, and polling.
Also, would it be possible to provide a type signature here? It's not necessary, but it makes the code quite a bit easier to review, especially when one doesn't have an IDE.
+ let mut sinks = HashMap::<InterfaceId, _>::new(); + + let mut app_num = 0; + + loop { + tokio::select! {
tokio::select! isn't cancellation-safe in general. Would it make sense to use multiple tasks instead?
+ driver_conn = driver_listener.accept() => { + info!("driver connected"); + match driver_conn { + Ok((stream, _addr)) => { + let device = VhostDeviceNet::from_unix_stream(stream).await?; + streams.insert(InterfaceId::Driver, Box::pin(device.tx().await?)); + sinks.insert(InterfaceId::Driver, Box::pin(device.rx::<Box<dyn io::Read + Send + Sync + 'static>>().await?)); + phys_mac = None; + } + Err(e) => error!("driver connection failed: {}", e), + } + } + app_conn = app_listener.accept() => { + info!("app connected"); + match app_conn { + Ok((stream, _addr)) => { + let device = VhostDeviceNet::from_unix_stream(stream).await?; + streams.insert(InterfaceId::App(app_num), Box::pin(device.tx().await?)); + sinks.insert(InterfaceId::App(app_num), Box::pin(device.rx().await?)); + app_num = app_num.checked_add(1).unwrap(); + } + Err(e) => error!("app connection failed: {}", e), + } + } + next_res = streams.next(), if !streams.is_empty() => { + let Some((key, Ok(buf))) = next_res else { + info!("incoming other"); + continue; + }; + match &key { + InterfaceId::Driver => { + use io::Read; + let mut ether_bytes = buf.take(size_of::<EtherFrame>() as u64); + let mut ether_frame = EtherFrame::read_from_io(&mut ether_bytes)?; + let remaining_buf = ether_bytes.into_inner(); + + if ether_frame.ether_type != 0x8100 { + warn!("untagged packet from driver");
Anything that is logged by default should be rate-limited to avoid filling the host disk or consuming too much CPU.
+ //eprintln!("{:x?}", ether_frame); + //let mut ipv6_bytes = remaining_buf.take(size_of::<Ipv6Header>() as u64); + //let ipv6_hdr = Ipv6Header::read_from_io(&mut ipv6_bytes)?; + //let remaining_buf = ipv6_bytes.into_inner(); + //eprintln!("{:x?}", ipv6_hdr);
Would it be possible to make these into proper logs at debug level?
+ continue; + } + + let mut vlan_bytes = remaining_buf.take(size_of::<VlanTag>() as u64); + let vlan_tag = VlanTag::read_from_io(&mut vlan_bytes)?; + let remaining_buf = vlan_bytes.into_inner(); + + let vlan_id = u16::from(vlan_tag.tag_control_information) & 0xfff; + if vlan_id != args.upstream_interface { + debug!("dropping packet with vlan {}", vlan_id); + continue; + } + if !phys_mac.is_some() { + debug!("set phys mac"); + phys_mac = Some(ether_frame.dst_addr); + } + + if vlan_tag.ether_type != 0x86dd { + continue; + }
Could this use a named constant instead of a magic number?
+ let mut ipv6_bytes = remaining_buf.take(size_of::<Ipv6Header>() as u64); + let ipv6_hdr = Ipv6Header::read_from_io(&mut ipv6_bytes)?; + let remaining_buf = ipv6_bytes.into_inner(); + trace!("{:x?}", ipv6_hdr); + let dst_addr = Ipv6Addr::from(ipv6_hdr.dst_addr); + + let Some((dst_mac, if_idx)) = fib.get(&dst_addr) else { + warn!("dropped incoming message to {} because no fib match", dst_addr); + continue; + }; + ether_frame.dst_addr = *dst_mac; + ether_frame.ether_type = vlan_tag.ether_type; + + sinks.get_mut(if_idx).unwrap().send(Box::new(io::Cursor::new(ether_frame).chain(io::Cursor::new(ipv6_hdr)).chain(remaining_buf))).await.unwrap(); + } + InterfaceId::App(_) => { + use io::Read; + let mut ether_bytes = buf.take(size_of::<EtherFrame>() as u64); + let mut ether_frame = EtherFrame::read_from_io(&mut ether_bytes)?; + let remaining_buf = ether_bytes.into_inner(); + trace!("{:x?}", ether_frame); + if ether_frame.ether_type != 0x86dd { + continue; + }
Same here.
+ let mut ipv6_bytes = remaining_buf.take(size_of::<Ipv6Header>() as u64); + let ipv6_hdr = Ipv6Header::read_from_io(&mut ipv6_bytes)?; + let remaining_buf = ipv6_bytes.into_inner(); + trace!("{:x?}", ipv6_hdr); + let src_addr = Ipv6Addr::from(ipv6_hdr.src_addr); + trace!("{:x?}", src_addr); + if !src_addr.is_unspecified() && !src_addr.is_multicast() && !fib.contains_key(&src_addr) { + debug!("adding fib entry for {}", src_addr); + fib.insert(src_addr, (ether_frame.src_addr.clone(), key)); + }
Does this break Duplicate Address Detection?
+ let Some(phys_mac) = &phys_mac else { + warn!("dropped message because phys mac is unknown"); + continue; + }; + let vlan_tag = VlanTag { + ether_type: ether_frame.ether_type, + tag_control_information: args.upstream_interface.into(), + }; + ether_frame.src_addr = phys_mac.clone(); + ether_frame.ether_type = 0x8100.into(); + + let Some(sink) = sinks.get_mut(&InterfaceId::Driver) else { + warn!("dropped message because driver is not ready"); + continue; + }; + + sink.send(Box::new(io::Cursor::new(ether_frame) + .chain(io::Cursor::new(vlan_tag)) + .chain(io::Cursor::new(ipv6_hdr)) + .chain(remaining_buf))) + .await?; + } + } + } + } + } +}
Some general notes that do not need to be addressed (yet): - The overall design looks somewhat inefficient. There are multiple copies and lots of heap allocations involved. Some optimization opportunities: - memcpy() headers between VM memory and on-stack buffers. - memcpy() packet bodies from one VM directly to another VM. - Batch operations so that each individual operation fits in the L1 I-cache if it doesn't already. This amortizes the cost of filling up the L1 I-cache across the entire batch. - Batch any needed table lookups. For instance, if the FIB is a longest-prefix lookup tree, one can prefetch all of the cache lines that must be accessed before actually loading any of them. - If there are a bunch of packets in the buffer and the buffer just became empty, spin for a small amount of time before going to sleep. - The general-purpose Tokio datastructures you are using are probably suboptimal performance-wise. With proper optimization, I expect that you should be able to beat both pasta and the Linux kernel, possibly by a substantial margin. This should *not* block merging, but I think it *should* be done at some point. If nothing else, it will improve battery life during heavy network I/O. - Many people (like myself) do not have IPv6 connectivity at all. Therefore, this completely breaks networking for them. I think that the old path should remain as an option until IPv4 support is implemented. -- Sincerely, Demi Marie Obenour (she/her/hers)
This removes the old host bridge + taps glue, and instead connects the apps to their net provider's router instance. Signed-off-by: Yureka Lilian <yureka@cyberchaos.dev> --- host/rootfs/default.nix | 4 +- host/rootfs/file-list.mk | 2 + .../data/service/spectrum-router/down | 0 .../template/data/service/spectrum-router/run | 13 ++++ host/rootfs/image/usr/bin/run-vmm | 12 ---- host/rootfs/image/usr/bin/vm-import | 13 ---- pkgs/overlay.nix | 1 + tools/start-vmm/ch.rs | 40 ++--------- tools/start-vmm/lib.rs | 68 ++++++++++++------- tools/start-vmm/meson.build | 2 +- tools/start-vmm/net-util.c | 39 ----------- tools/start-vmm/net-util.h | 6 -- tools/start-vmm/net.c | 55 --------------- tools/start-vmm/net.rs | 10 --- tools/start-vmm/tests/meson.build | 5 -- .../start-vmm/tests/tap_open-name-too-long.c | 20 ------ tools/start-vmm/tests/tap_open.c | 28 -------- vm/sys/net/default.nix | 4 +- 18 files changed, 68 insertions(+), 254 deletions(-) create mode 100644 host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down create mode 100644 host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run delete mode 100644 tools/start-vmm/net-util.c delete mode 100644 tools/start-vmm/net-util.h delete mode 100644 tools/start-vmm/net.c delete mode 100644 tools/start-vmm/tests/tap_open-name-too-long.c delete mode 100644 tools/start-vmm/tests/tap_open.c diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index 0ac70c7..ba0d24a 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -8,7 +8,7 @@ import ../../lib/call-package.nix ( }: pkgsStatic.callPackage ( -{ spectrum-host-tools +{ spectrum-host-tools, spectrum-router , lib, stdenvNoCC, nixos, runCommand, writeClosure, erofs-utils, s6-rc , busybox, cloud-hypervisor, cryptsetup, dbus, execline, inkscape , iproute2, inotify-tools, jq, mdevd, s6, s6-linux-init, socat @@ -34,7 +34,7 @@ let packages = [ cloud-hypervisor cryptsetup dbus execline inotify-tools iproute2 - jq mdevd s6 s6-linux-init s6-rc socat spectrum-host-tools + jq mdevd s6 s6-linux-init s6-rc socat spectrum-host-tools spectrum-router virtiofsd xdg-desktop-portal-spectrum-host (busybox.override { diff --git a/host/rootfs/file-list.mk b/host/rootfs/file-list.mk index ff6fd1b..3a595b7 100644 --- a/host/rootfs/file-list.mk +++ b/host/rootfs/file-list.mk @@ -25,6 +25,8 @@ FILES = \ image/etc/s6-linux-init/run-image/service/vm-services/run \ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/notification-fd \ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/dbus/run \ + image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down \ + image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run \ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/notification-fd \ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-fs/run \ image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/vhost-user-gpu/data/check \ diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/down new file mode 100644 index 0000000..e69de29 diff --git a/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run new file mode 100644 index 0000000..b8f831e --- /dev/null +++ b/host/rootfs/image/etc/s6-linux-init/run-image/service/vm-services/template/data/service/spectrum-router/run @@ -0,0 +1,13 @@ +#!/bin/execlineb -P +# SPDX-License-Identifier: EUPL-1.2+ +# SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> + +importas -i VM VM + +background { + ch-remote --api-socket ${VM}/vmm add-net id=router,vhost_user=on,socket=${VM}/router-driver.sock,mac=02:01:00:00:00:01 +} + +export RUST_LOG debug +spectrum-router --app-listen-path ${VM}/router-app.sock --driver-listen-path ${VM}/router-driver.sock --upstream-interface 2 + diff --git a/host/rootfs/image/usr/bin/run-vmm b/host/rootfs/image/usr/bin/run-vmm index 5649674..919a69d 100755 --- a/host/rootfs/image/usr/bin/run-vmm +++ b/host/rootfs/image/usr/bin/run-vmm @@ -36,18 +36,6 @@ background -d { if { test $client_id != $1 } test $router_id != $1 } - - backtick -E mac { - pipeline { ip -j link show client-${client_id} } - pipeline { jq -r ".[].ifindex" } - awk "{ - printf \"02:01:%02X:%02X:%02X:%02X\", $0 / 256 ^ 3 % 256, - $0 / 256 ^ 2 % 256, $0 / 256 % 256, $0 % 256 - }" - } - - ch-remote --api-socket /run/vm/by-id/${router_id}/vmm add-net - id=router-${client_id},tap=router-${client_id},mac=${mac} } unexport ! fdmove -c 3 0 diff --git a/host/rootfs/image/usr/bin/vm-import b/host/rootfs/image/usr/bin/vm-import index de88f08..c1d1bbc 100755 --- a/host/rootfs/image/usr/bin/vm-import +++ b/host/rootfs/image/usr/bin/vm-import @@ -14,19 +14,6 @@ if { ln -s -- ${dir} /run/vm/by-name/${1}.${name} } if { ln -s -- ${2}/${name} ${dir}/config } if { ln -s -- /run/service/vmm/instance/${id} ${dir}/service } -if { - if -t { elglob -0d " " providers ${name}/providers/net test -n $providers } - - if { ip link add br-${id} type bridge } - if { ip link set br-${id} up } - - if { ip tuntap add client-${id} mode tap } - if { ip link set client-${id} master br-${id} up } - - if { ip tuntap add router-${id} mode tap } - ip link set router-${id} master br-${id} up -} - if { create-vm-dependencies $id } s6-instance-create -- /run/service/vmm $id diff --git a/pkgs/overlay.nix b/pkgs/overlay.nix index 55cb00c..d2c8331 100644 --- a/pkgs/overlay.nix +++ b/pkgs/overlay.nix @@ -3,4 +3,5 @@ (final: super: { cloud-hypervisor = import ./cloud-hypervisor { inherit final super; }; + mailutils = super.mailutils.overrideAttrs (_: { doCheck = false; }); }) diff --git a/tools/start-vmm/ch.rs b/tools/start-vmm/ch.rs index 35519cd..5aca1dc 100644 --- a/tools/start-vmm/ch.rs +++ b/tools/start-vmm/ch.rs @@ -1,7 +1,7 @@ // SPDX-License-Identifier: EUPL-1.2+ // SPDX-FileCopyrightText: 2022-2024 Alyssa Ross <hi@alyssa.is> +// SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> -use std::convert::TryFrom; use std::ffi::OsStr; use std::fs::File; use std::io::Write; @@ -10,7 +10,6 @@ use std::num::NonZeroI32; use std::os::unix::prelude::*; use std::path::Path; use std::process::{Command, Stdio}; -use std::string::FromUtf8Error; use miniserde::{json, Serialize}; @@ -46,7 +45,7 @@ pub struct GpuConfig { #[derive(Serialize)] pub struct NetConfig { - pub fd: RawFd, + pub vhost_user_sock: String, pub id: String, pub mac: MacAddress, } @@ -136,13 +135,10 @@ pub fn create_vm(vm_dir: &Path, ready_fd: File, mut config: VmConfig) -> Result< } pub fn add_net(vm_dir: &Path, net: &NetConfig) -> Result<(), NonZeroI32> { - // TODO: re-enable offloading once - // https://lore.kernel.org/regressions/87y0ota32b.fsf@alyssa.is/ - // is fixed. let mut ch_remote = command(vm_dir, "add-net") .arg(format!( - "fd={},id={},mac={},offload_tso=false,offload_ufo=false,offload_csum=false", - net.fd, net.id, net.mac + "vhost_user=on,socket={},id={},mac={}", + net.vhost_user_sock, net.id, net.mac )) .stdout(Stdio::piped()) .spawn() @@ -156,31 +152,3 @@ pub fn add_net(vm_dir: &Path, net: &NetConfig) -> Result<(), NonZeroI32> { Err(EPROTO) } - -#[repr(C)] -pub struct NetConfigC { - pub fd: RawFd, - pub id: [u8; 18], - pub mac: MacAddress, -} - -impl<'a> TryFrom<&'a NetConfigC> for NetConfig { - type Error = FromUtf8Error; - - fn try_from(c: &'a NetConfigC) -> Result<NetConfig, Self::Error> { - let nul_index = c.id.iter().position(|&c| c == 0).unwrap_or(c.id.len()); - Ok(NetConfig { - fd: c.fd, - id: String::from_utf8(c.id[..nul_index].to_vec())?, - mac: c.mac, - }) - } -} - -impl TryFrom<NetConfigC> for NetConfig { - type Error = FromUtf8Error; - - fn try_from(c: NetConfigC) -> Result<NetConfig, Self::Error> { - Self::try_from(&c) - } -} diff --git a/tools/start-vmm/lib.rs b/tools/start-vmm/lib.rs index 4586094..112c56c 100644 --- a/tools/start-vmm/lib.rs +++ b/tools/start-vmm/lib.rs @@ -1,23 +1,23 @@ // SPDX-License-Identifier: EUPL-1.2+ // SPDX-FileCopyrightText: 2022-2024 Alyssa Ross <hi@alyssa.is> +// SPDX-FileCopyrightText: 2025 Yureka Lilian <yureka@cyberchaos.dev> mod ch; mod net; mod s6; use std::borrow::Cow; -use std::convert::TryInto; use std::env::args_os; use std::ffi::OsStr; use std::fs::File; -use std::io::{self, ErrorKind}; +use std::io::ErrorKind; use std::path::Path; use ch::{ - ConsoleConfig, DiskConfig, FsConfig, GpuConfig, LandlockConfig, MemoryConfig, PayloadConfig, - VmConfig, VsockConfig, + ConsoleConfig, DiskConfig, FsConfig, GpuConfig, LandlockConfig, MemoryConfig, NetConfig, + PayloadConfig, VmConfig, VsockConfig, }; -use net::net_setup; +use net::MacAddress; pub fn prog_name() -> String { args_os() @@ -40,8 +40,6 @@ pub fn vm_config(vm_dir: &Path) -> Result<VmConfig, String> { return Err(format!("VM name may not contain a colon: {vm_name:?}")); } - let name_bytes = vm_name.as_bytes(); - let config_dir = vm_dir.join("config"); let blk_dir = config_dir.join("blk"); let kernel_path = config_dir.join("vmlinux"); @@ -93,24 +91,44 @@ pub fn vm_config(vm_dir: &Path) -> Result<VmConfig, String> { shared: true, }, net: match net_providers_dir.read_dir() { - Ok(_) => { - // SAFETY: we check the result. - let net = unsafe { - net_setup( - name_bytes.as_ptr().cast(), - name_bytes - .len() - .try_into() - .map_err(|e| format!("VM name too long: {e}"))?, - ) - }; - if net.fd == -1 { - let e = io::Error::last_os_error(); - return Err(format!("setting up networking failed: {e}")); - } - - vec![net.try_into().unwrap()] - } + Ok(entries) => entries + .into_iter() + .map(|result| { + Ok(result + .map_err(|e| format!("examining directory entry: {e}"))? + .path()) + }) + .map(|result: Result<_, String>| { + let provider_name = result?.file_name().ok_or("unable to get net provider name".to_string())?.to_str().unwrap().to_string(); + + if provider_name.contains(',') { + return Err(format!("illegal ',' character in net provider name {provider_name:?}")); + } + + let mac = MacAddress::new([ + 0x02, // IEEE 802c administratively assigned + 0x00, // Spectrum client + 0x00, 0x00, 0x00, 0x01 // TODO + ]); + + let provider_id = std::fs::read_link(format!("/run/vm/by-name/{provider_name}")).map_err(|e| format!("unable to get net provider id: {e}"))?.file_name().ok_or("unable to get net provider id".to_string())?.to_str().unwrap().to_string(); + + let svc_dir = format!("/run/service/vm-services/instance/{provider_id}/data/service/spectrum-router"); + let svc_status = std::process::Command::new("s6-svc") + .args(["-U", &svc_dir]) + .status() + .expect("setting up the upstream router via s6-svc failed"); + if !svc_status.success() { + return Err(format!("setting up the upstream router via s6-svc failed with exit code {svc_status}")); + } + + Ok(NetConfig { + vhost_user_sock: format!("/run/vm/by-name/{provider_name}/router-app.sock").into(), + id: provider_name.into(), + mac, + }) + }) + .collect::<Result<_, _>>()?, Err(e) if e.kind() == ErrorKind::NotFound => Default::default(), Err(e) => return Err(format!("reading directory {net_providers_dir:?}: {e}")), }, diff --git a/tools/start-vmm/meson.build b/tools/start-vmm/meson.build index d07c5a0..aa9f6f3 100644 --- a/tools/start-vmm/meson.build +++ b/tools/start-vmm/meson.build @@ -1,7 +1,7 @@ # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2022-2024 Alyssa Ross <hi@alyssa.is> -c_lib = static_library('start-vmm', 'net.c', 'net-util.c', +c_lib = static_library('start-vmm', c_args : '-D_GNU_SOURCE') rust_lib = static_library('start_vmm', 'lib.rs', diff --git a/tools/start-vmm/net-util.c b/tools/start-vmm/net-util.c deleted file mode 100644 index 49003e9..0000000 --- a/tools/start-vmm/net-util.c +++ /dev/null @@ -1,39 +0,0 @@ -// SPDX-License-Identifier: EUPL-1.2+ -// SPDX-FileCopyrightText: 2022, 2024 Alyssa Ross <hi@alyssa.is> - -#include "net-util.h" - -#include <errno.h> -#include <fcntl.h> -#include <string.h> -#include <unistd.h> - -#include <sys/ioctl.h> - -#include <linux/if_tun.h> - -int tap_open(char name[static IFNAMSIZ], int flags) -{ - struct ifreq ifr; - int fd, e; - - if (strnlen(name, IFNAMSIZ) == IFNAMSIZ) { - errno = ENAMETOOLONG; - return -1; - } - - strncpy(ifr.ifr_name, name, IFNAMSIZ - 1); - ifr.ifr_flags = IFF_TAP|flags; - - if ((fd = open("/dev/net/tun", O_RDWR)) == -1) - return -1; - if (ioctl(fd, TUNSETIFF, &ifr) == -1) { - e = errno; - close(fd); - errno = e; - return -1; - } - - strncpy(name, ifr.ifr_name, IFNAMSIZ); - return fd; -} diff --git a/tools/start-vmm/net-util.h b/tools/start-vmm/net-util.h deleted file mode 100644 index 8f55206..0000000 --- a/tools/start-vmm/net-util.h +++ /dev/null @@ -1,6 +0,0 @@ -// SPDX-License-Identifier: EUPL-1.2+ -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> - -#include <net/if.h> - -int tap_open(char name[static IFNAMSIZ], int flags); diff --git a/tools/start-vmm/net.c b/tools/start-vmm/net.c deleted file mode 100644 index 78fe7f6..0000000 --- a/tools/start-vmm/net.c +++ /dev/null @@ -1,55 +0,0 @@ -// SPDX-License-Identifier: EUPL-1.2+ -// SPDX-FileCopyrightText: 2022-2024 Alyssa Ross <hi@alyssa.is> - -#include "ch.h" -#include "net-util.h" - -#include <assert.h> -#include <errno.h> -#include <stdio.h> -#include <stdlib.h> -#include <string.h> -#include <unistd.h> - -#include <arpa/inet.h> - -#include <linux/if_tun.h> - -static int get_tap_name(char tap_name[static IFNAMSIZ], - const char tap_prefix[static 1], - const char name[static 1], int name_len) -{ - int r = snprintf(tap_name, IFNAMSIZ, "%s-%*s", tap_prefix, name_len, name); - if (r >= IFNAMSIZ) - errno = ENAMETOOLONG; - return r < 0 || r >= IFNAMSIZ ? -1 : 0; -} - -struct net_config net_setup(const char name[static 1], int name_len) -{ - int e; - unsigned int client_index; - struct net_config r = { .fd = -1, .mac = { 0 } }; - - if ((get_tap_name(r.id, "client", name, name_len)) == -1) - return r; - - if (!(client_index = htonl(if_nametoindex(r.id)))) - return r; - - if ((r.fd = tap_open(r.id, IFF_NO_PI|IFF_VNET_HDR)) == -1) - goto fail_close; - - r.mac[0] = 0x02; // IEEE 802c administratively assigned - r.mac[1] = 0x00; // Spectrum client - memcpy(&r.mac[2], &client_index, 4); - - return r; - -fail_close: - e = errno; - close(r.fd); - errno = e; - r.fd = -1; - return r; -} diff --git a/tools/start-vmm/net.rs b/tools/start-vmm/net.rs index c94bca7..64f6014 100644 --- a/tools/start-vmm/net.rs +++ b/tools/start-vmm/net.rs @@ -2,14 +2,11 @@ // SPDX-FileCopyrightText: 2022-2024 Alyssa Ross <hi@alyssa.is> use std::borrow::Cow; -use std::ffi::{c_char, c_int}; use std::fmt::{self, Display, Formatter}; use miniserde::ser::Fragment; use miniserde::Serialize; -use crate::ch::NetConfigC; - #[repr(transparent)] #[derive(Copy, Clone)] pub struct MacAddress([u8; 6]); @@ -36,13 +33,6 @@ impl Serialize for MacAddress { } } -extern "C" { - /// # Safety - /// - /// The rest of the result is only valid if the returned fd is not -1. - pub fn net_setup(name: *const c_char, len: c_int) -> NetConfigC; -} - #[cfg(test)] mod tests { use super::*; diff --git a/tools/start-vmm/tests/meson.build b/tools/start-vmm/tests/meson.build index bfdfc46..5538822 100644 --- a/tools/start-vmm/tests/meson.build +++ b/tools/start-vmm/tests/meson.build @@ -4,11 +4,6 @@ rust_helper = static_library('test_helper', 'helper.rs', dependencies : rust_lib_dep) -test('tap_open', executable('tap_open', 'tap_open.c', '../net-util.c', - c_args : '-D_GNU_SOURCE')) -test('tap_open (name too long)', executable('tap_open-name-too-long', - 'tap_open-name-too-long.c', '../net-util.c', c_args : '-D_GNU_SOURCE')) - test('vm_command-basic', executable('vm_command-basic', 'vm_command-basic.rs', dependencies : rust_lib_dep, diff --git a/tools/start-vmm/tests/tap_open-name-too-long.c b/tools/start-vmm/tests/tap_open-name-too-long.c deleted file mode 100644 index ba4ebd6..0000000 --- a/tools/start-vmm/tests/tap_open-name-too-long.c +++ /dev/null @@ -1,20 +0,0 @@ -// SPDX-License-Identifier: EUPL-1.2+ -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> - -#include "../net-util.h" - -#include <assert.h> -#include <errno.h> -#include <net/if.h> -#include <string.h> - -int main(void) -{ - char name[IFNAMSIZ]; - int fd; - - memset(name, 'a', sizeof name); - fd = tap_open(name, 0); - assert(fd == -1); - assert(errno == ENAMETOOLONG); -} diff --git a/tools/start-vmm/tests/tap_open.c b/tools/start-vmm/tests/tap_open.c deleted file mode 100644 index bf5d00c..0000000 --- a/tools/start-vmm/tests/tap_open.c +++ /dev/null @@ -1,28 +0,0 @@ -// SPDX-License-Identifier: EUPL-1.2+ -// SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> - -#include "../net-util.h" - -#include <assert.h> -#include <errno.h> -#include <sched.h> -#include <string.h> - -#include <sys/ioctl.h> - -#include <linux/if_tun.h> - -int main(void) -{ - char name[IFNAMSIZ] = "tap%d"; - struct ifreq ifr; - int fd; - - unshare(CLONE_NEWUSER|CLONE_NEWNET); - - fd = tap_open(name, 0); - if (fd == -1 && (errno == EPERM || errno == ENOENT)) - return 77; - assert(!ioctl(fd, (unsigned)TUNGETIFF, &ifr)); - assert(!strcmp(name, ifr.ifr_name)); -} diff --git a/vm/sys/net/default.nix b/vm/sys/net/default.nix index fd5bf08..79ca0ad 100644 --- a/vm/sys/net/default.nix +++ b/vm/sys/net/default.nix @@ -7,7 +7,7 @@ pkgsMusl.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeClosure , erofs-utils, jq, s6-rc, util-linux, xorg -, busybox, execline, kmod, linux_latest, mdevd, nftables, xdp-tools +, busybox, execline, kmod, linux_latest, mdevd, nftables, xdp-tools, wpa_supplicant , s6, s6-linux-init }: @@ -52,7 +52,7 @@ let }); packages = [ - execline kmod mdevd s6 s6-linux-init s6-rc xdp-tools + execline kmod mdevd s6 s6-linux-init s6-rc xdp-tools wpa_supplicant (busybox.override { extraConfig = '' -- 2.51.2
On 11/24/25 11:35, Yureka Lilian wrote:
This removes the old host bridge + taps glue, and instead connects the apps to their net provider's router instance.
As mentioned in my reply to the last email, I'd like to see the old code kept as an option until the router gains IPv4 support. Right now, I have no way of testing this code except by setting up a VPN. -- Sincerely, Demi Marie Obenour (she/her/hers)
The code review of the router code is not helpful at this stage, as I hoped to convey in the cover letter. We should however address the IPv4 support question (see below). On 11/24/25 20:10, Demi Marie Obenour wrote:
On 11/24/25 11:35, Yureka Lilian wrote:
This removes the old host bridge + taps glue, and instead connects the apps to their net provider's router instance. As mentioned in my reply to the last email, I'd like to see the old code kept as an option until the router gains IPv4 support. Right now, I have no way of testing this code except by setting up a VPN.
You will be able to test the code using the integration tests once I have adapted them. You can also always test against a local IPv6 link-local target in your home network, or using pasta/passt emulation for testing handling of router advertisements. Supporting both the host network path and the spectrum router at the same time adds a lot of complexity in start-vmm because of how differently they function, and I wouldn't consider it an option. If IPv4 support is a requirement, I would propose adding rudimentary support for it to the router early in a similar fashion as IPv6 (with one IP per VM). Otherwise, I'm planning to add IPv4 support later with connection tracking and network address translation.
participants (4)
-
Alyssa Ross -
Demi Marie Obenour -
Yureka -
Yureka Lilian