[PATCH 0/3] Minor improvements to the Spectrum OS boot process
These are all very minor changes. Patch 1 switches from Busybox switch_root to the util-linux version. Since util-linux moves /dev, /sys, and /proc to the root filesystem, three calls to mount go away. Patch 2 adds /dev/fd, /dev/stdin, /dev/stdout, and /dev/stderr symlinks, to avoid compatibility problems with Spectrum OS's third-party dependencies. Patch 3 avoids mounting /proc and /sys in the main system, as the initramfs already did this. Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- Demi Marie Obenour (3): host/initramfs: Use util-linux switch_root Add /dev/fd and /dev/std* host/rootfs: Avoid redundant mounts of /proc and /sys host/initramfs/default.nix | 3 ++- host/initramfs/etc/init | 4 ---- host/rootfs/etc/fstab | 2 -- host/rootfs/etc/init | 8 ++++++++ vm/sys/net/etc/init | 8 ++++++++ 5 files changed, 18 insertions(+), 7 deletions(-) --- base-commit: f14abeb3db2c1c47658fa1809f3fde379e41d632 change-id: 20250914-util-linux-switch-root-10034b149e45 -- Sincerely, Demi Marie Obenour (she/her/hers)
Busybox switch_root doesn't move /proc, /dev, and /sys mounts, whereas the switch_root from util-linux does. Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/initramfs/default.nix | 3 ++- host/initramfs/etc/init | 4 ---- 2 files changed, 2 insertions(+), 5 deletions(-) diff --git a/host/initramfs/default.nix b/host/initramfs/default.nix index 1123feacc7533abaf079d84b04a269f784eb43b5..28378da220bb61a51e58b8b25846097f08746f5f 100644 --- a/host/initramfs/default.nix +++ b/host/initramfs/default.nix @@ -43,6 +43,7 @@ let CONFIG_MODINFO n CONFIG_MODPROBE n CONFIG_RMMOD n + CONFIG_SWITCH_ROOT n ''; }) ]; @@ -56,7 +57,7 @@ let # TODO: this is a hack and we should just build the util-linux # programs we want. # https://lore.kernel.org/util-linux/87zgrl6ufb.fsf@alyssa.is/ - cp ${util-linuxMinimal}/bin/{findfs,lsblk} $out/bin + cp ${util-linuxMinimal}/bin/{findfs,lsblk,switch_root} $out/bin ''; microcode = runCommand "microcode.cpio" { diff --git a/host/initramfs/etc/init b/host/initramfs/etc/init index 719488741b6d31564c2c17c0e41f15d16b1c0a08..b3c5b7fbcf45e9646cf0404bc44dac58aa2dfe24 100755 --- a/host/initramfs/etc/init +++ b/host/initramfs/etc/init @@ -45,9 +45,5 @@ background { rm /dev/rootfs /dev/verity } if { mount /dev/mapper/root-verity /mnt/root } wait { $mdevd_pid } -if { mount --move /proc /mnt/root/proc } -if { mount --move /sys /mnt/root/sys } -if { mount --move /dev /mnt/root/dev } - switch_root /mnt/root /etc/init -- 2.51.0
Demi Marie Obenour <demiobenour@gmail.com> writes:
Busybox switch_root doesn't move /proc, /dev, and /sys mounts, whereas the switch_root from util-linux does.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/initramfs/default.nix | 3 ++- host/initramfs/etc/init | 4 ---- 2 files changed, 2 insertions(+), 5 deletions(-)
So on the whole, I'm in favour of getting rid of Busybox wherever possible, because its implementations tend to be bad. In the case of initramfs though, the potential for unexpected circumstances is very constrained, and we need something to implement mkfifo, head, kill, and rm. So given we're using Busybox for that anyway, we might as well use Busybox's smaller implementation of switch_root, right?
diff --git a/host/initramfs/default.nix b/host/initramfs/default.nix index 1123feacc7533abaf079d84b04a269f784eb43b5..28378da220bb61a51e58b8b25846097f08746f5f 100644 --- a/host/initramfs/default.nix +++ b/host/initramfs/default.nix @@ -43,6 +43,7 @@ let CONFIG_MODINFO n CONFIG_MODPROBE n CONFIG_RMMOD n + CONFIG_SWITCH_ROOT n ''; }) ]; @@ -56,7 +57,7 @@ let # TODO: this is a hack and we should just build the util-linux # programs we want. # https://lore.kernel.org/util-linux/87zgrl6ufb.fsf@alyssa.is/ - cp ${util-linuxMinimal}/bin/{findfs,lsblk} $out/bin + cp ${util-linuxMinimal}/bin/{findfs,lsblk,switch_root} $out/bin '';
microcode = runCommand "microcode.cpio" { diff --git a/host/initramfs/etc/init b/host/initramfs/etc/init index 719488741b6d31564c2c17c0e41f15d16b1c0a08..b3c5b7fbcf45e9646cf0404bc44dac58aa2dfe24 100755 --- a/host/initramfs/etc/init +++ b/host/initramfs/etc/init @@ -45,9 +45,5 @@ background { rm /dev/rootfs /dev/verity } if { mount /dev/mapper/root-verity /mnt/root } wait { $mdevd_pid }
-if { mount --move /proc /mnt/root/proc } -if { mount --move /sys /mnt/root/sys } -if { mount --move /dev /mnt/root/dev } - switch_root /mnt/root /etc/init
-- 2.51.0
On 9/17/25 07:44, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
Busybox switch_root doesn't move /proc, /dev, and /sys mounts, whereas the switch_root from util-linux does.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/initramfs/default.nix | 3 ++- host/initramfs/etc/init | 4 ---- 2 files changed, 2 insertions(+), 5 deletions(-)
So on the whole, I'm in favour of getting rid of Busybox wherever possible, because its implementations tend to be bad. In the case of initramfs though, the potential for unexpected circumstances is very constrained, and we need something to implement mkfifo, head, kill, and rm. So given we're using Busybox for that anyway, we might as well use Busybox's smaller implementation of switch_root, right?
I'd prefer to use util_linux mount and switch_root, and Toybox mkfifo, head, kill, and rm. That said, Busybox can *work*. It's just annoying. -- Sincerely, Demi Marie Obenour (she/her/hers)
Demi Marie Obenour <demiobenour@gmail.com> writes:
On 9/17/25 07:44, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
Busybox switch_root doesn't move /proc, /dev, and /sys mounts, whereas the switch_root from util-linux does.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/initramfs/default.nix | 3 ++- host/initramfs/etc/init | 4 ---- 2 files changed, 2 insertions(+), 5 deletions(-)
So on the whole, I'm in favour of getting rid of Busybox wherever possible, because its implementations tend to be bad. In the case of initramfs though, the potential for unexpected circumstances is very constrained, and we need something to implement mkfifo, head, kill, and rm. So given we're using Busybox for that anyway, we might as well use Busybox's smaller implementation of switch_root, right?
I'd prefer to use util_linux mount and switch_root, and Toybox mkfifo, head, kill, and rm. That said, Busybox can *work*. It's just annoying.
toybox has no awk, so we'd probably want to come up with some replacement for /etc/getuuids. Aside from that, I'd be happy to be rid of Busybox.
On 9/17/25 07:44, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
Busybox switch_root doesn't move /proc, /dev, and /sys mounts, whereas the switch_root from util-linux does.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/initramfs/default.nix | 3 ++- host/initramfs/etc/init | 4 ---- 2 files changed, 2 insertions(+), 5 deletions(-)
So on the whole, I'm in favour of getting rid of Busybox wherever possible, because its implementations tend to be bad. In the case of initramfs though, the potential for unexpected circumstances is very constrained, and we need something to implement mkfifo, head, kill, and rm. So given we're using Busybox for that anyway, we might as well use Busybox's smaller implementation of switch_root, right?
I think so, unless we care about eliminating calls to /usr/bin/if and /usr/bin/mount. -- Sincerely, Demi Marie Obenour (she/her/hers)
This is nearly same as 14483e1a690c (img/app: add /dev/fd and /dev/std*), but for the host and for vm/sys/net. The only difference is that the symlinks are created in init, rather than in rc.init. This ensures that nothing in the main system can run before they the links are created. While only Spectrum-provided code should run in these VMs, third-party dependencies are likely to only be tested in an environment where these links are present. Since the BSDs also have these links, some code might even use them on purpose. Not having these links could cause severe bugs. For instance, if /dev/stdout or /dev/stderr is missing, writing to them will create a regular file instead. This could cause other code to misbehave. Furthermore, /dev is a devtmpfs, so this could consume a lot of memory. In the host, the links are added in the main system, rather than in the initramfs. This is per Alyssa's suggestion in [1]. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/87h5xdjo9p.fsf@alyssa.... Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/etc/init | 8 ++++++++ vm/sys/net/etc/init | 8 ++++++++ 2 files changed, 16 insertions(+) diff --git a/host/rootfs/etc/init b/host/rootfs/etc/init index 4085fa55545e7309004967e443e47fc2b82b0663..b6e6122ddb987a5463c0924cd56e1a26cf5bfd98 100755 --- a/host/rootfs/etc/init +++ b/host/rootfs/etc/init @@ -1,5 +1,13 @@ #!/bin/execlineb -s0 # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> +# Symlink creation (if it's copyrightable): +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com> + +/usr/bin/if { /usr/bin/ln -s /proc/self/fd /dev } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/0 /dev/stdin } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/1 /dev/stdout } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/2 /dev/stderr } /bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@ diff --git a/vm/sys/net/etc/init b/vm/sys/net/etc/init index 6424e221e0fb929e107271bd52b61706b9f1855e..de409e9296a24d42cef50605496efc520790559e 100755 --- a/vm/sys/net/etc/init +++ b/vm/sys/net/etc/init @@ -1,5 +1,13 @@ #!/bin/execlineb -s0 # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> +# Symlink creation (if it's copyrightable): +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com> + +/usr/bin/if { /usr/bin/ln -s /proc/self/fd /dev } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/0 /dev/stdin } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/1 /dev/stdout } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/2 /dev/stderr } /bin/s6-linux-init -Bc /etc/s6-linux-init -- $@ -- 2.51.0
Demi Marie Obenour <demiobenour@gmail.com> writes:
This is nearly same as 14483e1a690c (img/app: add /dev/fd and /dev/std*), but for the host and for vm/sys/net. The only difference is that the symlinks are created in init, rather than in rc.init. This ensures that nothing in the main system can run before they the links are created.
s6-linux-init is certainly designed to be run before these links exist, so I don't think we need them in init. I'd rather keep setup in a single place as much as possibl,e and that's rc.init.
While only Spectrum-provided code should run in these VMs, third-party dependencies are likely to only be tested in an environment where these links are present. Since the BSDs also have these links, some code might even use them on purpose.
Not having these links could cause severe bugs. For instance, if /dev/stdout or /dev/stderr is missing, writing to them will create a regular file instead. This could cause other code to misbehave. Furthermore, /dev is a devtmpfs, so this could consume a lot of memory.
In the host, the links are added in the main system, rather than in the initramfs. This is per Alyssa's suggestion in [1].
[1]: https://spectrum-os.org/lists/archives/spectrum-devel/87h5xdjo9p.fsf@alyssa....
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- host/rootfs/etc/init | 8 ++++++++ vm/sys/net/etc/init | 8 ++++++++ 2 files changed, 16 insertions(+)
diff --git a/host/rootfs/etc/init b/host/rootfs/etc/init index 4085fa55545e7309004967e443e47fc2b82b0663..b6e6122ddb987a5463c0924cd56e1a26cf5bfd98 100755 --- a/host/rootfs/etc/init +++ b/host/rootfs/etc/init @@ -1,5 +1,13 @@ #!/bin/execlineb -s0 # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> +# Symlink creation (if it's copyrightable): +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com> + +/usr/bin/if { /usr/bin/ln -s /proc/self/fd /dev } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/0 /dev/stdin } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/1 /dev/stdout } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/2 /dev/stderr }
/bin/s6-linux-init -c /etc/s6-linux-init -s /run/param -- $@ diff --git a/vm/sys/net/etc/init b/vm/sys/net/etc/init index 6424e221e0fb929e107271bd52b61706b9f1855e..de409e9296a24d42cef50605496efc520790559e 100755 --- a/vm/sys/net/etc/init +++ b/vm/sys/net/etc/init @@ -1,5 +1,13 @@ #!/bin/execlineb -s0 # SPDX-License-Identifier: EUPL-1.2+ # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> +# Symlink creation (if it's copyrightable): +# SPDX-License-Identifier: CC0-1.0 +# SPDX-FileCopyrightText: 2025 Demi Marie Obenour <demiobenour@gmail.com> + +/usr/bin/if { /usr/bin/ln -s /proc/self/fd /dev } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/0 /dev/stdin } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/1 /dev/stdout } +/usr/bin/if { /usr/bin/ln -s /proc/self/fd/2 /dev/stderr }
/bin/s6-linux-init -Bc /etc/s6-linux-init -- $@
-- 2.51.0
The initramfs already mounted both of them. Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- It's true that this does increase the reliance on an initramfs, but it is also trivial to revert if Spectrum OS ever stops using one. Linux supports creating dm-verity devices via dm-mod.create= and (if needed) dm-mod.waitfor=, so eliminating an initramfs might be possible but is a separate project. --- host/rootfs/etc/fstab | 2 -- 1 file changed, 2 deletions(-) diff --git a/host/rootfs/etc/fstab b/host/rootfs/etc/fstab index 6a82ecc85090a37b13603b29f74ca6e554a28c33..4d303d277409279976bc102969d902466bc39c47 100644 --- a/host/rootfs/etc/fstab +++ b/host/rootfs/etc/fstab @@ -1,6 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 # SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> -proc /proc proc defaults 0 0 devpts /dev/pts devpts defaults,gid=4,mode=620 0 0 tmpfs /dev/shm tmpfs defaults 0 0 -sysfs /sys sysfs defaults 0 0 -- 2.51.0
Demi Marie Obenour <demiobenour@gmail.com> writes:
The initramfs already mounted both of them.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- It's true that this does increase the reliance on an initramfs, but it is also trivial to revert if Spectrum OS ever stops using one. Linux supports creating dm-verity devices via dm-mod.create= and (if needed) dm-mod.waitfor=, so eliminating an initramfs might be possible but is a separate project.
I'd still prefer we don't rely on initramfs behaviour any more than we need to — it just makes things more brittle and difficult to understand. IIRC I tried using those kernel parameters before, but they weren't workable for what we needed. I don't remember the details.
--- host/rootfs/etc/fstab | 2 -- 1 file changed, 2 deletions(-)
diff --git a/host/rootfs/etc/fstab b/host/rootfs/etc/fstab index 6a82ecc85090a37b13603b29f74ca6e554a28c33..4d303d277409279976bc102969d902466bc39c47 100644 --- a/host/rootfs/etc/fstab +++ b/host/rootfs/etc/fstab @@ -1,6 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 # SPDX-FileCopyrightText: 2020-2021 Alyssa Ross <hi@alyssa.is> -proc /proc proc defaults 0 0 devpts /dev/pts devpts defaults,gid=4,mode=620 0 0 tmpfs /dev/shm tmpfs defaults 0 0 -sysfs /sys sysfs defaults 0 0
-- 2.51.0
On 9/17/25 07:31, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
The initramfs already mounted both of them.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- It's true that this does increase the reliance on an initramfs, but it is also trivial to revert if Spectrum OS ever stops using one. Linux supports creating dm-verity devices via dm-mod.create= and (if needed) dm-mod.waitfor=, so eliminating an initramfs might be possible but is a separate project.
I'd still prefer we don't rely on initramfs behaviour any more than we need to — it just makes things more brittle and difficult to understand.
Does the mount we are using blindly mount a second instance of procfs and sysfs? If so, I'd rather reuse the initramfs versions. Otherwise, keeping things as-is is fine.
IIRC I tried using those kernel parameters before, but they weren't workable for what we needed. I don't remember the details.
I suspect they are meant for embedded systems with known hardware. Ideally partition table parsing would be in userspace anyway. -- Sincerely, Demi Marie Obenour (she/her/hers)
Demi Marie Obenour <demiobenour@gmail.com> writes:
On 9/17/25 07:31, Alyssa Ross wrote:
Demi Marie Obenour <demiobenour@gmail.com> writes:
The initramfs already mounted both of them.
Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- It's true that this does increase the reliance on an initramfs, but it is also trivial to revert if Spectrum OS ever stops using one. Linux supports creating dm-verity devices via dm-mod.create= and (if needed) dm-mod.waitfor=, so eliminating an initramfs might be possible but is a separate project.
I'd still prefer we don't rely on initramfs behaviour any more than we need to — it just makes things more brittle and difficult to understand.
Does the mount we are using blindly mount a second instance of procfs and sysfs? If so, I'd rather reuse the initramfs versions. Otherwise, keeping things as-is is fine.
I don't see a second instance on edd53bb after a normal boot, whereas I do if I manually mount -t proc proc /proc.
This is nearly same as 14483e1a690c (img/app: add /dev/fd and /dev/std*), but for the host and for vm/sys/net. The only difference is that the symlinks are created in init, rather than in rc.init. This ensures that nothing in the main system can run before they the links are created. While only Spectrum-provided code should run in these VMs, third-party dependencies are likely to only be tested in an environment where these links are present. Since the BSDs also have these links, some code might even use them on purpose. Not having these links could cause severe bugs. For instance, if /dev/stdout or /dev/stderr is missing, writing to them will create a regular file instead. This could cause other code to misbehave. Furthermore, /dev is a devtmpfs, so this could consume a lot of memory. [1]: https://spectrum-os.org/lists/archives/spectrum-devel/87h5xdjo9p.fsf@alyssa.... Signed-off-by: Demi Marie Obenour <demiobenour@gmail.com> --- Changes in v2: - Do not switch to util-linux switch_root as Busybox switch_root is good enough. - Leave /proc and /sys in /etc/fstab as they will not be remounted if already mounted. - Link to v1: https://spectrum-os.org/lists/archives/spectrum-devel/20250914-util-linux-sw... --- host/rootfs/image/etc/s6-linux-init/scripts/rc.init | 5 +++++ vm/sys/net/image/etc/s6-linux-init/scripts/rc.init | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init index 674fd38cc76837c7be25a5ef060f0f4d4b786394..46e5cd6d1b99c12fab84a2797b37d9f4eccaa56a 100755 --- a/host/rootfs/image/etc/s6-linux-init/scripts/rc.init +++ b/host/rootfs/image/etc/s6-linux-init/scripts/rc.init @@ -4,6 +4,11 @@ if { s6-rc-init -c /etc/s6-rc /run/service } +if { ln -s /proc/self/fd /dev } +if { ln -s /proc/self/fd/0 /dev/stdin } +if { ln -s /proc/self/fd/1 /dev/stdout } +if { ln -s /proc/self/fd/2 /dev/stderr } + if { mount --make-shared /run } if { mount -a --mkdir } diff --git a/vm/sys/net/image/etc/s6-linux-init/scripts/rc.init b/vm/sys/net/image/etc/s6-linux-init/scripts/rc.init index 1016d0c62bc6103bc9e865a389f5d482ef6c2b76..4378400b0b34f12bfa6a994029546f9c38058d10 100755 --- a/vm/sys/net/image/etc/s6-linux-init/scripts/rc.init +++ b/vm/sys/net/image/etc/s6-linux-init/scripts/rc.init @@ -4,6 +4,11 @@ if { s6-rc-init -c /etc/s6-rc /run/service } +if { ln -s /proc/self/fd /dev } +if { ln -s /proc/self/fd/0 /dev/stdin } +if { ln -s /proc/self/fd/1 /dev/stdout } +if { ln -s /proc/self/fd/2 /dev/stderr } + if { mkdir -p /dev/pts /dev/shm } if { mount -a } --- base-commit: edd53bb5dd9b682fec744d6df7921b64e0f56565 change-id: 20250914-util-linux-switch-root-10034b149e45 -- Sincerely, Demi Marie Obenour (she/her/hers)
This patch has been committed as b319a21dd023e23e2dddb1bbfcc18202970faae2, which can be viewed online at https://spectrum-os.org/git/spectrum/commit/?id=b319a21dd023e23e2dddb1bbfcc1.... This is an automated message. Send comments/questions/requests to: Alyssa Ross <hi@alyssa.is>
participants (3)
-
Alyssa Ross -
Alyssa Ross -
Demi Marie Obenour