Overriding Linux kernel in host/rootfs/default.nix
Hello everyone. Recently I faced an issue with overriding Linux kernel in rootfs recipe: Rootfs has an attribute kernel in the `let ... in` section, which is used for setting up correct paths to kernel modules in rootfs derivation. But this attribute is also used by the derivation `packagesSysroot` which is used by the derivation `packagesTar`, whiсh, in its turn, used by rootfs derivation. All that derivations introduced in the scope of rootfs derivation (the same `let ... in` section) and are anonymous. What happens when I override kernel attribute of the rootfs derivation: 1. Nix evaluates original host/rootfs/default.nix file and replaces all ${kernel} occurences with the actual value of kernel attribute (which is linux_latest at this moment). 2. Nix sets linux_latest as the input for rootfs and packagesSysroot derivations. 3. Nix evaluates overlay with kernel override and overrides (in fact, introduces new) kernel attribute. 4. Nothing happens after that. Now we have: - `kernel` attribute of rootfs derivation pointing to the imx8 kernel. - `MODULES_ALIAS` and `MODULES_ORDER` variables of rootfs derivation are pointing to linux_latest modules. The root of the issue - line `ln -s ${kernel}/lib/modules ${firmware}/lib/firmware $out/lib` from `packagesSysroot` derivation build command, which creates the /lib link in the rootfs pointing to linux_latest modules. As I found, kernel attribure is not even passed as an attribute to the rootfs derivation file. But it is possible to override MODULES_ALIAS and MODULES_ORDER with correct kernel modules path instead. This also touches initramfs The possible solution is just to override linux_latest, but in this case all VMs will use host kernel version, and we need to avoid this. I would propose something like this for rootfs' default.nix, because this approach solves all kernel issues in an easy and clear way: diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index ad6ea1f..7bf16d2 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage ( { lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, kernel ? linux_latest }: let @@ -73,8 +73,6 @@ let imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; }); - kernel = pkgs.linux_latest; - appvm = import ../../img/app { inherit config; inherit (foot) terminfo; I played a bit with this, and seems that it worked. Thanks.
On Wed, Dec 21, 2022 at 01:18:36PM +0200, Ivan Nikolaenko wrote:
Hello everyone.
Recently I faced an issue with overriding Linux kernel in rootfs recipe:
Rootfs has an attribute kernel in the `let ... in` section, which is used for setting up correct paths to kernel modules in rootfs derivation. But this attribute is also used by the derivation `packagesSysroot` which is used by the derivation `packagesTar`, whiсh, in its turn, used by rootfs derivation.
All that derivations introduced in the scope of rootfs derivation (the same `let ... in` section) and are anonymous. What happens when I override kernel attribute of the rootfs derivation: 1. Nix evaluates original host/rootfs/default.nix file and replaces all ${kernel} occurences with the actual value of kernel attribute (which is linux_latest at this moment). 2. Nix sets linux_latest as the input for rootfs and packagesSysroot derivations. 3. Nix evaluates overlay with kernel override and overrides (in fact, introduces new) kernel attribute. 4. Nothing happens after that. Now we have: - `kernel` attribute of rootfs derivation pointing to the imx8 kernel. - `MODULES_ALIAS` and `MODULES_ORDER` variables of rootfs derivation are pointing to linux_latest modules. The root of the issue - line `ln -s ${kernel}/lib/modules ${firmware}/lib/firmware $out/lib` from `packagesSysroot` derivation build command, which creates the /lib link in the rootfs pointing to linux_latest modules. As I found, kernel attribure is not even passed as an attribute to the rootfs derivation file. But it is possible to override MODULES_ALIAS and MODULES_ORDER with correct kernel modules path instead.
This also touches initramfs The possible solution is just to override linux_latest, but in this case all VMs will use host kernel version, and we need to avoid this.
I would propose something like this for rootfs' default.nix, because this approach solves all kernel issues in an easy and clear way:
diff --git a/host/rootfs/default.nix b/host/rootfs/default.nix index ad6ea1f..7bf16d2 100644 --- a/host/rootfs/default.nix +++ b/host/rootfs/default.nix @@ -7,7 +7,7 @@ pkgs.pkgsStatic.callPackage (
{ lib, stdenvNoCC, nixos, runCommand, writeReferencesToFile, s6-rc, tar2ext4 , busybox, cloud-hypervisor, cryptsetup, execline, jq, kmod -, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg +, mdevd, s6, s6-linux-init, socat, util-linuxMinimal, xorg, kernel ? linux_latest }:
let @@ -73,8 +73,6 @@ let imports = [ (modulesPath + "/profiles/all-hardware.nix") ]; });
- kernel = pkgs.linux_latest; - appvm = import ../../img/app { inherit config; inherit (foot) terminfo;
I played a bit with this, and seems that it worked.
Thanks.
Seems reasonable! Happy to apply if you resend with a Signed-off-by. Although I think it needs to still be pkgs.linux_latest, as linux_latest isn't defined. (We don't want to take it as a callPackage input because we don't need pkgsStatic.linux_latest.) And it might be a good idea to put the "kernel" option on a new line, so that overridable options are different from package inputs.
participants (2)
-
Alyssa Ross -
Ivan Nikolaenko