Cannot build release.nix without KVM
On a Qubes OS VM (which doesn't have nested virtualization), building release.nix fails because start-vmm-test (rightly) errors out. Having KVM be a *build*-time dependency of Spectrum OS seems rather strict, especially because I can always copy the generated files over to another machine and run them there. In particular, I can copy the resulting image and run it in QEMU. Ideally I could use Nix's remote build support for this. However, that is designed for the case where I am offloading work to a powerful and trusted server, whereas in my case the remote machine is for testing purposes only. Therefore, to save money, it is cheap, and it is also generally treated as untrusted because I don't hold it to the same standards I hold my highly trusted main laptop to. For instance, it doesn't have credentials to send emails. Is it possible to copy what I need for the tests (using 'nix copy' or 'nix-copy-closure' perhaps?) and skip them locally? Can I run the tests that do not need KVM while still running the rest? This is probably simple for anyone familiar with Nix, but that is not me. -- Sincerely, Demi Marie Obenour (she/her/hers)
Demi Marie Obenour <demiobenour@gmail.com> writes:
On a Qubes OS VM (which doesn't have nested virtualization), building release.nix fails because start-vmm-test (rightly) errors out. Having KVM be a *build*-time dependency of Spectrum OS seems rather strict, especially because I can always copy the generated files over to another machine and run them there. In particular, I can copy the resulting image and run it in QEMU.
Ah, I've made a mistake here I think. KVM shouldn't be a build dependency, because the Scaleway aarch64 builders used to populate the binary cache don't have it. It's only recently that release.nix includes start-vmm-test, so I haven't run those builders since, and hadn't noticed it. One option would be to introduce a build configuration[1] option that just disables any tests that need KVM. Another would be to run those tests inside QEMU TCG if KVM is available, which would have the benefit of the tests still running (and not introducing more config options). I wonder what the most lightweight way to optionally run some code in a TCG VM would be… [1]: https://spectrum-os.org/doc/development/build-configuration.html
Ideally I could use Nix's remote build support for this. However, that is designed for the case where I am offloading work to a powerful and trusted server, whereas in my case the remote machine is for testing purposes only. Therefore, to save money, it is cheap, and it is also generally treated as untrusted because I don't hold it to the same standards I hold my highly trusted main laptop to. For instance, it doesn't have credentials to send emails.
Using a TCG VM as a remote builder would also be an option, but it would probably be nicer to have the build take care of that.
Is it possible to copy what I need for the tests (using 'nix copy' or 'nix-copy-closure' perhaps?) and skip them locally? Can I run the tests that do not need KVM while still running the rest? This is probably simple for anyone familiar with Nix, but that is not me.
You can pass --keep-going to your build command, and then any failures won't get in the way of the rest of the build. nix-output-monitor[2] gives nice output that makes it obvious what failed. [2]: https://github.com/maralorn/nix-output-monitor
Alyssa Ross <hi@alyssa.is> writes:
Demi Marie Obenour <demiobenour@gmail.com> writes:
On a Qubes OS VM (which doesn't have nested virtualization), building release.nix fails because start-vmm-test (rightly) errors out. Having KVM be a *build*-time dependency of Spectrum OS seems rather strict, especially because I can always copy the generated files over to another machine and run them there. In particular, I can copy the resulting image and run it in QEMU.
Ah, I've made a mistake here I think. KVM shouldn't be a build dependency, because the Scaleway aarch64 builders used to populate the binary cache don't have it. It's only recently that release.nix includes start-vmm-test, so I haven't run those builders since, and hadn't noticed it.
One option would be to introduce a build configuration[1] option that just disables any tests that need KVM. Another would be to run those tests inside QEMU TCG if KVM is available, which would have the benefit of the tests still running (and not introducing more config options). I wonder what the most lightweight way to optionally run some code in a TCG VM would be…
[1]: https://spectrum-os.org/doc/development/build-configuration.html
Sadly the obvious solution of running it under QEMU user emulation doesn't work… https://gitlab.com/qemu-project/qemu/-/issues/871
Alyssa Ross <hi@alyssa.is> writes:
Alyssa Ross <hi@alyssa.is> writes:
Demi Marie Obenour <demiobenour@gmail.com> writes:
On a Qubes OS VM (which doesn't have nested virtualization), building release.nix fails because start-vmm-test (rightly) errors out. Having KVM be a *build*-time dependency of Spectrum OS seems rather strict, especially because I can always copy the generated files over to another machine and run them there. In particular, I can copy the resulting image and run it in QEMU.
Ah, I've made a mistake here I think. KVM shouldn't be a build dependency, because the Scaleway aarch64 builders used to populate the binary cache don't have it. It's only recently that release.nix includes start-vmm-test, so I haven't run those builders since, and hadn't noticed it.
One option would be to introduce a build configuration[1] option that just disables any tests that need KVM. Another would be to run those tests inside QEMU TCG if KVM is available, which would have the benefit of the tests still running (and not introducing more config options). I wonder what the most lightweight way to optionally run some code in a TCG VM would be…
[1]: https://spectrum-os.org/doc/development/build-configuration.html
Sadly the obvious solution of running it under QEMU user emulation doesn't work…
… and even if it did, QEMU user emulation doesn't implement the KVM API. Anyway, I've come to the conclusion that since start-vmm-test isn't testing anything that isn't covered by the integration tests we recently gained (which run a full Spectrum system in QEMU, and can therefore use TCG), it can just be removed, and I've now done so. I think that means that everything can now be built without KVM, so the problem is solved for now. (Let me know if not.)
participants (2)
-
Alyssa Ross -
Demi Marie Obenour