Michael Raskin <7c6f434c@mail.ru> writes:
Given that it is sometimes convenient to spawn multiple VMs with almost identical settings on the fly (e.g. Firefox instances but with different state directories for downloads/uploads), should we commit to define a CLI layer for starting VMs (probably similar to the underlying VMM software CLI, but adapted to the notions we want to use), with the Nix-based management of VM fleet built on top of this CLI?
Yeah, I've been thinking about this too and agree. We should have a nix run analogue for temporary VMs. It would be nice for scripts that use multiple VMs to be able to start them on the fly as well. (But we should definitely not have a nix-env -i analogue, because there should be no hidden state on a host.)>
I think it is better to avoid evaluation, actually.
So I want a level where there is a Spectrum Profile Generation with named prebuilt VMs complexity levels (console-only, for stuff like ImageMagick on images downloaded from 4chan; GUI-but-no-HW, to remove GPU attack surface and guarantee lack of sound; full-GUI, which is still not guaranteed to get access to everything, of course), and prebuilt application sets (just buildEnv stuff, no fancy stuff).
Then I say:
spectrum-vm --base full-gui --net nat --app firefox-esr-js-profile \ --app gvim --dri card0 --rw-directory user:conf/nixcon-2020 \ -- firefox https://nixos.org/
This should not do anything in run time that can be safely cached. VM is prebuilt, application bunches are prebuilt and just mounted into store, even Firefox profile is prebuilt and changes are going to be discarded. No builds, no Nix evaluations.
(yes, there is also the annoying question of RAM amount management, not sure how to do it well, ballooning would be the nicest thing if it is feasible)
I think I had imagined you building a "firefox" VM, but yeah, maybe there's really no point to that. In which case, I'd be happy for this sort of thing to be handled outside of Nix.
I suspect that building things the other way round is not much simpler, but building a single VM image usable with slightly different settings will have smaller overhead than evaluating a Nix expression for each extra VM.
Maybe. I'll have to think more about that. tbh I think we can assume that the evaluator will get faster at some point (because there's big potential for optimisations and caching in safe mode). It might be nice
Oh well, in the safe mode there is even a potential for precompiling Nix functions. It's another question whether we want to experiment with workflows without having access to such speedups.
And even a pure evaluation of a checkout needs to stat all the relevant files (even if we accept the risk of not checksumming them).
Another good point.
to invent yet another way to do options, and just keep that in Nix. But I could be swayed either way.
I assume there are good arguments for the config to be structured (and in my current system there are some interface boundaries with structured config and some wrapper scripts that manage to generate almost all the usecases using command-line arguments, but sometimes parts of the structure need to be passed to them).
On the other hand, JSON can be parsed and written easily in all languages we might want to use, even in Bash using jq. And we need some code to setup the namespaces, but nsjail/firejail/bubblewrap are C code so in the long term there would be some rewrite, probably in Rust??? And having Nix preprocess arguments for Rust code sounds strange.
Not sure about this -- I think it's something we'll have to experiment with. Maybe structured configuration isn't necessary -- it's a pain to do in a CLI...
And whatever you do, Nix evaluation is always just another layer before running the actual code for setting up the isolated environments that still needs to interpret its arguments.
Also, I might want to use some binary cache, but when I am offline Nix builds wait for a long time for a reply from a binary cache. It's not so bad if I am intentionally building something and can pass an empty value for binary-caches via the command line, but doing it for each new command I execute sounds excessive.
Yes, this is true, although I consider it a Nix bug. I shouldn't have to remember to say --option substitute false for trivial offline rebuilds.
Also, it might be convenient to pass entire Nix store inside some VMs, but if all the _environments_ of all the currently running VMs are visible in the Nix store??? I really shouldn't.
I guess if I mention multiuser systems you would say that we need to allow chmod a-x on /nix/store?
I don't plan on doing anything to accomodate multi-user systems, at least for now.