+State directories and applications will be <var>m</var>:<var>n</var>.
Note (probably not affecting the design right now): this can also be a source of annoyance with running everything under unique single-use UIDs: to have multiple UIDs access the same resource, setting ACLs is a natural idea. It works well, but might require some cleanup of obsolete ACLs for a workflow with a large churn of instances ??? the maximum length of an ACL is limited.
Yeah, we'll need something like that. ACLs sound sensible.
Of course, this is a persistent state related to a large number of inodes that requires keeping track of used unique UIDs (to avoid reuse before cleanup) and cleanup (I have learned it hte hard way: one does run out of ACL capacity without removing the obsolete ones), including some kind of cleanup in case of VM crash or unclean host reboot.
Given that it is sometimes convenient to spawn multiple VMs with almost identical settings on the fly (e.g. Firefox instances but with different state directories for downloads/uploads), should we commit to define a CLI layer for starting VMs (probably similar to the underlying VMM software CLI, but adapted to the notions we want to use), with the Nix-based management of VM fleet built on top of this CLI?
Yeah, I've been thinking about this too and agree. We should have a nix run analogue for temporary VMs. It would be nice for scripts that use multiple VMs to be able to start them on the fly as well. (But we should definitely not have a nix-env -i analogue, because there should be no hidden state on a host.)>
I think it is better to avoid evaluation, actually. So I want a level where there is a Spectrum Profile Generation with named prebuilt VMs complexity levels (console-only, for stuff like ImageMagick on images downloaded from 4chan; GUI-but-no-HW, to remove GPU attack surface and guarantee lack of sound; full-GUI, which is still not guaranteed to get access to everything, of course), and prebuilt application sets (just buildEnv stuff, no fancy stuff). Then I say: spectrum-vm --base full-gui --net nat --app firefox-esr-js-profile \ --app gvim --dri card0 --rw-directory user:conf/nixcon-2020 \ -- firefox https://nixos.org/ This should not do anything in run time that can be safely cached. VM is prebuilt, application bunches are prebuilt and just mounted into store, even Firefox profile is prebuilt and changes are going to be discarded. No builds, no Nix evaluations. (yes, there is also the annoying question of RAM amount management, not sure how to do it well, ballooning would be the nicest thing if it is feasible)
I suspect that building things the other way round is not much simpler, but building a single VM image usable with slightly different settings will have smaller overhead than evaluating a Nix expression for each extra VM.
Maybe. I'll have to think more about that. tbh I think we can assume that the evaluator will get faster at some point (because there's big potential for optimisations and caching in safe mode). It might be nice
Oh well, in the safe mode there is even a potential for precompiling Nix functions. It's another question whether we want to experiment with workflows without having access to such speedups. And even a pure evaluation of a checkout needs to stat all the relevant files (even if we accept the risk of not checksumming them).
to invent yet another way to do options, and just keep that in Nix. But I could be swayed either way.
I assume there are good arguments for the config to be structured (and in my current system there are some interface boundaries with structured config and some wrapper scripts that manage to generate almost all the usecases using command-line arguments, but sometimes parts of the structure need to be passed to them). On the other hand, JSON can be parsed and written easily in all languages we might want to use, even in Bash using jq. And we need some code to setup the namespaces, but nsjail/firejail/bubblewrap are C code so in the long term there would be some rewrite, probably in Rust��� And having Nix preprocess arguments for Rust code sounds strange. And whatever you do, Nix evaluation is always just another layer before running the actual code for setting up the isolated environments that still needs to interpret its arguments. Also, I might want to use some binary cache, but when I am offline Nix builds wait for a long time for a reply from a binary cache. It's not so bad if I am intentionally building something and can pass an empty value for binary-caches via the command line, but doing it for each new command I execute sounds excessive. Also, it might be convenient to pass entire Nix store inside some VMs, but if all the _environments_ of all the currently running VMs are visible in the Nix store��� I really shouldn't. I guess if I mention multiuser systems you would say that we need to allow chmod a-x on /nix/store?