Immutable is fantastic in theory. Where it falls apart is having to basically rebuild the whole distro every time you want to make a change. It should be there your base distro is immutable, then any extra changes go on an additional mutable layer but that would be difficult to set up. (You’d need a package manager like Nixos or something.)
your base distro is immutable, then any extra changes go on an additional mutable layer
That is exactly how OsTree and other layering solutions work. Only Nix requires a whole distro rebuild.
It isn’t, though. Package layering modifies the install itself. See: https://docs.fedoraproject.org/en-US/fedora-silverblue/getting-started/#_flatpak_command_line
The big problem with the way ostree works is that installing things has side effects. Every item you install with ostree makes all future items slower to install, including regular os updates. This is a significant flaw in the way they designed it and really makes immutable oses less attractive.
It’s not a flaw. Ostree is a last resort, you should be using containerized software. Layering a package should only be done when strictly necessary and not as the regular way to manage packages. If you need an overtly customized system, you use Nix or universal blue to design your new system declaratively and create your custom image.
