For years I’ve had a dream of building a rack mounted PC capable of splitting its resources to host multiple GPU intensive VMs:
- a few gaming VMs
- a VM for work that can run Davinci Resolve and Blender renders
- an LLM server
- a Stable Diffusion server
- media server
Just to name a few possibilities…
Everytime I’ve looked into it, it seemed like the technology just wasn’t there yet. I remember a few years ago Linus TT took a shot at it, but in the end suggested the technology (for non-commercial entities) just wasn’t in a comfortable spot yet.
So how far off are we? Obviously AI focused companies seem to make it work, but what possibilities exist for us self-hosters who might also want to run multiple displays in addition to the web gui LLM servers? And without forking out crazy money for GPU virtualization software licenses?
Hey, sorry I didn’t reply until now but life has been pretty hectic and I also kinda borked my streaming VM right at the same time as I wrote that. I ran Nobara Linux for a while with KDE on Xorg and it actually worked pretty well. Then I decided I wanted to give Bazzite a try but I didn’t like the whole immutable thing. I went back to Nobara just to find that Steam Remote Play straight up didn’t work and I couldn’t know if I had failed to set up something properly or Valve just broke it while I was “away”. A couple of days ago I decided to just abandon Remote Play for the time being and deployed Games on Whales and it seems very promising so far. Much easier than fiddling with VM:s and GPU passthrough and Sunshine/Moonlight has never failed me.
