Multi-socketed virtual machine out of a cluster of independent systems.
37TS opened this issue · comments
Does anybody know if xcp-ng let you use multiple independent systems as one huge virtual machine?
quote "ScaleMP is the leader in virtualization for high-end computing, providing increased performance and reduced total cost of ownership (TCO). The innovative Versatile SMP™ (vSMP) architecture provides software-defined computing and software-defined memory by aggregating multiple independent systems or high-performance SSDs into single virtual systems"
ScaleMP vSMP can do just that but it's basically out of the reach of "normies".
I'd like to simulate a multi-socketed system with cheap hardware which doesn't scale nor supports multi-socket options, which is what ScaleMP excels at.
To repeat, does anybody know if this is even remotely possible with XCP-ng?
Sorry for the redundancy in the sentences.
Hi,
No, it doesn't work like this. VM locality is for one physical host, you doesn't share resources with multiple hosts for one VM.
What are you searching is completely different that what XCP-ng offers. @Nambrok might now some software name for this (it's mostly used in HPC, if I remember correctly)
Thanks for the answer, extremely appreciated.
Basically, this model is impossible with XCP-ng, correct?
Not possible "only" with XCP-ng. You might have software "on top of it" that might emulate this behavior.
That concept requires also a lot of bandwith.
Intersocket links nowadays go between 25 and up to 250 GB/s - your network does 0.1, maybe 1.x GB/s usually. That does not include storage BW.
I don't know how ScaleMP works, but it's definitely a niche product, not suitable for the general public.
I'm totally against the "not suitable for the general public" way of thinking, it's a syllogism. Given the possibility, anyone can become more knowledgeable.
Nevertheless, PCIe and/or MIPI are good candidates for a bus on "cheap" hardware. After all, LAN is often communicating through PCIe.
I'm merely investigating the possibility. Maybe emulating chipsets like the C621 on the hosts' operating systems. Given known hardware topologies, these could interface on compatible interfaces, setting the logic required for communications.
I've seen some Tx-Tx and Tx-Rx cables, the ADT-link R33SS and R33NS, and I was wondering about ways to have hypervisors communicate on that.
EDIT: I was inspired by this and I was thinking about using a similar approach to let "non scalable" CPUs communicate, primarily on PCIe.
https://github.com/shodruky-rhyammer/gl-streaming
You don't get what I said, obviously.
It is not suitable, as the bandwith of common networks, people use, is way too slow (in speed and reaction times) to suit this.
What you will get is a potential slowdown as thread-syncs take too much time. And PCIe is not a bus, it's a port. PCI was a bus.
As you're pointing out that LAN is communicating through PCIe: I told the problem of the concept, and this argumentation makes no sense at all -> I'm out.
Look up the max. lenght of PCIe lanes... also here: good luck. (hint: look for Inifinband/RDMA)
Good luck!
Edit: You should try the XCP-ng forums or whatever to continue - if you think someone might be interested or able to help you. This is just a bugtracker.
@37TS I guess what you're looking for is rather a Single-system image cluster
"good luck" Understood, it's a bugtracker. Sorry.
These kinds of systems are complicated, I don't think it's in the scope of XCP-ng. They mostly exist in research and like @nagilum99 said, they are limited by network usage. Look at this one using QEMU/KVM GiantVM they found themselves limited with the shared memory model. You might find what you want in these. Good luck :)