At the risk of resurrecting a zombie post. I'll respond.
I'm not sure on the specifics of xcp-ng, since I haven't run it myself, but, I know proxmox and VMware can both do PCIe pass thru to VMs. Recently L1 techs have done videos on the Intel flex GPUs and their potential with vdi for video rendering (basically for a virtual GPU), which worked excellently. I'm not sure if there's a large feature gap between the a380 and the flex series, but I suspect not. Given the cost of an A380 it's probably worth the risk to try it. With all the recent updates for the Intel GPUs which have been increasing performance and stability, the a380 is a solid buy, even if it's "only" able to be passed through to the VM ...