On the surface, that looks like it should work (assuming all the keys are correct and 51820/udp is open to the world on your VPS).
Can you ping the VPS's WG IP from your homeserver and get a response? If so, try pinging back from the VPS after that.
Until you get the bidirectional traffic going, you might try pulling out the iptables rules from your wireguard script and bringing everything back up clean.