WoL packets are usually sent to the ip broadcast address for the network as they’re not ip based. I don’t know if this would ever work well across networks. Can you do send the wol packet from the opnsense router instead? Does it work then?
If you’re sending it to the IP of the server, it likely works soon after your turn the machine off because the ARP entry hasn’t timed out yet, but once it times out it won’t work anymore. The router doesn’t know how to get to the machine. You may be able to add a static arp mapping to get it to work long term.