I managed this by using tailscale, with a kind of weird setup I think, but it just works.
I have tailscale on the VPS and my local server, let's say its tailscale name is potatoserver
Then with Caddy on the VPS i have something like:
mywebsite.com {
reverse_proxy potatoserver:port
}
And so mywebsite.com is accessible on the clearnet through the VPS
Though given you're getting rid of cloudflare tunnles I don't know if you'd want to get into Tailscale. There's Headscale too but I haven't worked with it so I can't comment