There are a few ways around it. The simplest is to add the --privileged option.
The more secure method with podman is by specifying a user (ex -u 10001:10001) from your extended subuid:subgid range after your full and proper setup of rootless podman :-)
Then instead of chown you'll want to use the oddly named podman unshare tool to automatically set the permissions of the host directory. You would then want to start your service with systemctl --user instead of sudo systemctl