Based on their claims Siri also works primarily on-device. It wasn't entirely clear if you can manually prevent the usage of their AI infrastructure, but they definitely implied it. So if that's true, there's no real reason to avoid just Siri while still using other AI stuff, cause they are one and the same. And since it runs locally, they can't even store the voice clips.