Had a longer answer that went away for some reason 😭
Your criticism is fair, but:
Using the email as a seed on the generator goes halfway there, here's an example, although of course it could never change
Centralizing the avatar opens that service up to be grabbed for tracking like Gravatar did, or breaking the security of a site by injecting code without the consent of the site owner. The usual tradeoff between security/privacy and convenience.
Would be good to have a good user-managed way to do the same (well, beyond using an image on your own hosting, i guess)