Obviously you need someone joining the room for the room metadata to be shared between homeservers. But that is really only a minor barrier and once that has happened the worst case scenario takes place immediately. On other messengers (federated or not) a newly joining member has very limited access to past room metadata. Not so with Matrix, where a joining homeserver get full retroactive access to all the room metadata since the room's creation. If you can't see the problem with that, you really need to stop privacy LARPing 🙄