That would be the case if it wasn't a flagrant violation like, "These matrix rooms are spying on these chat rooms without announcing the users in the matrix room", and Matrix refusing to fix it when informed.
Imagine if Matrix did that to their user base. "We'll occasionally drop in to listen, or pipe your conversation to other people, but you won't see the, "x joined", notice. Cheers!" Creepy spying, the polar opposite of privacy.