According to how the UK's Matrix/Element "privacy" messager app acts, that is correct. If, for example, you request a GDPR compliant data deletion of your messages in a room that contains 100 people, they will continue storing your data and delivering it to those 100 people, as well as propagating your data across any other servers where those people may be.
If you've lost access to any of those rooms, screw you, your data doesn't belong to you but it does belong to anybody who was there at the time.