There is a lot more metadata than just avatars and reactions. Accounts and their room membership over time, timing of messages (and thus online times), individual interactions between specific users (based on the timing of their messages) and so on. That is all in the unencrypted metadata of a Matrix room and can't be moved to the encrypted message part like avatars and reactions.