It's not "inherently insecure" at least not to that degree. (Once could argue that lack of E2EE is insecure.) If you stand up an unrelated instance you shouldn't be able to access private messages that don't relate to an account on your instance. So only bugs in your instance, or your conversation partner's instance, will be able to leak those messages.