Servers are public and Private messages are stored without any envryption. If you delete your account then the messages stay and can still be found with your unique ID (just like Reddit). From what ive read Discord also stores your HWIDs and monitors your running processes (with a valid reason considering their game integration). Some say they only store that locally, others claim something else, haven't seen any proof for either side so far.
The problem really boils down to the fact that people treat discord as a private messenger instead of a public forum despite it clearly beeing the latter.