This actually happened to me: I arrived at my destination and discovered my load was loose, ready to fall. There have also been times that I dropped something. And times that my backpack was mistakenly unzipped and I could have lost something worth keeping.
So if I operate with your assumption (that honking drivers are always assholes), then I lose the opportunity to pick up something I dropped or correct insecure cargo. Why should I give that up?
Since a horn is an ambiguous signal, in this circumstance of a car following a cyclist it should come to be universally understood to mean a cyclist dropped their phone or wallet, as this is the legit scenario.