If you understand SSH keys, it's basically the same thing made more general.
Whatever website (e.g. lemmy.world) has a copy of the public key, they encrypt something with the public key, you decrypt it, reencrypt it with your private key and send it back (where they can then decrypt it and verify what they got back is what they expected). By performing that round trip, you've verified you have the correct key, and the "door opens."
The net effect is you can prove who you are, without actually giving someone the ability to impersonate you. It's authentication via "secret steps only you would know" instead of authentication by a fixed "password" (that anyone who hears it can store and potentially use for their own purposes).
That's all wrapped up in an open protocol anyone can implement and use to provide a variety of (hopefully) user friendly implementations (like the one Proton made) 🙂