no, it isn't bidirectional, public = encrypt, private = decrypt, that's it. You can address a message to multiple recipients though (when using GPG), so often in case of email a message is addressed both to yourself and your recipient, so both you and your recipient have access to message text