IMHO a RasPi is just not reliable enough. Your internet connection is just not reliable enough.
You are going to lose some of your incoming mail and NOT notice it, unless you have somebody who hosts a secondary MX for your domain.
Chances are also that it's not powerful enough when some of these automated attacks come knocking.