I did have LUKS and a USB flash drive with a key to be inserted on boot. It was definitely difficult and caused performance issues. It was particularly difficult to add/remove drives from the array. These days I only encrypt my off-site backups that sit at the office where my coworkers potentially have physical access.
There have been recent advancements in TPM so disk encryption is easier to maintain and doesn't affect performance. I'll need to investigate this one day. My server/NAS is a 4th-gen i5, so it may not support the functions I would need. Full disk encryption will land in Ubuntu soon. I'm hanging out for that.