No drawbacks for my use case, but there are a few quirks.
Some apps that I've dabbled with might show vertically, but most of the Android TV compatible apps were fine.
Some apps seem to expect a touchscreen input, and don't really show what's highlighted or selected very well, so navigating with just a TV style remote control can be tricky. I have a mouse and keyboard that I also use, definitely the easiest option.
Lastly, the Shield is a pretty powerful device, so I always kinda feel like I'm not using it to its fullest potential. I haven't really tried any gaming with it, I don't use it as a media server or anything, just a simple stream box. That's less of a quirk with the device itself, and more an issue with me feeling like I should be putting it to better use.