Radio interference and roaming are the primary concerns. There's only so many channels that Wi-Fi can run on, and they will clash if there's too many APs near the same band. You also have to ensure that each device is configured to disconnect from a weak signal and connect to a stronger one when moving from room to room, which realistically doesn't work flawlessly. You want to instead have a few powerful access points in each wing or whatever needs dictate.