He can see the whole room though. This comic fails completely at what it's trying to say. The fish could think the house has infinite rooms if the room doesn't have any windows. But it probably does so he'd think whatever he sees outside is infinite, and much of that is probably the sky which makes it pretty close to what we humans think.
