I wore shorts and flip flops to a grocery store in the dead of winter. Guy asked if I was cold, and I was like "I went from one climate controlled box into a drivable climate controlled box, and now I'm in this climate controlled box"
Yeah, things could go wrong like a car wreck or something where I get stuck outside for an extended period of time and would become uncomfortable. But it is what it is.