There's certainly a tipping point where light becomes too yellow to accurately represent color. I was recently shopping bathroom vanities and some showed what the greens and blues would look like under different color temps, with 2700K just about ruining the appearance. I also painted a room in light blues and had to change the adjustable lights to 3500K, if I remember correctly.
I'm just intrigued and thinking out loud. I'm having a hard time describing yellower as harsh. I could see the overhead lights doing a better job at flooding an area and minimizing shadows, whereas window light would be diffused but still somewhat of a point-source depending on distance. The "backrooms" image of the empty office space certainly comes to mind where it's all a vague shade of yellow-green.
As far as people who can't seem to see anything under wandering daylight, IME, they tend to be a mix of people who are either older (reduced dark vision, reduced focus) and impatient people (who don't understand your eyes take 5 seconds to adjust pupil size but 20 minutes to refill rhodopsin, your night vision juice). Or just people who demand conformity. Or a 4th group I suppose, who have max-brightness screens that doesn't play with eyes well against dark backgrounds. I do personally prefer natural light and wait for my eyes to prove they can't see enough before using lights, except for when I have physical tasks to do like cook or repair something.
Apologies for seeming like I was telling you you're wrong. I was trying to get your perspective but just rambled in my own opinions. Lights are a notable hobby for me, sort of. Headlights, flashlights, night lights, street lights, light pollution, night sight, neon lights, uv lights... I read up on lights weirdly often.