The argument I see in articles is engagement. Remote workers are less likely to give feedback and are less loyal to the company they say.
I'm not sure remote work is responsible for that. There have been enough meetings where everybody in the office just nods along and loyalty went out the window after layoff waves became normal.
Although there is a camaraderie that is easier to build in office that keeps people around a bit longer, office politics also erodes that goodwill.