Yeah... while FO4 can revolve heavily around settlement-building, many (even most) players choose to avoid getting heavily-invested in it beyond what the game requires and choose, instead, to focus on the quests.
I'm not one of those people (i get obsessed with settlements), but most don't.