The fediverse just needs time to grow. It's still a very new thing. Growth will take a long time, but the fediverse needs to be an alternative to corporate social media, not integrated with and dependent on it.
It will be more difficult and damaging to defederate later when Meta starts throwing its weight around, because people will become accustomed to a much larger community. The fact that threads has many more users will give Meta disproportionate power over the fediverse, but unlike other open source/free projects, their only goals are to profit and gain more influence over society.
It's best to never federate with them, even if that means losing growth opportunities in the short term.