Why would i ask an LLM a question that i cant verify the accuracy of instead of just doing a traditional search of trusted resources? They give you the answer they think you want. Search engines don't want to crack down on SEO techniques because it will ultimately harm their business. But they can get around that by lighting a few acres of rainforest on fire, make up some random crap that sounds believable, and boost their stock price.
I'm sure that there are some niche use cases currently that can benefit from these programs, but most are just the next project for crypto grifters, and any legitimate uses are only really gonna be useful at commercial/industrial scale and won't be actually useful for the general public.