I've seen suggestions that the AI Overview is based on the top search results for the query, so the terrible answers may be more to do with Google Search just being bad than any issue with their AI. The AI Overview just makes things a bit worse by removing the context, so you can't see the glue on pizza suggestion was a joke on reddit or it was The Onion suggesting eating rocks.