People thinking LLMs should be even serviceable at chess didn't understand LLMs. They really aren't problem solving applications. They're optimized for making responses to questions that look like what a response should look like, not for being accurate. That's really clear if you ask them for mathematical proofs. They will generate proofs that look like the right sort of thing, but they won't be correct unless they have the specific proof in their training data.