Can an LLM really teach you chess?

but when you let them play chess they will give a lot of illegal moves

Yes, but you ground the LLMs in chess context and then it works well quite well :) I DM'ed you!