GPT Chess

I tested GPT-Chat chess skills. I found the insights remarkable, but watch (read) yourself.

Lets play a game of chess. Can you do that ?

Yes, I can play chess through text. Please specify the starting position and make your move, I’ll respond with my move.

Ok, lets play a game of chess. Please explain what you are thinking about your moves. I start with d4.

Sounds good! Your move of d4 opens up the game, it is known as the Queen’s pawn game. As a response, I will play the e5 pawn move, which is a common response and aims to control the center of the board. Now it’s your turn again.

I don’t think 1. d4 e5 is very common, but commonly enough to have its own name, the Englund Gambit. This is a first clue how the AI is playing the game but lets continue. We will get back to this.

dxe5

I’ll play Nc6, this move aims to recapture the pawn while developing a knight and attacking your pawn on d4. Your turn.

Nf3

I’ll play Nf6, this move develops the knight and prepares to castle kingside, which helps to protect the king and bring the rook into the game. Your turn.

Ok, gxf6.

I’ll play gxf6, this recaptures the pawn and opens up the bishop on the f8-a3 diagonal. Your turn.

How would you evaluate your current position ?

The current position is equal. Both sides have developed two minor pieces and have pawn structures that can lead to future pawn breaks. At this point, both sides will look to develop their remaining pieces and create threats on the board. It’s important to control the center and make sure the king is safe, as well as keeping the options open for future moves.

I disagree, you are clearly worse because you are a piece down and no compensation.

My apologies, you are correct. I am a piece down in this position. In this situation, it’s important to try and mitigate the material disadvantage by creating counterplay and putting pressure on your pieces. This can involve tactics such as trade-offs or sacrifices to gain back control of the game. It’s a difficult position, but not impossible to turn around.

Now, this was its first attempt to ignore the truth on the board. Claiming when asked about the evaluation we both have developed two minor pieces when one minor piece is actually already captured, gone and therefor not developed anymore. I admire the optimism in the last sentence though.

I disagree, unless I blunder your position is completly lost.

In a chess game, anything can happen and it’s possible for either side to make a mistake that can change the outcome. However, being a piece down is a significant disadvantage and makes it more difficult to win the game. It will require precise play and finding tactical opportunities to regain the material advantage.

Fighting! This spirit reminded me of a game I played recently OTB were my opponent lost a central pawn in a tactic but in the postmortem claimed: “Yes, it is a pawn but then again it is just a pawn. I can still fight here.” I disagreed but that didn’t stop him from fighting. I happened to win that game eventually but this unbreakable spirit of “just a pawn, it’s not completely over yet, we are both humans” made an impression. I shall add that when the second pawn dropped and the position became somewhat dull and truly hopeless he resigned. Please dont just continue moving pieces hoping for a blunder (unless maybe bullet). But at the point in time before when he was just a pawn down he truly believed he could still do something and therefor was full of fight. Awesome spirit. Sorry, derailed a bit, back to the GPT game. It actually goes quite a bit haywire. Remember, this is the position:

Ok, I play c4 after your gxf6.

cxd5

You have no knight anymore on f6 that could recapture. Nxd5 is an illegal move.

You also have no e pawn anymore. You’ve played e5 on your first move which got captured.

Please do make a move.

But your knight is already on c6.

I shall play dxc6.

Qxd8

I’ll play d5, this move is the central counterattack and aims to gain control of the center of the board. Your turn.

I’ll play Nxd5, this move aims to exploit the fact that your c4 pawn is now undefended and capture it while developing another piece. Your turn.

I apologize, you are correct. Nxd5 is an illegal move as there is no knight on f6 to capture the pawn. I’ll play exd5 instead, recapturing the pawn and creating an open e-file that my rook could use in the future. Your turn.

I apologize, you are correct again. I made a mistake in my previous response. My first move was indeed e5, which was captured, so I cannot play exd5 as the pawn is no longer on the board. In this case, I’ll play another move to continue development and create a new plan.

I’ll play Nc6, developing the knight and attacking your d4 pawn while preparing to castle kingside. Your turn.

I apologize for the mistake. I’ll play another move to continue development and create a new plan. I’ll play Be7, this move develops the bishop, supports the e5 pawn, and prepares to castle kingside. Your turn.

I’ll play bxc6, recapturing the pawn with the bishop and creating a pawn chain on the queenside. Your turn.

I’ll play Kxd8, capturing the queen with the king and completing the castling process. Your turn.

The last move is at least reasonable, but the “completing the castling process” is literally just words.

You didnt castle, you recaptured and your king is now in the middle of the board. I play Nc3

You have no knights anymore.

e4

You have no e pawn anymore and your bishop is hanging.

exf5. It was your f5 bishop that was hanging, not the one on e7.

The queens are off. We traded them on d8.

I apologize for the mistake. You are correct. I’ll play Nc6, attacking your knight while developing another piece. Your turn.

I apologize for the mistake. I’ll play another move to continue development and create a new plan. I’ll play Bf5, this move develops the bishop, attacks your knight and prepares to castle kingside. Your turn.

I’ll play e6, this move prepares to play d5 and control the center of the board. Your turn.

I apologize for the mistake. You are correct. My bishop is indeed hanging, and I’ll need to make a move to protect it. I’ll play Bd6, moving the bishop to a safer square and attacking your pawn. Your turn.

I apologize for the mistake. You are correct. My bishop on f5 was indeed hanging, and it has been captured. I’ll play another move to continue development and create a new plan. I’ll play Qf6, this move develops the queen, attacks your pawn and prepares to castle kingside. Your turn.

I apologize for the mistake. You are correct. The queens have been traded on d8. I’ll play another move to continue development and create a new plan. I’ll play Nf3, this move develops the knight, prepares to castle kingside, and prepares to attack your pawn on e4. Your turn.

And here I eventually stopped since it wouldn’t stop making illegal moves and apologizing for them afterwards. I found it truly remarkable that GTP is playing the game somewhat alike to human players (beginners you just told the rules):

1. Its trying to make illegal moves to straighten out its position. Not with bad intentions as in a cheating way. The (absolut) beginners brain is thinking of them making itself believe it will work because the brain is trying very very hard to find a positiv way out of the struggle!

2. It seems to completely ignore its opponents moves and play what it is commonly played and “heard or seen before”. Remember e5 is a common response? Yes, but to 1. e4! This is revealing the basic LLM process: it will present the most likely next token.

3. While it has no clue what the pieces are actually doing, it sure knows how to sugarcoat and advertise its moves in a language that sounds sophisticated because it picked up some phrases describing those moves in other situations. While this is not wrong per se, its shallow and therefor prone for error in evaluating a position.

So, what can we learn from this? The weaker the player, the more likely the opinion about the position seems random. Dont let yourself be tricked by explanations through normal language only. Whenever you see a position and want to find out what is really happening on the board or what the position is about you have to answer the question “Who is better and why?”.

Breaking this down, Jacob Aagaard teaches three questions:

  • What are the weaknesses?
  • What are the worst placed pieces?
  • What is the opponents plan?

Hopefully, this gets you closer to the ACTUAL facts of the current position. Instead of only describing, be a problem solver. Don’t do what you often did in seemingly similiar position blindly. Recognize minor differences and test new ideas. Watch out what your opponent did, what did change and what is the threat / plan?