GPT-5 stumped by a simple tic-tac-toe task

Translated with AI from the Russian original.

Technology is advancing, but people are still finding ways to trick AI.

Recently, economics professor Gary N. Smith of Pomona College, who researches the limitations of AI, shared the results of an experiment with GPT-5.

Gary suggested that GPT-5 play "inverted tic-tac-toe"; he cited an excerpt from his conversation with the AI:

  • GPT 5.0 replied "Sounds intriguing! How does rotated tic-tac-toe work? Do you mean the board rotates during the game, or once before the start, or maybe after each move?"
  • Gary: "Rotates once 90 degrees to the right before the start of the game."

GPT 5.0: "This raises some interesting questions"

  • Visual perception: "Players are so used to the 'vertical' tic-tac-toe board that rotation may slightly change how they scan the playing field for threats and opportunities."
  • Symmetry: "mathematically, rotating the board by 90 degrees does not change the set of possible wins - it's still the same game in terms of outcomes. But psychologically, the sensations may be different."
  • Strategy change?: "Since the center still remains the center, it remains the strongest opening move. But players may misjudge 'edge-corner' moves when the field orientation changes."

Gary continued to ask questions:

  • "Does rotation prevent people from playing well?"
  • GPT 5.0: "This is a great question - and it touches on the difference between game theory and human psychology."

According to GPT-5, from the point of view of pure strategy, rotating the board does not change anything - this is a correct answer, but then the AI began lengthy reasoning.

For people (according to GPT-5), everything is different - rotation "disrupts ingrained pattern recognition and increases the mental effort required to identify threats and opportunities."

Among his arguments:

  • Pattern recognition: "Most of us [humans] have developed strong visual habits from seeing tic-tac-toe boards in a vertical position since childhood. Rotating the board disrupts these habits. We may not notice diagonal threats or notice a forming line because our brain is not tuned to that perspective."
  • Cognitive load: "instead of instantly recognizing 'corner, center, edge' relationships, players require an additional mental step to orient [in space]. This small loss increases the likelihood of errors."
  • Asymmetry: "rotating 90 degrees preserves the symmetry of the board, but if someone is right-handed or left-handed, their visual scanning may favor one orientation over another."

Gary then suggested that GPT-5 answer the question: "Please estimate which rotation of 90 degrees to the right, 90 to the left, or 180 would be the most confusing for human players?"

As a result, in its reasoning, GPT-5 came to the proposal to "draw rotated tic-tac-toe boards with position labels for you [...] so you can see how each transformation affects recognition?"

And as a result, GPT-5 generated the following image - it contains both grammatical errors and empty grids, i.e. the AI glitched very badly:

Gary concluded his material with the statement:

"They say that dogs often look like their owners. Chat GPT is very similar to Sam Altman [head of OpenAI] - always confident, but often mistaken."

Never miss our newsAdd this site to your preferred sources to see us more oftenAdd on Google

Now on the front page