Reinforcement learning

"Beat the Computer!"

Reinforcement learning is a psychology-inspired paradigm of machine learning: the computer learns to master a task through reward and punishment in interaction with its environment, trying to maximize the possible reward.
Using a mini chess game, we want to make the ideas underlying reinforcement learning algorithms tangible here.
 
Have fun!
 
 
!   Play the game "Beat the Computer!" first against the dumb robot, which is not able to learn anything. Start by reading the game instructions.
 
 
 
Game Instructions:
Each game piece moves like a pawn in normal chess, i.e., it can only move forward and capture opposing pieces diagonally. A side wins if it manages to

  • move one of its own pieces to the other end of the board,
  • capture all opponent pieces,
  • or prevent the opponent from making a move in the next turn.

Here you see an example of each of the three ways the game can end.


 
 
 
 
!   

Too easy? Now the robot really starts to learn. Do your best to beat it again! But before jumping into the game, please read the following information, which explains how the robot learns and improves. As the saying goes: "Know your enemy"...

 
 
 

The human player starts and can move freely according to the game rules. Then it’s the computer’s turn: it analyzes the current board, checks possible moves, and chooses the appropriate game situation from the available options.

 
Rules

Then the computer randomly draws one of the colored tokens next to the respective game situation. The color of the token determines which move will be executed. For example, if a red token is drawn, the robot moves according to the red arrow.

This procedure is repeated until a winner is determined. Before a new round is played, the computer adjusts its strategy as follows:
 
  1. Computer wins: A token in the color of the last winning move is added to that move's field.
  2. Human wins: The token that determined the computer's last move is removed from that move's field.
At first, the computer will have little chance of winning since it chooses moves randomly (by drawing a token). The more games it plays, the better it gets: it "learns" which moves lead to victory and which to avoid, as they led to losses in the past. This gradually refines the computer's strategy. Because the computer is punished for losing and rewarded for winning, this is called reinforcement learning – learning through reward and punishment:

    Punishment = removing a token for a move that led to a loss
    Reward = adding a token for a move that led to a win
 


Through this process, moves that resulted in losses are gradually "eliminated," so that eventually only "good" moves remain.
 
The presented algorithm is meant to explore the principle of reinforcement learning. In practice, one would, for example, not immediately eliminate unsuccessful strategies, but only reduce their probability of occurring. The AI system gradually learns which actions are most suitable in each situation, but does not completely exclude moves that once led to failure.
 

Video Tutorial
  
 

 
In this game, you experienced how reinforcement learning works. The computer starts without a strategy and chooses its moves randomly, represented by the colored tokens. After each round, it adjusts the distribution of these tokens: moves that led to a win are "rewarded" (a token is added), and moves that led to a loss are "punished" (a token is removed). In this way, the computer gradually increases the probability of selecting successful moves again and avoids those that previously led to a loss. It develops a better strategy purely from experience, without anyone explicitly teaching it rules or tactics.