Tic-Tac-Toe: one neural network, trained two ways

9 inputs → 128 hidden neurons → 9 outputs, the same kind of network as Pick Activity. One copy learns from answers that a perfect player (minimax) labels. A second copy gets no answers at all: it plays games and learns only from winning and losing. Train both, then play them.

Play

Switch opponents mid-game: the board stays, and the numbers change to show what that model would play from here.

What the network sees

The board becomes 9 numbers from the point of view of whoever moves next: +1 my piece, −1 their piece, 0 empty. The 9 outputs are a probability for each square; occupied squares are forced to 0%. The dot marks minimax's best move(s) when minimax is the opponent.

Training

Supervised — learn by studying minimax

Untrained — weights are random

Epoch

Loss

—

Optimal (never seen)

—

Reinforce — learn from wins and losses only

Untrained — weights are random

Games played

Lost (recent)

—

Optimal (all)

—

How often each network picks an optimal move

supervised reinforce · measured over all 4,520 positions

The one line that changes

// supervised: predicted minus CORRECT
dz2[k] = probs[k] - y[k];

// reinforce:  predicted minus WHAT I DID, scaled by
//             how the game ended (+1 won, −1 lost, 0 drew)
dz2[k] = G * (probs[k] - (k === myMove ? 1 : 0));

// everything below this line — backpropagation
// and the weight update — is identical

Show the supervised training data (4,520 positions)

Every position that can arise in play, labeled with minimax's best square(s). The supervised network trains on 80% of them; the other 20% are set aside, never shown to it, and used only to test it. The reinforce network never sees this table.

#	Board	To move	Best square(s)

Scoreboard

100 games per cell, networks play their most likely move (no exploration). Minimax picks randomly among its equally-best moves. Playing O against minimax is the hard case: X has the initiative and O must find the one saving reply every time.

Player	Optimal moves	vs random as X	vs random as O	vs minimax as X	vs minimax as O

Cells show won / drew / lost.

Notice: the supervised network picks an optimal move ~99% of the time and can still drop a game as O, because its rare mistakes land in exactly the forced positions. The reinforce network is only ~75% "optimal" yet never loses as X: it never cared which of several equally good moves it took, it only learned not to lose in the positions it actually reached. Same network, different training signal, different lesson learned.