100 inputs (a 10×10 picture) → 8 hidden neurons → 1 output, the same kind of network as Pick Activity and Tic-Tac-Toe. This time there are no rules and no perfect player to copy. You make the training data: pick the faces you think are happy, and the network learns your definition. Then we test it on faces it has never seen. New in this one: a second model, convolutional, that slides a few tiny filters across the picture instead of wiring every pixel to every neuron.
Press every face you'd call happy. Everything you pick becomes a training example labeled happy; faces you skip become not happy. Pick a handful, not all of them: some happy faces must stay unpicked so we have unseen ones to test on. Arrow keys move between faces, Space or Enter picks.
| Training set | Test set | |
|---|---|---|
| Happy | 0 | 0 |
| Not happy | 0 | 0 |
| Junk images | 0 | 0 |
| Total | 0 | 0 |
Up to 12 of the training faces with the network's current “happy” probability. Watch them move while it trains.
// forward: 100 pixels → 8 hidden neurons → 1 probability z1[j] = b1[j] + Σ_i x[i] · W1[i][j] a1[j] = max(0, z1[j]) // relu z2 = b2 + Σ_j a1[j] · W2[j] p = 1 / (1 + e^(−z2)) // sigmoid // loss: binary cross-entropy. y = 1 for happy, 0 for not loss = −( y · log p + (1 − y) · log(1 − p) ) // backward: the same "predicted minus correct" line as the other two demos dz2 = p − y dW2[j] += a1[j] · dz2 dz1[j] = (z1[j] > 0) ? dz2 · W2[j] : 0 dW1[i][j] += x[i] · dz1[j] then every weight -= learning rate × its gradient
// convolutional front end: 6 filters of 3×3 weights, each slid over the whole picture
map[k][r][c] = relu( bF[k] + Σ_{i,j} F[k][i][j] · x[r+i][c+j] ) // the SAME 9 weights at every (r, c)
pooled[k][q] = max of map[k] over the left / right half // whether it fired matters, not where up or down
// then the same tail as the fully connected model, on the 12 pooled numbers
z1[j] = b1[j] + Σ_i pooled[i] · W1[i][j] a1[j] = max(0, z1[j])
z2 = b2 + Σ_j a1[j] · W2[j] p = 1 / (1 + e^(−z2))
// backward: identical down to the pooled layer, then the gradient of each pooled
// number goes back to the one spot that won the max, and the filter's gradient
// adds up what it learned at EVERY spot it was applied
dF[k][i][j] += dmap[k][r][c] · x[r+i][c+j] summed over all r, c
These were never in the gallery and are not faces at all. The network has to say something about them; notice that it never says “I don't know.” They are not counted in the accuracy above.
Click or drag to paint pixels. This face is only for testing: it never goes into the training set. Arrow keys move, Space toggles. The arrows under the grid slide the whole drawing so you can see how position changes the verdict; nothing is lost off the edge.
Try it: start from a random happy face and erase one corner of the mouth. Draw a mouth with no eyes. Draw just the eyes. Scribble. Where does your definition of happy and the network's come apart?
Where the training faces live: the generator puts eyes around rows 3–4 and the mouth on rows 7–9, and slides faces left or right but never up or down. Draw a smile two rows higher and the network has never seen a mouth on those pixels. A plain fully connected network has no idea that the same shape in a different place is the same thing: every pixel is a separate input. Turn up shifted copies in step 2 and retrain to see how much, and how little, that helps this model. Then switch to Convolutional and retrain: one small filter slid across the whole picture, so a mouth corner is a mouth corner wherever it is, and with shifted copies it handles most positions.
Things to notice: the network can score 100% on the training faces and still miss on the exam. If you only picked faces with big grins, narrow smiles may fail. Junk gets a confident answer either way. Draw a smile two rows too high and the fully connected model fails, with or without shifted copies; the convolutional model usually handles it, and with copies shifted up to 3 it handles most positions. And the Show the network picture tells you which pixels each hidden neuron is watching: usually the mouth corners.