THE EXPERIMENT
What the drawing
becomes
What does a model keep when it has to squeeze a drawing into just 32 values?
I'm interested in that representation: what gets kept, what gets lost, and how it can help predict what comes next.
This demo reads a digit as you draw. The encoder state feeds a guess and a possible finished drawing. Try it before you've finished the last stroke.
A little more about the model
A convolutional encoder turns the drawing into 32 activation values. A classifier reads that state to guess the digit; a decoder uses the state and the class probabilities to generate a possible finish.
The squares show the actual activations, rather than assigned features such as “loop” or “line”. A finish is a prediction, and partial handwriting can be ambiguous.
Draw a number
It guesses as you goPlay with the values
Draw a number to explore its values
Pick any square, then move the slider to see what the model decodes on the right
What am I looking at?
Each square shows one value in the model's shared representation of your drawing. Darker squares are more active. Select a square to change its value, or read the full vector below. Both predictions use this same state. Edited values are decoded by the model and may no longer describe your original drawing. The squares aren't assigned meanings like “a loop” or “a line”.