An image can be divided into patches that are converted into numerical representations and combined with positional information. Attention mechanisms learn relationships among these representations for a visual task.
Intermediate features are learned numerical structures rather than necessarily recognizable image parts. Their usefulness depends on training data, architecture and evaluation; an intuitive patch description does not fully explain the model's internal computation.
ELI5
Imagine dividing a picture into small tiles and considering how the tiles relate to understand the whole scene.
For example, a model can connect information from patches covering an object's shape and surroundings rather than treating each patch as an independent picture.
Are the tokens necessarily words?
No. In a vision transformer, tokens can represent visual patches or other image-related information.
Are its intermediate features always easy to interpret?
No. Learned numerical representations need not correspond to simple human-readable image features.

