An encoder turns the visible context into a representation. A predictor uses it to estimate a target representation, such as the features of a hidden region of an image, and training compares that estimate with an encoding of the actual target.
Predicting features instead of reconstructing every pixel can help the system learn patterns that matter for later tasks. What the model learns depends on which context and targets it sees and how those representations are trained.
ELI5
Imagine looking at a photo with one part covered. You might work out what sort of thing is behind the cover without being able to draw every detail.
For example, the visible part of a bicycle can help an AI system predict useful features of the hidden part without guessing the exact color of each pixel.
What is an embedding in this architecture?
It is a list of numbers produced by an encoder to capture useful features of the context or target.
Does the architecture have to generate a picture?
No. It can learn by comparing predicted features with target features without producing an image for a person to view.
