Multimodal input lets a model combine visual information with language instructions or other signals. In development tasks, it may inspect a page image, reason about its layout and produce code or an explanation based on both modes.
Support for an input type does not guarantee strong performance on every cross-modal task. Evaluations should test whether the model preserves detail, follows instructions and connects information correctly across the provided formats.
ELI5
A multimodal model can work with more than one kind of information, such as text and images. This lets the model connect a written instruction with something visual, spoken, or recorded in another supported format.
For example, a user can upload a screenshot and ask why a button is difficult to read. A multimodal model can inspect the image, combine it with the question, and discuss visible contrast or layout problems.
