Concept· 2 min

What is multimodal AI?

Multimodal models handle more than text — images, audio, video and even actions — in a single model.

Key idea

All modalities are converted to embeddings so the model can reason across them jointly (see this image, hear this clip, then answer).

Example

GPT-4o can look at a photo of your fridge, listen to your voice question, and reply out loud with a recipe.