Latest from Google AI – MaMMUT: A simple vision-encoder text-decoder architecture for multimodal tasks
Posted by AJ Piergiovanni and Anelia Angelova, Research Scientists, Google Research Vision-language foundational models are built on the premise of a single pre-training followed by subsequent adaptation to multiple downstream tasks. Two main and disjoint training scenarios are popular: a CLIP-style contrastive learning and next-token prediction. Contrastive learning trains the model to predict if image-text…