Understanding Representational Alignment in Vision and Beyond

Phillip Isola

Massachusetts Institute of Technology, US

Abstract

How alike or different are the representations in different computer vision models? Recent work has found that modern models have a surprising degree of similarity in how they represent the world, and this similarity is increasing over time. My lecture will give an introduction to the study of “representational alignment” between different models. We will try to understand how representations can be compared, and in what ways are different representations similar and different. I will cover some of the reasons why we might expect (or not expect) representations to align. I will also talk about convergence between vision models and models for other modalities, such as LLMs.