Speaker
Description
This talk addresses the challenge of interpreting the high-dimensional hidden representations in Transformer models, a critical issue given their widespread use in sequential data tasks. We propose using Topological Data Analysis (TDA), a powerful mathematical approach that allows us to understand the shape and structure of complex data. Using TDA, we develop a framework that follows the evolution of representations across layers of the transformer, treating it as a dynamical system that evolves in time. The framework allows us to measure the change in the degree of similarity of relations among representations across the model's depth, providing insights into how the models organize information by moving representations in high-dimensional space.
