DeepMind & Toulouse U Contribute Composable Function Preserving Transformations to Boost Transformer Training | Synced

In a new paper Composable Function-preserving Expansions for Transformer Architectures, a research team from Google DeepMind and University of Toulouse introduces parameter expansion transformations for transformer-based neural networks while preserving functionality, enabling the expansion of the capability of the model as needed.