References & Citations
Computer Science > Computation and Language
Title: I Have an Attention Bridge to Sell You: Generalization Capabilities of Modular Translation Architectures
(Submitted on 27 Apr 2024 (v1), last revised 30 Apr 2024 (this version, v2))
Abstract: Modularity is a paradigm of machine translation with the potential of bringing forth models that are large at training time and small during inference. Within this field of study, modular approaches, and in particular attention bridges, have been argued to improve the generalization capabilities of models by fostering language-independent representations. In the present paper, we study whether modularity affects translation quality; as well as how well modular architectures generalize across different evaluation scenarios. For a given computational budget, we find non-modular architectures to be always comparable or preferable to all modular designs we study.
Submission history
From: Timothee Mickus [view email][v1] Sat, 27 Apr 2024 14:10:51 GMT (7083kb,D)
[v2] Tue, 30 Apr 2024 05:53:10 GMT (7083kb,D)
Link back to: arXiv, form interface, contact.