Improved multilingual performance with language specific heads

Title	Improved multilingual performance with language specific heads
Authors	Alexander Terentyev, Anastasia Voznyuk, Andrei Semenov, Nikita Okhotnikov
Advisor	Andrey Grabovoy, PhD
Consultant

🔎 Idea and Novelty

Following the work of Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs, we suggest to compare performance of various tasks aimed to measure ability of factual transition between different languages. One of the possible task can be MCQA. We want to evaluate langiage-specific neurons, activate and deactivate them and see how LLM performance on specific tasks change.

💡 Abstract

💡 Baselines

🔗 Useful links

📜 Datsets

1. CLIKE (from Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMs) - intended for evaluate factual knowledge on 35 languages