🔍 Current Research: Mechanistic Interpretability in LLMs

I am a 2nd-year master’s student, currently pursuing my thesis under Prof. Matthias Bethge and Dr. Çağatay Yıldiz at the Bethge Lab. In my research, I am delving deep into the fascinating world of large language models. My research focuses on developing novel approaches to quantify and track knowledge acquisition in these complex systems through mechanistic interpretability.

🧪 Current Research Directions

Our research extends Geva et al.’s (2023) foundational work on factual associations in auto-regressive models, developing more sophisticated measurement techniques. A critical insight emerged from Öncel et al.’s (2024) findings that traditional perplexity metrics can be deceptive – lower perplexity scores don’t always indicate true domain understanding.

I’m currently advancing research along two primary axes:

1. Knowledge Measurement Pipeline: Developing transferable methodologies for measuring domain-specific knowledge across model layers, building on “Dissecting Recall of Factual Associations in Auto-Regressive Language Models”.

2. Domain-Specific Pre-training: Investigating how specialized pre-training shapes model capabilities, extending insights from “Investigating Continual Pretraining in Large Language Models”.

A key innovation in our approach is the application of activation engineering through steering vectors – essentially creating interpretable maps of knowledge representation across model layers. This builds on Arditi et al.’s (2024) demonstration that LLMs encode features as linear directions in their activation space, complemented by Merullo et al.’s (2024) work on vector arithmetic in language models.

💡 Why Mechanistic Interpretability?

Richard Feynman’s principle, “What I cannot create, I do not understand,” perfectly encapsulates the mission of mechanistic interpretability. The field has achieved remarkable breakthroughs in demystifying neural networks, from Elhage et al.’s (2021) mathematical frameworks for transformer circuits to Elhage et al.’s (2022) understanding of superposition phenomena. Recent advances in automated circuit discovery (Conmy et al., 2023) and interpretable circuits in GPT-2 (Wang et al., 2022) demonstrate that systematic reverse engineering of these complex systems is not just possible – it’s revolutionizing our understanding.

🔬 Practical Applications

These theoretical advances have enabled significant practical breakthroughs:

• Enhanced model efficiency through targeted knowledge editing (Meng et al., 2023) (Dai et al., 2021)
• Enable direct model control through activation engineering and steering vectors, eliminating the need for traditional fine-tuning approaches (Turner et al., 2024) (Hendel et al., 2023) (Ilharco et al., 2022)
• Implement stronger safety constraints by understanding how outputs can be steered and regulated (Arditi et al., 2024) (Bereska et al., 2024)
• Enhance model performance through targeted modifications of internal representations, as demonstrated by work on singular value decomposition and rank reductions (Chen et al., 2024) (Sharma et al., 2023)
• Develop more robust evaluation methods by understanding how knowledge is stored and accessed (Geva et al., 2020) (Dar et al., 2022)

🎯 Research Focus

My research explores the crucial intersection of knowledge representation and model behavior. By combining activation engineering techniques with causal intervention methods, I investigate how models process and adapt domain-specific knowledge while maintaining their core capabilities. This work is particularly inspired by knowledge localization techniques (Meng et al., 2023), factual association tracking (Geva et al., 2023), and advances in model steering (Arditi et al., 2024).

The most compelling aspect of this work is its potential to transform theoretical insights into practical applications, advancing both our understanding and our ability to create more reliable, controllable AI systems. Apart from the papers mentioned above, I draw significant inspiration from several groundbreaking works in the field, including:

• A Mathematical Framework for Transformer Circuits (Elhage et al., 2021)
• Editing Factual Knowledge in Language Models (Cao et al., 2021)
• Language Agents Meet Causality – Bridging LLMs and Causal World Models (Gkountouras et al., 2024)
• Talking Heads: Understanding Inter-layer Communication in Transformer Language Models (Merullo et al., 2024)
• CoSy: Evaluating Textual Explanations of Neurons (Kopf et al., 2024)
• Mechanistic Interpretability for AI Safety – A Review (Bereska et al., 2024)