What if you could understand and control an LLM by studying its *smaller* sibling? Our new paper introduces the Linear Representation Transferability Hypothesis. We find that the internal representations of different-sized models can be translated into one another using a simple linear(affine) map.

I'm at #Neurips2024 this week! My work (arxiv.org/abs/2406.17692) w/ @gregdnlp.bsky.social & @eunsol.bsky.social exploring the connection between LLM alignment and response pluralism will be at pluralistic-alignment.github.io Saturday. Drop by to learn more!

Interpretability can be used to improve LLM fine-tuning - check out our poster at #NeurIPS2024! Where: East Exhibit Hall A-C #3402 (Poster Session 2 East) When: 11 Dec 4:30 - 7:30 pm PST (Vancouver time) See you in Vancouver! Would love to chat about PEFT, interp, alignment, and more