Anthropic Releases NLA Weights for Gemma 3 27B Instruct
Explore the new NLA weights for Gemma 3 27b instruct to visualize token‑level internal thoughts.
Explore the NLA weights for Gemma 3 27b instruct and experiment with neuronpedia to visualize token‑level internal thoughts.
Summary
Anthropic has released research on Natural Language Autoencoders (NLAs), a pair of models that translate an LLM’s internal thoughts into readable text for any specific token. The research includes NLA weights for Gemma 3 27b instruct, with separate Auto Verbalizer and Activation Reconstructor models. The Auto Verbalizer translates Gemma’s activations to human‑readable explanations, while the Activation Reconstructor verifies that the explanation can be mapped back to the original activations. The weights are hosted on Hugging Face under the kitft/nla-gemma3-27b-L41-av and kitft/nla-gemma3-27b-L41-ar repositories. Neuronpedia hosts the models and provides an interactive interface where users can ask Gemma 3 a question, click a token, and view the model’s internal thought process. The platform demonstrates how NLAs can reveal token‑level reasoning in real time. This release enables developers to inspect and debug Gemma 3’s token generation.
Key changes
- Anthropic released NLA research translating LLM internal thoughts to readable text.
- NLA weights for Gemma 3 27b instruct released: Auto Verbalizer and Activation Reconstructor.
- Auto Verbalizer weight available at Hugging Face kitft/nla-gemma3-27b-L41-av.
- Activation Reconstructor weight available at Hugging Face kitft/nla-gemma3-27b-L41-ar.
- Neuronpedia hosts the models and provides an interface to view token explanations.
- Users can ask Gemma 3 a question, click a token, and view internal thoughts via neuronpedia.
- Auto Verbalizer translates activations to text; Activation Reconstructor verifies back‑translation.