Definition
The Jacobian Lens (J-lens) is an interpretability technique that reads out what an internal neural activation is “disposed to make the model say” by linearly transporting residual-stream vectors to the final-layer basis using average input–output Jacobians, then decoding via the model’s unembedding.
Key Points
- Formula:
lens_l(h) = unembed(J_l @ h)whereJ_l = E[∂h_final / ∂h_l] - Enables layer × position visualization of silent internal concepts
- Used to discover j-space in claude
- Reference implementation:
anthropics/jacobian-lens(Apache 2.0); companion to paper “Verbalizable Representations Form a Global Workspace in Language Models” - Interactive demo on open-weights models via Neuronpedia partnership