Activation Oracles

Models that accept neural activations as input to interrogate internal states, surfacing secret knowledge and hidden misalignment. Activation Oracles are LLMs trained to take activations from a target model and answer questions about them in plain language — general-purpose activation explainers, in the words of the Anthropic Alignment Science write-up. Pointed at fine-tuned models they surfaced secret knowledge and misalignment the fine-tuning had tried to hide, which is why the newsletter calls the result a machine psychoanalyzing its own weights. They open the machine-introspection theme through the development reported on 21 December; DeepMind’s Gemma Scope 2 three days later is the corpus’s next attempt to open the black box.