One of the biggest challenges in AI is understanding how neural networks actually work.

We can observe what they do, but explaining how they produce particular results remains difficult.

New research suggests their internal workings may be more understandable than previously assumed, with important implications for AI reliability and safety.

Nick Cammarata, formerly of OpenAI's interpretability team and now at Goodfire, is challenging an influential theory of how neural networks represent information, including his own earlier research.

Researchers have previously found that individual artificial neurons sometimes respond to seemingly unrelated things. This is known as polysemanticity.

One explanation, known as superposition, is that networks learn to pack unrelated features into the same neurons, making their internal workings difficult to disentangle.

Cammarata argues that important evidence supporting this view of how neural networks organise information was misinterpreted.

For example, a neuron thought to respond to both cats and cars appears instead to detect a V-shaped feature common to both. What looked like two unrelated functions may actually have been one.

Neurons that detect edges and shapes have long been recognised in both biological vision and artificial neural networks. Earlier researchers had considered whether the responses reflected a shared visual feature but rejected that explanation, partly because their analytical tools produced misleading results that obscured the underlying pattern.

Using improved analytical methods and AI agents, his new investigations suggest that neurons and their connections often do have coherent, understandable functions.

These findings remain a working hypothesis, but they suggest that understanding how AI systems reach decisions may be more achievable than previously thought.

The implications could be substantial. If neural networks are more interpretable than previously assumed, researchers may be able to understand how AI systems work by examining their existing neurons and circuits directly, rather than relying on additional models to reconstruct and interpret their internal activity.

Combined with AI agents capable of investigating these mechanisms at scale, this could accelerate our ability to explain AI behaviour, identify the causes of errors and improve reliability and safety.

Nick Cammarata, Neural Networks Are Already Clean, Goodfire. https://already-clean.goodfire.com/