Disentangling Superpositions: Interpretable Brain Encoding Model with Sparse Concept Atoms

The short version

Encoding models predict brain activity from word embeddings, and they predict well. But an embedding compresses many thousands of concepts into a few hundred dimensions, so concepts share directions, and the fitted weights cannot say which concept a voxel is actually tuned to. We unpack the embedding into 1,000 sparse concept atoms, one concept per axis, and refit. Prediction is unchanged; the weights become legible. The first thing they show is that time, space and number occupy the same regions of cortex.

• 300 → 1,000GloVe dimensions unpacked into concept atoms
• 0.0005change in R² from switching to sparse features (not significant)
• 0.26 vs 0.09agreement of concept maps across people, sparse vs dense
• IPS + IFGwhere time, space and number all overlap, in both hemispheres

Encoding models, briefly

Seven people lay in an fMRI scanner and listened to true stories told on stage for The Moth Radio Hour: eleven stories each, a little over two hours in all (Huth et al., 2016). Ten stories are used for training; the eleventh, heard twice, is held out for testing.

An encoding model is a deliberately simple instrument. Every word of the story is turned into a vector of features. For every voxel — a 3 × 3 × 4 mm volume of brain — a ridge regression is fit from those features to the voxel’s response. The model is then judged only on the story it has never heard.

The object of interest is not the prediction but the weights. Each voxel receives one weight per feature. If the features were concepts, a voxel’s weights would be its tuning curve, and one could read it as a sentence: responsive to numbers, indifferent to family, suppressed by small talk.

The catch: more concepts than dimensions

The features that predict best come from language models; here they are GloVe word embeddings, 300 numbers per word, learned from patterns of co-occurrence — Wittgenstein’s meaning-as-use, made geometric. Language contains far more than 300 concepts, so the embedding does what any crowded space must: concepts share dimensions. Each concept is a direction, and the directions are correlated. In the interpretability literature this is called superposition (Elhage et al., 2022).

That undoes the readout. Ask whether a voxel’s weight vector points along number and some of time comes with it, because the two directions lean the same way. Some concept pairs point in nearly opposite directions, so a negative weight may mean the voxel is suppressed by one, driven by the other, or both. An overlap seen on a brain map may be a fact about the brain or an artefact of the embedding’s geometry, and the weights alone cannot tell the two apart.

How crowded is the space? Learn 1,000 concept directions from GloVe and measure how nearly parallel they are. In 300 dimensions, 1,000 random directions are almost orthogonal: only 0.03% of pairs have a cosine above 0.2. Among the learned concepts the figure is 2.18% — seventy times the overlap expected by chance.

The fix: give every concept its own axis

The tool is sparse coding, proposed by Olshausen and Field in 1996 as an account of how visual cortex represents natural images: find a dictionary of atoms such that each input is the sum of only a few of them. Word vectors are well suited to this. Queen is royalty together with femaleness; royalty recurs in king and castle, femaleness in daughter and mother, and most words are composed of a handful of such factors.

Non-negative sparse dictionary learning on the GloVe vectors yields an overcomplete dictionary: 1,000 atoms in the original 300-dimensional space. Each word becomes a sparse, non-negative combination of atoms. Walking, for instance:

Most atoms are equally legible. A sample, with the words that activate each most strongly:

Then the substitution. Every word is re-encoded as its 1,000-dimensional sparse code, and the same ridge regression is fit on those features in place of the 300 dense ones. Each weight now belongs to a single atom, and because the codes are non-negative, a positive weight means the voxel is driven by that concept and a negative weight that it is suppressed.

One refinement made the atoms cleaner. Very frequent words lie close to the origin of an embedding, so sparse coding tends to collect unrelated frequent words into a single atom. Before learning the dictionary we reparameterize each vector by its norm, spreading the short vectors apart and drawing the long ones together. This Vector Norm Reparameterization gives more coherent atoms, lower reconstruction error and sparser codes.

What follows

Prediction: unchanged

The first concern is that expanding 300 dimensions to 1,000 sparse ones should cost accuracy. It does not. Across the seven subjects, the mean difference in held-out R² between the sparse and dense models is 0.00047 ± 0.00071, paired t(6) = −1.77, p = 0.13. Voxel for voxel, the two models lie on the diagonal.

Maps you can read

A concept map is now immediate: take one atom and plot its weight in every voxel. Atom 456 (standing, sitting, crouch) and atom 115 (grandparents, sister, dad) each give a distributed, bilateral pattern, and the pattern is nearly the same in two different brains.

The dense model has no principled way to produce such a map, only heuristics, and the one used here yields maps that are patchier, less symmetric and less consistent between people. Across the 20 most active atoms and all seven subjects, sparse maps agree between people at cosine 0.26 ± 0.04; dense heuristic maps at 0.09 ± 0.04.

Time, space and number share cortex

A long-standing hypothesis holds that the brain uses one shared magnitude system for time, space and number (Walsh, 2003; Dehaene). Behaviour supports it; imaging has been inconsistent. Concept atoms make the test direct: the number atom (eleven, sixteen), the time atom (morning, afternoon) and the space atom (near, distance) are independent axes in one space, so their maps can be laid over one another.

Take the top 5,000 voxels for each and colour them red, green and blue. White, where all three coincide, falls in the intraparietal sulcus and the inferior frontal gyrus, bilaterally, in both subjects shown — the regions the shared-magnitude literature has pointed to. Averaged over seven people, each pair of domains overlaps with an intersection over union of about 0.32–0.34, and all three together at 0.18.

This result has its own page, starting from the theory of magnitude →

Concepts have a geometry in cortex

With twenty legible maps one can ask how they relate. Compare every pair, cluster the 20 × 20 similarity matrix in one subject, and the same block structure appears in the next; across all seven, the matrices correlate at 0.70 ± 0.08. Is this the embedding’s structure showing through? It is not: the sparse features themselves are uncorrelated (|r| = 0.00 ± 0.10 across the 190 pairs), while the same directions in GloVe space correlate at 0.16 ± 0.14. The geometry is cortical.

Takeaways

• Weights on dense embedding features are not tuning curves. Superposition makes them non-identifiable, so a positive projection onto a concept establishes nothing.
• Sparse coding is a factorial design for data one could not design. When the stimulus is a story rather than a controlled experiment, it decorrelates the features of interest after the fact.
• It costs nothing in prediction, and it turns a black-box encoding model into a set of concept maps that can be compared.

The limits should be stated. The word vectors are static and context-free, and contextual features from language models are the natural next step; 1,000 atoms was a choice rather than a result; and atoms inherit whatever biases the embedding carries. Better semantic decoding is also a double-edged instrument, and the questions it raises about mental privacy are real.

Abstract

Encoding models using word embeddings or artificial neural network (ANN) features reliably predict brain responses to naturalistic stimuli, yet interpreting these models remains challenging. A central limitation is superposition: distinct semantic features become entangled along correlated directions in dense embeddings when latent features outnumber embedding dimensions. This entanglement renders regression weights non-identifiable—different combinations of semantic directions can produce identical predictions, precluding principled interpretation of voxel selectivity. To address this, we introduce the Sparse Concept Encoding Model, which transforms dense embeddings into a higher-dimensional, sparse, non-negative space of learned concept atoms. This transformation yields an axis-aligned semantic basis where each dimension corresponds to an interpretable concept, enabling direct readout of conceptual selectivity from voxel weights. When applied to fMRI data collected during story listening, our model matches the prediction performance of conventional dense models while substantially enhancing interpretability. It enables novel neuroscientific analyses such as disentangling overlapping cortical representations of time, space, and number, and revealing structured similarity among distributed conceptual maps. This framework offers a scalable and interpretable bridge between ANN-derived features and human conceptual representations in the brain.

Poster

The whole story on one sheet, as presented at NeurIPS 2025. The talk is on SlidesLive; the paper is on OpenReview.

BibTeX

@inproceedings{zeng2025disentangling,
  title     = {Disentangling Superpositions: Interpretable Brain Encoding Model
               with Sparse Concept Atoms},
  author    = {Zeng, Alicia and Gallant, Jack},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2025},
  url       = {https://openreview.net/forum?id=3aNvX9TQTo}
}