Mechanistic interpretability dashboard
Towards Monosemanticity Reproduction
An evidence-backed catalogue of sparse
autoencoder features learned from
gelu-1l
activations at
blocks.0.mlp.hook_post.
14/
14
Core contextual tests correct
90.9%
Expanded contextual accuracy
32.08×
Feature 3513 control ratio
25/
25
Registry checks passed
Interpretation warning
Feature labels are empirical summaries of observed behaviour, not complete definitions of everything represented by each feature.
lexicallocal continuationcausally validated
Natural activation
4.264
Mean activation on controlled
according prompts
Causal specificity
32.08×
Effect relative to the mean of the
unrelated control directions
Closest raw neuron
1643
SAE/control selectivity ratio:
1.29×
Probability interventions
According prompts at strength -4
18.05 percentage points lower
According prompts at strength +4
1.51 percentage points higher
Control prompts at strength +4
12.84 percentage points higher
Steering selectivity
| Measurement |
SAE |
Raw neuron |
| Control-prompt selectivity |
4.72
|
3.66
|
| Control target-change rank |
3.50
|
106.25
|
| According-prompt selectivity |
3.34
|
3.04
|
Evidence types
- cross-document examples
- controlled prompts
- natural-contribution ablation
- positive steering
- dose-response test
- unrelated-direction controls
- raw-neuron steering comparison
- vocabulary selectivity analysis
Known limitations
- The decoder direction also changes unrelated vocabulary fragments.
- The causal experiment tested one target token on a small prompt set.
- Decoder-direction intervention is approximate rather than perfectly isolated.
Source result files
results/step_22b_feature_3513_steering_validation.jsonresults/step_23b_feature_neuron_selectivity.json
lexicalgrammatical context
Controlled accuracy
100.0%
8/
8
expected examples
Exclusive activations
84.9%
Activations not shared with the
other tested contextual features
Closest raw neuron
548
Pearson correlation:
0.403
Activation and separation
Active token percentage
2.18%
Exclusive activation percentage
84.90%
Mean top-100 overlap
7.50%
Separation metrics
| Measurement |
Mean result |
| Active-set Jaccard |
0.0598
|
| Activation correlation |
0.0500
|
| Encoder cosine similarity |
0.0349
|
| Decoder cosine similarity |
0.0391
|
Strong cross-document examples
-
4.959
— strike amid rising tensions with North Korea, seems [[ to]] have sparked an unusually rapid response on Capitol Hill
-
4.928
— /LEGO, the source of which seems [[ to]] be early listings on the Mexican version of Amazon
-
4.283
— refraction of light water in some places seems [[ to]] be dark
-
4.143
— A record-setting reality show appears [[ to]] be paddling in the wrong direction as its
-
3.741
— box at the bottom of my page which seems [[ to]] be inheriting CSS style from bootstrap.min
Evidence types
- cross-document examples
- controlled prompts
- raw-neuron comparison
- pairwise feature separation
- controlled classification
Known limitations
- It sometimes responds to broader infinitive constructions.
- Its boundary is not restricted to one formal grammatical category.
Source result files
results/step_21b_contextual_feature_neuron_examples.jsonresults/step_24a_contextual_feature_separation.jsonresults/step_24b_contextual_feature_classification.json
lexicalgrammatical context
Controlled accuracy
75.0%
6/
8
expected examples
Exclusive activations
78.3%
Activations not shared with the
other tested contextual features
Closest raw neuron
882
Pearson correlation:
0.331
Activation and separation
Active token percentage
1.33%
Exclusive activation percentage
78.26%
Mean top-100 overlap
5.00%
Separation metrics
| Measurement |
Mean result |
| Active-set Jaccard |
0.0579
|
| Activation correlation |
0.0418
|
| Encoder cosine similarity |
0.0297
|
| Decoder cosine similarity |
0.0311
|
Boundary examples
-
We decided to travel tomorrow.
— Feature
1997
won by
0.273
-
They began to laugh.
— Feature
1997
won by
0.100
Strong cross-document examples
-
4.956
— model of, but was soon overwhelmed by having [[ to]] guess so much of the flight dynamics. Is
-
4.914
— but uncomfortable. I felt as if I had [[ to]] go diarrhea every five minutes. If this is
-
4.849
— goes where, you’re going to have [[ to]] tell her you’ll help a little.
-
4.773
— handwritten characters. Why did they have [[ to]] use a quantum computer to do that?
-
4.375
— ? A: You have [[ to]] return the return value of the recursive call.
Evidence types
- cross-document examples
- controlled prompts
- raw-neuron comparison
- pairwise feature separation
- controlled classification
Known limitations
- It is not a general detector for every ordinary infinitive.
- Some broader infinitives are shared with Feature 1997.
Source result files
results/step_21b_contextual_feature_neuron_examples.jsonresults/step_24a_contextual_feature_separation.jsonresults/step_24b_contextual_feature_classification.json
lexicalcapitalizationposition-sensitive
Controlled accuracy
100.0%
6/
6
expected examples
Exclusive activations
63.9%
Activations not shared with the
other tested contextual features
Closest raw neuron
2037
Pearson correlation:
0.166
Activation and separation
Active token percentage
0.37%
Exclusive activation percentage
63.93%
Mean top-100 overlap
3.50%
Separation metrics
| Measurement |
Mean result |
| Active-set Jaccard |
0.0391
|
| Activation correlation |
0.0174
|
| Encoder cosine similarity |
0.0493
|
| Decoder cosine similarity |
0.0446
|
Strong cross-document examples
-
4.693
— 's exactly what it is today! [[To]] say there is a lot going on in the
-
4.595
— ureteric stents in vivo. [[To]] examine the relationship between encrustation and microbial
-
4.488
— Team was chosen by popular vote of fans. [[ To]] select the team, a panel of experts first
-
4.258
— of Western Rajasthan, India. [[To]] asses the impact of drought on nutritional status
-
4.154
— , long length, String[] hosts). [[To]] pass the starting offset you can use Configuration parameters
Evidence types
- cross-document examples
- controlled prompts
- raw-neuron comparison
- pairwise feature separation
- controlled classification
Known limitations
- Capitalization is a major component but may not be the only represented property.
- Weak activations were observed for some lowercase purpose constructions.
Source result files
results/step_21b_contextual_feature_neuron_examples.jsonresults/step_24a_contextual_feature_separation.jsonresults/step_24b_contextual_feature_classification.json