A probe for vision-language models' "conscious" access: MC-Lens reads image concepts and rewrites multi-step answers
Synopsis
Vision-language models can answer questions about an image, but the answer does not show where a concept appears inside the model. MC-Lens maps intermediate states from the vision encoder and the language decoder into the decoder's vocabulary space, reads concepts by vocabulary rank, and runs counterfactual replacements along concept directions. On flag-and-capital questions, a greedily selected concept direction set redirected nearly every one-hop answer; the same set also redirected direct answers, but only about 15% of one-hop cases passed both the necessity and sufficiency tests.
Interpretation
Concept information inside a vision-language model can be read out and selectively retrieved according to the question. Earlier Jacobian-based vocabulary readouts were built for language-only models and used a single token as the carrier of a concept. On Qwen3-VL-8B, across 100 single-word-answer cases from A-OKVQA, image evidence, text evidence, or both raised the target concept token into the top readout ranks; in a skateboard example the target token moved from rank 9 to rank 1.
The concepts that are read out are not merely displayed in intermediate states; changing them changes downstream answers. Readout rank alone cannot show whether those concepts take part in later computation. On 50 national-flag images from Noto Color Emoji, replacing country-related components from Canada to Australia changed the capital answer from "Ottawa" to "Canberra"; in the one-hop task the smallest greedy direction set redirected nearly every answer, while the broadest word-form set redirected fewer than half.
The same concept directions can influence several downstream questions, not only the one asked directly. Earlier counterfactual interventions on intermediate concepts mostly targeted a single question relation. Across 51 flag source-to-target pairs covering country name, ISO-3 code, capital, currency code, continent, internet domain and calling code, greedy direction sets gave higher target-answer rates than word-form sets on one-hop tasks, while calling codes stayed hard to redirect even with the largest greedy set.
Perspective
Here "conscious" refers only to functional properties associated with conscious access in global workspace theory, not to any claim of subjective experience. The experiments cover vision-language models from the Qwen3-VL, DeepSeek-VL2 and Gemma 3 families, with readouts and interventions concentrated on controlled visual question answering and flag-knowledge questions.
The results do not yet identify the mechanism that broadcasts selected information to different downstream processes, which is a central property of global workspace theory. In one-hop reasoning, most recovered answers did not also pass the necessity test, suggesting other components may support the original answer. Workspace intervals were estimated manually from direction geometry, effective dimensionality and readout profiles, and those profiles do not independently validate the boundary locations.
