Anthropic has released new research exploring whether its large language model, Claude, exhibits signs of consciousness. The study, titled 'J-Space,' investigates how the model processes language and whether it has distinct internal representations for automatic processing and explicit reasoning. The research builds on previous work that identified hidden multi-step reasoning and multilingual circuits in Claude 3.5 Haiku. The new findings suggest that the model may have at least two representational components for using the same fact: one for automatic language processing and another for explicit report and flexible reasoning. Source: huggingface
The researchers conducted an experiment where they gave Claude a Spanish passage without naming the language and asked it to continue in the same language. The model successfully continued the passage in Spanish and named 'García Márquez' as a famous author. In a second condition, the researchers replaced the internal representation of Spanish with French, and the model responded with 'Victor Hugo' but still continued the passage in Spanish. This suggests that the model has separate mechanisms for language processing and explicit reasoning. The study also introduces a new technique called the Jacobian lens, or J-lens, to analyze the model's internal representations. Source: huggingface
The paper discusses the transformer architecture and how researchers typically look for concepts in an LLM's 'thoughts.' The transformer model consists of token embeddings, attention blocks, and MLP blocks, with the residual stream acting as a central communication channel. The residual stream records the model's evolving state as information is processed through the layers. The logit lens is a method used to inspect intermediate layers by sending the residual-stream activation directly through the model’s final normalization and unembedding. This allows researchers to see how the model's next-token prediction evolves with depth. Source: huggingface
