Researchers at South Korea's KAIST and Naver AI Lab discovered that reasoning steps in AI models can be separated into distinct internal patterns. The study shows that different reasoning operations, such as extraction and computation, produce unique activation patterns within the models' numerical representations.
The team defined eight recurring reasoning operations and tested them across three models: Qwen2.5-7B, Qwen3-8B, and Gemma4-31B. They had the models solve math problems, split the solution paths into segments, and used GPT-5 to label each segment with one of the operations. The same response produces a different activation pattern depending on which reasoning operation is being probed.
Segments cluster along the corresponding direction in the scatter plot, showing that reasoning steps are clearly separable inside the model.
Common function words like 'a,' 'is,' or 'the' show up across very different reasoning steps but separate according to the surrounding operation in the middle and later layers.
The same word gets a different internal representation depending on which reasoning step it belongs to.
The researchers also tested whether a reasoning step forms in isolation. When they blocked attention to the preceding 30 tokens through a targeted intervention, the signal for that operation weakened.
Reasoning steps don't emerge on their own but build on the preceding context. Even on incorrectly solved problems, the type of step the model was performing stayed identifiable, whether it was computing, retrieving a formula, or deducing.
The separability held up in additional tests too. It replicated with Llama-3-8B, and for Qwen3-8B the trained classifiers transferred successfully to GPQA-Diamond and MATH-500. That said, the experiments are limited to math tasks and a handful of models.
Whether these findings can be used to catch errors or steer a model mid-generation remains an open question for future work.
Source: thedecoder
