A study conducted by researchers at Bocconi University, in collaboration with OpenAI Economic Research, found that access to ChatGPT and training in causal reasoning improved student work in complementary ways. The study involved more than 1,000 first-year undergraduate students who worked on a real-world business case developing marketing recommendations for the university’s merchandise store. Students were randomly assigned to one of four groups that received access to ChatGPT (GPT‑4o), training in causal reasoning, both, or neither. The results showed that ChatGPT access improved the quality and coherence of students’ work, while causal reasoning training led students to generate more unique ideas.
Students who had access to ChatGPT scored almost a full point higher on a five-point scale. Their answers included more ideas, followed clearer logic, and were more similar to recommendations written by experts. Importantly, students were not simply handing over their assignments to ChatGPT. They still had to decide what to ask, evaluate the responses, and choose what went into their final submission. The critical-thinking exercise produced a more unexpected result. Students who completed the exercise explained more clearly why their ideas might work and when they might fail, but did not score higher on the grading rubric, which only measured how well the recommendations addressed two standard marketing goals.
The study highlights the importance of a holistic approach in assessing student progress. As AI makes it easier for students to produce polished answers, assignments may need to adapt to measure other desired qualities such as originality. AI helped students close an expertise gap, while critical-thinking training encouraged them to develop a wider range of original ideas, question assumptions, and explain why their ideas should work. The experiment’s randomized design allowed researchers to dig into these different factors, separating the effects of ChatGPT access and the critical-thinking exercise from the effect of combining them. This makes the experiment a particularly useful contribution to a rapidly growing body of research on the impact of AI on students and how to best structure and support their learning.
Source: openai
