More Research on RAG and Teaching

This week I would like to share a research article on how AI can be enhanced to reduce hallucinations. These updates can enhance how we and our students can integrate functional AI into teaching and learning. The first article is entitled “Deeper insights into retrieval augmented generation: The role of sufficient context” by Rashtchian and Juan (May 2025). 

The authors remind us that Retrieval augmented generation (RAG) enhances large language models (LLMs) by providing them with relevant external context. For example, when using a RAG system for a question-answer  task, the LLM receives a context that may be a combination of information from multiple sources, such as public webpages, private document corpora, or knowledge graphs. Ideally, the LLM either produces the correct answer or responds with “I don’t know” if certain key information is lacking.

At the center of the study is RAG, a method that improves LLMs by supplying them with external, task-relevant information retrieved from various sources—such as scholarly databases, institutional repositories, or curated knowledge graphs. In higher education contexts, this mechanism could allow faculty and students to engage with AI tools that pull from legitimate, discipline-specific corpora, enhancing instructional activities ranging from writing support to complex problem-solving.

However, the authors found a paradox: while RAG systems generally improve performance, they also reduce the likelihood that a model will abstain when it lacks sufficient information. Instead of replying with “I don’t know,” the AI often generates speculative responses—what we recognize as hallucinations. This is particularly concerning in educational environments, where learners rely on systems for formative feedback, content review, and decision-making guidance.

To address this, the authors propose a selective generation framework. This approach trains AI systems not only to provide correct responses but also to recognize when the context is sufficient—defined as complete, conclusive, and non-contradictory—versus insufficient, where the model should abstain. The framework introduces two key metrics:

• Selective accuracy: the proportion of correct answers among those attempted.

• Coverage: the proportion of questions answered (rather than left unanswered).

From a constructivist learning theory perspective (Bruner, 1966; Vygotsky, 1978), the model’s ability to abstain parallels the idea of recognizing one’s zone of proximal development—knowing the boundary between what is known and what requires further scaffolding. Similarly, metacognition—a foundational concept in self-regulated learning theory (Flavell, 1979)—is mirrored in the AI’s capacity to evaluate whether it has enough information to proceed confidently. Teaching students to critically assess AI outputs, especially in cases of low-context sufficiency, becomes a valuable educational opportunity. 

Integrating selective generation into AI tools used in higher ed can help foster epistemic trust, an essential element of academic culture (Barzilai & Chinn, 2018). Tools that can withhold judgment in the absence of robust information model a more ethical and academically responsible approach to inquiry. For instructors, this translates into more dependable AI-assisted formative assessments, research assistants, and co-creative partners for curriculum development.

Finally this week I would like to summarize a recent article shared by a colleague (thanks Kay!) from Harvard Business Review on “How People are using GenAI in 2025” by Zao-Sanders (April 2025). This may be an interesting read to observe patterns comparing usage from 2024 to 2025. For teaching and learning, it is interesting to see people are using AI more to “Enhance Learning” (moving from #8 to #4). Although using AI for “Personalized Learning” dropped from #9 to #17 in 2025.

References

Rashtchian, C., & Juan, D. (2025, May). Deeper insights into retrieval augmented generation: The role of sufficient context. arXiv preprint. https://research.google/blog/deeper-insights-into-retrieval-augmented-generation-the-role-of-sufficient-context/ 

Barzilai, S., & Chinn, C. A. (2018). An epistemic perspective on the role of epistemic aims and emotions in epistemic cognition. Educational Psychologist, 53(1), 1–20. https://doi.org/10.1080/00461520.2017.1386134

Bruner, J. S. (1966). Toward a theory of instruction. Harvard University Press.

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906

Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.