Ed Chatbots

There has been much written about automation in higher ed over the years, at times supported by related research. Many of us use automated technology in various forms in our daily lives, in particular chatbots (“computer program that simulates and processes human conversation allowing humans to interact with digital devices as if they were communicating with a real person.”). This week I would like to share a recent article on this topic entitled, “How can an educational chatbot’s feedback influence human attention?” by Sylvester and Soekadar (2024).

The authors ask “how does feedback of an ed chatbot impact a learner's attention?” The study measures changes in human attention when learning by using an educational chatbot to improve human reasoning ability. During the interaction, the attention of participants was measured using an EEG.”

The study reminds us that there are “qualitative and quantitative measurement methods of attention. The focus was on quantitative methods that can be categorized into indirect and direct types. The indirect measurement relies on observing an individual, e.g., body movement and eye-tracking (Mancas & Ferrera, 2016). The direct measurement method is based on physiological responses of individuals, e.g., brain signals. The human brain runs our body functions and our thinking. When the brain tells our body what to do, neurons are activated, producing a current flow (Teplan, 2002). To measure electrical brain activity, an EEG can be applied.”

The study revealed three findings:

A significant attention effect occurred 54% of the time, after the chatbot showed feedback.
Positive feedback had a significant effect in 52% cases and negative feedback in 48%.
No significant difference was found in attention (at 0.05) during the 10 seconds before, and 10 seconds after positive feedback was received. Similar is the case for negative feedback.”

A bit contrary to common belief, the data did seem to indicate that a decrease in attention occurred 73% of the times, when a participant received positive feedback. This shows that the positive feedback of a chatbot may decrease human attention and perhaps reconsider its use.

So, overall, the results were inconclusive. When it comes to finding a reason for why, it is important to keep in mind that sometimes, feedback simply has no effect and human attention has no fixed state, as it can fluctuate at any time (Bunce et al., 2010).

Note: Call for Special Issue: The open-access journal "Glokalde" is pioneering a special issue dedicated to the transformative potential of generative AI (GenAI) in higher education. The journal is seeking your unique voice, whether you're a seasoned researcher investigating GenAI's impact or an educator actively implementing it in your classroom. Please do share your insights, practical approaches, or research findings to contribute to this vivid dialogue and shape the future of learning with our team, Nil Goksel, Ayse Taskiran, Hüseyin Altunlu, and Jace Hargis.

You may send your articles to glokalde.editor@gmail.com

Date of submissions: July 1, 2024

Reviews: July 2-14, 2024

Revision of Papers: July 15-30, 20124

Special Edition Published: August 15, 2024

References

Sylvester, F., Le, T. & Soekadar, S. (2024). How can an educational chatbot’s feedback influence human attention? Research and Practice in Technology Enhanced Learning,19, 7. https://doi.org/10.58459/rptel.2024.19007