Correlation Analysis of Conversational Focus with Learning Experience in Python Courses

Abstract

Students interact with AI tutors in very different ways — some drill deep into a single topic, others explore broadly. We propose a quantitative framework for semantic divergence in tutor conversations and show that divergence has differentiated, sometimes opposite effects across learning outcomes. The findings argue against one-size-fits-all tutor guidance.

Problem & Motivation

Students interact with AI tutors in markedly different ways: some drill deep into a single topic, building cumulative depth, while others explore broadly across topics, building breadth. Existing AI-in-education work has focused on output quality and aggregate performance, while paying little quantitative attention to student questioning patterns. The deeper open question is how the depth and divergence of student questioning relate to motivation, project application, knowledge transfer, and self-regulated learning.

Method

We collected complete student–UeduGPTs dialogue logs from a Python programming course and built a computational framework to quantify questioning patterns: (1) semantic vectorization via OpenAI embeddings (1,536-dim, with a text-description bridge layer for code); (2) core topic vector extraction via weighted-mean pooling and cosine retrieval for interpretable semantic anchors; (3) the semantic divergence index D_PR, computed as the minimum number of principal components needed to explain 90% of semantic variance (higher = more divergent). Students were stratified by question count (low: N < 25; high: N > 25), and we ran Pearson correlations and regressions against four psychological measures.

Findings

• Intrinsic motivation (positive): low-N group r = 0.61, p < 0.001; high-N group r = 0.35, p = 0.029.
• Project application (positive): high-N group r = 0.41, p = 0.014.
• Knowledge transfer (negative): low-N group r = −0.31, p = 0.066; high-N group r = −0.35, p = 0.057.
• Self-regulated learning (conditional): low-N group r = −0.33 (p = 0.048) but high-N group r = 0.34 (p = 0.027) — divergence helps only when question volume is sufficient.

Implications

Semantic divergence plays a dual role: it can expand breadth and engagement, yet it can also impair knowledge integration. AI-tutor guidance strategies should not be one-size-fits-all — students with low question volume should be encouraged to focus and deepen, while students with high question volume benefit from exploring divergent perspectives. D_PR offers a quantitative lever for instructors to design more targeted AI-assisted teaching.

Citation

@inproceedings{yen2025conversation_focus,
  author    = {Chia-Chien Yen and Yan-Yu Chang and Yu-Zhen Chai and Po-Tang Hsieh and Kuei-Hao Li and Chia-Kai Chang},
  title     = {Correlation Analysis of Conversational Focus with Learning Experience in {Python} Courses},
  booktitle = {Proc. Int. Conf. on Modern Educational Technology (ICMET)},
  pages     = {258--262},
  year      = {2025},
  month     = dec,
  doi       = {10.1109/ICMET67594.2025.11451926},
}