Developing AI-Based Automated Post-Rating System to Scaffold Interdisciplinary Knowledge-Sharing

Abstract

Asynchronous online discussion forums foster cross-disciplinary knowledge sharing, but instructor feedback rarely scales to large posting volumes. We deploy a BERT-based automated post-rating system that classifies student posts as Informative, Neutral, or Non-informative within 10 seconds of submission. The real-time scaffold reduces plagiarism, improves post depth, and lifts students' reflective engagement across two courses.

Problem & Motivation

Asynchronous online discussion forums are an effective medium for cross-disciplinary knowledge sharing, but when post volume grows the instructor cannot review every entry quickly enough to provide useful feedback. Students post without any immediate quality signal, which produces a familiar pattern of low-effort, completion-driven submissions and uneven discussion quality.

Method

We developed a BERT-based post-rating system that classifies each newly submitted post within 10 seconds into three quality tiers — Informative, Neutral, or Non-informative. The system was tuned to three discussion types: top-level Discussion (67% accuracy), Comment (68%), and Reply (75%). It was deployed and evaluated in Python Programming and Introduction to Artificial Intelligence undergraduate courses using survey and behavioral measures.

Findings

• Students rated the system positively as a quality scaffold for their own posts.
• Plagiarism declined noticeably; students were more likely to compose original content.
• Reflective depth of posts improved across the semester.
• Real-time AI feedback before submission lowered the burden of instructor post-hoc moderation.
• Open-response feedback indicated that the automatic rating motivated more careful contributions.

Implications

Automated post rating is not a replacement for instructor evaluation — it is an immediate scaffold that lets students self-check at the moment of posting. This changes forum culture from passive assignment completion toward active knowledge sharing, which is particularly valuable in cross-disciplinary general-education contexts where students from different backgrounds need lightweight guidance to contribute substantively.

Citation

@inproceedings{chang2023post_rating,
  author    = {Chia-Kai Chang and Po-Chao Chen and Zhao-Shun Chen and T. M. Kuo},
  title     = {Developing {AI}-Based Automated Post-Rating System to Scaffold Interdisciplinary Knowledge-Sharing},
  booktitle = {Proc. IEEE Int. Conf. on Advanced Learning Technologies (ICALT)},
  pages     = {239--241},
  year      = {2023},
  month     = jul,
  doi       = {10.1109/ICALT58122.2023.00076},
}