AI-Powered Instant Textual Feedback on Physiotherapist Student Practical Performance
This study addresses the need for comprehensive feedback in academic physiotherapy programs. Existing methods often fall short of providing coherent feedback using keywords, leaving a gap in evaluating crucial clinical …
Oluwasegun Odesola May 20, 2026 0 views 0 reactions
17/20
4Us Score
Problem Description
This study addresses the need for comprehensive feedback in academic physiotherapy programs. Existing methods often fall short of providing coherent feedback using keywords, leaving a gap in evaluating crucial clinical skills. The challenge of producing timely, high-quality feedback based on keywords that evaluate the systematic approach during an open practical exam is notable, given the scant research in this domain. Providing timely feedback is crucial in promoting academic achievement and student success. However, for multifarious reason (e.g. limited teaching recourses), feedback often arrives too late for leaners to act on the feedback and improves learning. Nonetheless, the challenge of providing prompt and detailed feedback to every student consistently proves to be daunting and repetitive task for tutors and examiners.
Context
The latest development in the area of pre-trained language models has attracted researchers to adopt the use of large-scale pre-trained models such as GPT-2, BART and BERT for feedback generation in an open-ended or complex task. During practical exams, tutors or examiners critically observe students' performance in both subjective and objective assessments, offering structured feedback based on these evaluations. This feedback is organized into three primary categories: 'Good', 'Improvement', and 'Research suggestions'. This feedback mechanism is a staple across all academic institution with health-related programs.
Target Audience
physiotherapist students, tutors, examiners, academic institutions with health-related programs
17/20
4Us Score
⏳ Pending
Validation
Education
Category
4Us Problem Worthiness Score
1️⃣ Unworkable
4/5
80%
The challenge of producing timely, high-quality feedback based on keywords that evaluate the systematic approach during an open practical exam is notable, given the scant research in this domain. Nonetheless, the challenge of providing prompt and detailed feedback to every student consistently proves to be daunting and repetitive task for tutors and examiners. For multifarious reason (e.g. limited teaching recourses), feedback often arrives too late for leaners to act on the feedback and improves learning.
Feedback is an essential part of a student's journey towards academic success. Feedback helps students fully develop at every stage of their education, identify areas of strength and growth, and identify what must be done to meet expectations for their level of instruction. Feedback is information given to a student with the aim of closing the performance gap between present performance and the intended outcome. This feedback mechanism is a staple across all academic institution with health-related programs.
Providing timely feedback is crucial in promoting academic achievement and student success. However, for multifarious reason (e.g. limited teaching recourses), feedback often arrives too late for leaners to act on the feedback and improves learning. The most important component of feedback is that it provides specific information regarding the achievement of learning goals. Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative.
Existing methods often fall short of providing coherent feedback using keywords, leaving a gap in evaluating crucial clinical skills. The challenge of producing timely, high-quality feedback based on keywords that evaluate the systematic approach during an open practical exam is notable, given the scant research in this domain. Consequently, this study aims to bridge this gap by introducing an AI-driven, Keyword-Based system design to access student practical performance, thereby optimizing education for physiotherapists.
Total Score: 17/20
(85% on rubric scale)
— Decision:
✅ ACCEPT - Problem worth solving
Evidence Quality
7.3/10
⭐ Tier 1: 5📊 Tier 2: 0📄 Tier 3: 0💬 Tier 4: 0
Methodology
Utilizing a Seq2seq framework with diverse LSTM and attention mechanisms, iAtexF excels in relevance. The iAtexF feedback generation process is in 3 stages. Stage one involves data preparation. Stage two, Recurrent Neural A network (RNN) sequence-to-sequence-based model is set up and trained on input Fm(X) and feedback Y. At stage three an unsupervised method of content summarization was used on the generated feedback from each category from subjective and objective assessment. RNN is the backbone of Seq2seq learning with other parameters like bidirectional RNN, attention mechanism and beam search. LSTM operate on sequences which implies that an increase in the layer gives more level of abstraction of the observation and timescale. Bidirectional Recurrent Neural Network was introduced for a better performance and generate coherent feedback by ensuring the model can identify the semantic structure between the source text and target text.
This study, approved by the relevant authority of a university, utilized a dataset from the Physiotherapy department of School of health and life Sciences, including both undergraduate and postgraduate students. Initial data comprised 1,100 anonymized student records, deemed insufficient for robust analysis. Consequently, senior therapists from various institutions contributed additional feedback based on the same template, enlarging the dataset to 32,616 entries. Combined with 12 ChatGPT-generated responses from OpenAI, the total dataset reached 32,628 entries across five categories: Keywords, Improvement, Read-more, and Feedback. Further analysis of the initial 1,100 records resulted in a condensed 5-category dataset with summary feedback.
Methodology Diagram
64636407_3-Figure2-1.png
Resources & Links
Results & Findings
iAtexF excels in relevance, achieving a high similarity score of 73% and a ROUGE score of 34%. It surpasses both ChatGPT and experts in providing relevant suggestions (80.7%), maintaining an appropriate tone (86.7%), and ensuring a logical structure order (100%). User experience evaluation of the iAtexF web application yielded a favourable 92% rating, indicating its usability and effectiveness. Bidirectional with attention has 81% accuracy compared to ordinary Bidirectional and by employing human evaluation it was observed that the model has a coherent flow and potentially recognizing expected keywords from the input. The average feedback word length from senior physiotherapists and ChatGPT feedback are 119 and 104 words respectively while the average tokens are 135 and 122.
Key Findings
['iAtexF achieves a high similarity score of 73% and a ROUGE score of 34%', 'It surpasses both ChatGPT and experts in providing relevant suggestions (80.7%), maintaining an appropriate tone (86.7%), and ensuring a logical structure order (100%)', 'User experience evaluation of the iAtexF web application yielded a favourable 92% rating', 'Bidirectional with attention has 81% accuracy compared to ordinary Bidirectional', 'iAtexF outperforms instructor feedback in structure by 100% to 93%']
Limitations
Results indicate iAtexF's proficiency in generating coherent and concise feedback, though it may overlook context-specific problems. Despite suboptimal text generation speed, the iAtexF web application received a 92% rating for user experience from a web developer expert.
Validation Status
Current Status
🏛️ Peer Reviewed
— IEEE
Method Selected:🏛️ Publisher Verified — DataIntell validation not required
What this means
🏛️ This research has been externally peer reviewed
✅ DataIntell validation was not required and has not been applied
🎓 Your hard-earned peer-review status is fully respected and displayed
No comments yet. Be the first to share your thoughts!
S
Oluwasegun Odesola
@segreen
I'm a researcher and data scientist at the intersection of artificial intelligence, algorithmic fairness, Financial and educational technology, with a …
Discussion (0)
No comments yet. Be the first to share your thoughts!