ACM TOCE: A method to analyse computer science students’ teamwork in online collaborative learning environments

CSER is pleased to announce that our work on developing a teamwork learning analytics dashboard for analysing and assessment online collaborations for Computer Science and Software Engineering students has been accepted for publication in ACM Transactions on Computing Education.

Our Teamwork Analysis Dashboard builds upon earlier work where we developed a qualitative analysis framework for analysing students’ online collaborations. The idea of the dashboard is to produce an easy to use, and scalable, visualisation system to aid lecturers in managing courses with teamwork. The dashboard is designed to assist teaching by analysing online team discussions to identify what kinds of roles individuals are taking on in the discussion, i.e. leadership, support, coordination, etc, and also the sentiment expressed in the discussion, i.e. frustration, anger, joy, concern. This automated analysis, based on NLP and text analysis techniques, will assist teachers by helping them quickly identify teams where progress is not being made, where required roles are not being undertaken by the team members, or where discussion has reached a point where it needs assistance from the teaching staff.

CSER has been exploring different mechanisms for assessing and analysing teamwork over the past few years. We have explored the use of Contributing Student Pedagogy, and peer assessment within our first year courses as a way of integrating collaboration with more explicit understanding of why collaboration is there. (You can read more about this here and here).

More recently, however, we have been developing an online collaboration assessment framework based on Dickinson and McIntryre’s teamwork model. We selected the teamwork roles defined by Dickinson and McIntyre because the teamwork model focuses on practical roles of self-organising teams. Dickinson and McIntyre classified seven core components of teamwork: Team Leadership, Team Orientation, Monitoring, Coordination, Communication, Feedback and Backup Behaviour and provided with their model a guide to measure performance. Our manual classification framework has proved to be a good basis for the development of an automated analysis framework that incorporates team role classification of online team discussions, as well as classification of the emotional climate, using sentiment analysis, of the team discussion.

Rebecca Vivian, Katrina Falkner, Nick Falkner and Hamid Tarmazdi. “A method to analyse computer science students’ teamwork in online collaborative learning environments”. Accepted for ACM Transactions in Computing Education. (To appear).

This entry was posted in Active Learning, CS Analytics, Publications, Research, Software Engineering. Bookmark the permalink.

Comments are closed.