In late 2017, the Learning Enhancement & Innovation team were invited to investigate a high failure rate (>20%) course at ECMS and make some design recommendations for possible adjustments and/or interventions with the aim of improving the pass rate and enhancing the overall student learning experience.
The LEI team began by looking at Learning Analytics within MyUni and beyond. As expected, a correlation was found between online engagement in the LMS (student views and participations of content and activity) and final grade. Of more concern however, was a further correlation between low performance in assignments prior to census date and overall performance in the course. These findings were presented in a report to the new course co-ordinator with recommendations for possible changes.
The new course co-ordinator – Dr Braden Phillips – adopted the LEI recommendations and used Learning Analytics in the next instance of the course (S1, 2018) to identify students likely to be at risk.
First, a list of 12 students who had not logged onto MyUni by Week 2 was created and contact was made by email. Following this intervention, 11 had logged in by the end of week 3 with one having withdrawn. Six of the remaining 11 students are now back on track, with 5 still at risk.
Additionally, a list of 63 students who had failed the first two assignments (quizzes) prior to census date, was compiled using learning analytics and an email sent to Succeed@Adelaide who made calls to these students. Of these, 33 (>50%) are now back on track, with 19 needing further intervention. There was insufficient data on 11 others to evaluate the impact of intervention.
Dr Phillips initiated a number of other changes to content and activities. Where Analytics had shown that course content had not been widely accessed in S2, 2017, he re-worked it into multiple formats (short key concept videos and slides). Data from LMS Analytics (left course menu in MyUni) in S1, 2018 shows that the slide presentations have been accessed more widely than the key concept videos (with the same slides), with access to content improved overall.
In response to SELTS feedback, Dr Phillips also made adjustments to the timing of delivery of lectures, tutorials and workshops to optimise the constructive alignment of content, activities and assessments. He also adjusted formative quizzes so that students only needed to re-take questions that they had answered incorrectly, rather than re-doing the whole quiz in any subsequent attempts.
A mid-term survey found that >80 % of students were satisfied with the course. Learning Design/Analytics staff involved in the investigation – John Murphy, Marziah Zarazillah, Daniel Barry and Course Co-ordinator Dr Braden Phillips – await the end of Semester, when the effect of the analysis, recommendations and interventions can be evaluated more fully.