With the help of tools like Canvas New Analytics, faculty can leverage learning management system (LMS) data to hone their instructional techniques and improve their online students' experience. In this piece, we provide an introduction to learning analytics in online higher education and detail some analytics best practices.
What Are Learning Analytics?
In the business world, diagnostic and predictive analytics are crucial for gauging corporate performance and consumer preferences; however, analytics has been slower to take root in higher education (Johri, 2019). Historically, when analytics have been used in this sphere, they have been used to draw conclusions about student life (dining habits, financial aid, etc.). And research has shown that even when analytics are applied to university student success, the implementation can be inconsistent and ad hoc.
The “analytics revolution in higher education,” as some have called it, was made possible by the proliferation of learning management systems in the past few decades. Per Jonathan Gagliardi, the author of The Analytics Revolution in Higher Education, the glut of data that the learning management system provides has led some universities to be “data rich and insight poor” (Gagliardi, 2018). At its best, however, learning management system analytics (learning analytics, for short) look to employ data generated by course engagement to understand students and improve their learning processes (Siemens, 2013). Matthew Pistilli argues that analytics in higher education should accomplish two goals, promoting student success and gaining insights into the learning process (Pistilli, 2019). Promoting student success might involve tracking assignments submitted, discussion posts, and level of engagement. By contrast, questions about the learning process might seem more subjective, aiming to understand how instructors can empower students to better understand the course content.
Regardless of the end goal, analytics must be accurate, relevant, and timely in order to be effective (Gagliardi, 2018). To be maximally useful, data should also be widely available: in the higher education sphere, this means that everybody from institutional researchers to administrators and faculty to students should be able to embrace the insights that emerge from analytics. In addition to generating good, real-time data, learning analytics offer a way of utilizing data that is widely accessible on campus and not sequestered among a few data specialists, which means it is capable of supporting what Gagliardi terms a “culture of data” with campus-wide buy-in and adoption (Gagliardi, 2018).
Promoting Student Success
Find below some best practices for using analytics to promote student success.
Review grades and participation 3-4 weeks into the term to identify opportunities for support.
Relevant and timely data can facilitate student improvement. Per Foster & Siddle, instructors should aim to identify student growth points in the third or fourth week of the term. This is sufficient time for instructors to identify a pattern, and it gives students sufficient time to mobilize their instructors' feedback and improve their performance (and ideally, their grades) in the course (Foster & Siddle, 2019).
Use analytics in tandem with strategies for developing a strong instructor presence in your course.
If you lack a strong instructor presence, students are more likely to react negatively to any performance concerns you might flag (Klein & Hess, 2019). To mitigate this risk, focus on getting to know your students and helping them get to know you, creating a space in which students will not become demoralized (and possibly even drop your course) when you help them to identify growth opportunities. Develop an LMS profile, post video announcements, and so on. When an instructor is already active in a course space, analytics feedback reinforces to students that their instructor cares about their academic growth. Read more about strategies for cultivating presence in the Envision blog Instructor Presence in Online Courses.
Encourage students to participate in the data culture of your course.
As a function of the LMS, data has become democratized, a tool that everyone in your course can use to chart their growth and development. If you are using analytics, you can reduce the chance that your students feel watched by encouraging them to set their own goals using grades, submission dates, and other key data points. In Canvas, late work is marked with a red "Late" icon, and students can arrange their grades by assignment group or module, which can help students to identify patterns in their performance. If you don’t want to talk about data explicitly, you can still help students develop analytics skills by encouraging students to check their grades at regular intervals, referring to previous grades or patterns in your assignment feedback, and using data to identify class strengths and weaknesses.
Gaining Insight Into the Learning Process
Find below some best practices for using analytics to gain insight into learning and, ultimately, improve your course.
Don’t read too much into page views.
There is no way to gauge how deeply someone has engaged with a page. A student with a large number of page views might be returning to a page to study, or they might be having trouble accessing a page or have a sporadic or interrupted workflow. Similarly, a student with a low number of page views might be disengaged, or they might prefer to keep pages open for long durations. We recommend treating page views as a suggestive but inconclusive metric. If, for example, you see that a video announcement had a lot of views and a text announcement had almost none, you might form a hypothesis about the types of announcements your students preferred to see.
Do filter assessments by assignment type to see if any discussions, assignments, or projects have outlier grades.
Review assignments by type to see average grades, paying special attention to outliers. If you notice an assignment that has a particularly low average, you may wish to revisit the assignment and clarify your expectations or provide additional scaffolding or resources for students.
Next Steps
If you are interested in viewing analytics for your online course, ask your program administrator what analytics tools are available in your LMS. For tips and tricks for navigating Canvas' New Analytics specifically, see the companion Envision guide Navigating Canvas New Analytics.
References
Foster, E., & Siddle, R. (2019). The effectiveness of learning analytics for identifying at-risk students in higher education. Assessment & Evaluation in Higher Education, 45(6), 842-854.
Gagliardi, J. S. (2018). The analytics revolution in higher education. In J.S. Gagliardi, A.R. Parnell, and J. Carpenter-Hubin (Eds.), The analytics revolution in higher education: Big data, organizational learning, and student success (pp. 1-14). Stylus Publishing, LLC.
Johri, A. (2019). Absorptive capacity and routines: Understanding barriers to learning analytics adoption in higher education. In J. Lester, C. Klein, A. Johri & H Rangwala (Eds.), Learning analytics in higher education: Current innovations, future potential, and practical applications (pp. 1-19). Routledge.
Klein, C., & Hess, R.M., (2019). Using learning analytics to improve student learning outcomes assessment: Benefits, constraints, & possibilities. In J. Lester, C. Klein, A. Johri & H Rangwala (Eds.), Learning analytics in higher education: Current innovations, future potential, and practical applications (pp. 160-186). Routledge.
Pistilli, M.D. (2019). Data, data everywhere: Implications and considerations. In J. Lester, C. Klein, A. Johri & H Rangwala (Eds.), Learning analytics in higher education: Current innovations, future potential, and practical applications (pp. 160-186). Routledge.
Siemens, G. (2013). Learning analytics: The emergence of a discipline. American Behavioral Scientist, 57(10), 1380–1400.