| | November 20208CIOReviewBy Dr. Richard J. Novak, Vice President for Continuing Studies and Distance Education, Rutgers UniversityTHE ECOSYSTEM OF LEARNING ANALYTICSHow do we know that a teaching and learning experience has been effective, especially in these pandemic days when nearly everything is in an online format? How do we know that the learners have engaged with the material and with each other? How can we improve instruction and strategies to make the most of educational technology tools to enhance learning? These are all excellent questions and some of the answers lie in the area of learning analytics and educational technology.When we talk about learning analytics, there are many data points that are associated with a course that comprise an ecosystem of learning analytics for that course. Not all are of equal value in answering the aforementioned questions about learning effectiveness. The end of course grade often does not tell us much, or enough, about the effectiveness of the take-away learning from a course. It certainly does provide information about performance scores on graded assessments for a course. Those assessments, likewise, may or may not tell us much about learning effectiveness.Similarly, student-generated course evaluations tell us something about student perceptions of the course, how students thought the course went, how they perceived the effectiveness of instruction, how they assess how much and what they learned from the course, and so on. This is another source of data analytics for the course but it too has several shortcomings. First, these are self-reported data. Second, they are subjective data without an objective verification check. Third, even if we trusted these data, the scope of what they tell us is very limited. Luckily, there are many more options available to us to provide more robust learning analytics that can be useful in improving instruction, increasing learner engagement and expanding effective learning outcomes. When I first became involved with online learning and educational technology, approximately 25 years ago, learner analytics at the course level were really limited. There was a built-in clock in the learning management system that could record the minutes that a student was logged in to the course. Later, this was refined to specific sections of a course. Once it was revealed that students could log in and walk away from their computer and still keep the meter running, a "time out" function was added that would automatically log a student out of a course after a set period of time without keyboard activity within that course. Today, dramatic improvements have been made in this area and more are on the horizon. Educational technology innovations facilitate the collection of more and better data than ever before. We can finely tune our analysis of student activity in a course, down to the level of each individual content item. We can track time spent on task, measure difficulty with a concept or exam question and relate an individual student's progress across an entire class. We can analyze the class across a wide variety of demographic or psychometric variables. Even video viewing has been IN MY OPINION
< Page 7 | Page 9 >