On Analytics Going Away

A packed house of higher ed innovators at our annual Partner Summit this month.

When it comes to the seemingly ever-present articles and conference conversations on analytics in education, there are educators, policy makers, and non-profit leaders muttering some version of “Please make it go away!” I’m one of them.

However, I mean “go away” differently than you might think—and I don’t think I’m alone. What many dedicated educators are looking forward to is not a time when we turn the lights back off and “bowl in the dark,” as Melinda Gates cautions against. Rather it’s about moving toward a time when analytics tools and the infrastructures upon which they are built are so common that they neither dominate the dialogue nor are positioned as the ultimate answer to education’s challenges.

Going away for analytics in this sense allows us to turn our focus more fully to our missions, learning strategies, and the many ways in which people, processes, and technology can be applied to our various educational causes. Analytics conversations simply blend into the background and the tools and infrastructure power the larger work behind the scenes. Much like we don’t wax poetic about the amazing innovation that powers our lights in our colleges and universities—and trust me, electrical engineers will tell you, light bulbs and power grids are a thing of beauty and ridiculously important for how we live, learn, play and work. Rather, we are more concerned with the expansion of innovative evening classes, the safety of parking lots, the championship football game played at 8:00 pm on Saturday. Power and lights may make all these possible, but they are decidedly not the focus.

In this sense, we’re getting closer to making analytics “go away”. For example, this spring Civitas Learning formally launched Illume Impact. This module is shining a light not on itself, but on the effectiveness of everything from flipped-classroom initiatives to orientation programs to tutoring services to targeted strategic outreach. As Impact rolls out, we’re watching Civitas Learning’s partner colleges and universities explore the persistence gains of an array of programs and processes, revealing often promising and sometimes problematic empirical outcomes. Moreover, powered by their Student Insights Engines, our partners are launching and learning about the effectiveness of a host of other efforts to help support their students, including:

  • Institution-wide retention campaigns at University of South Florida that have increased first-year persistence rates to historic highs, supported double-digit gains in graduation rates, and close the achievement gaps between diverse students–and unlocked millions in performance-based funding in the process.
  • Mindset messaging campaigns at Lone Star College aimed at high-GPA students at high-risk of leaving are seeing up to 9.5% persistence gains.
  • Opt-in, online tutoring registered 5% persistence gains for first-year and African American students.
  • Degree Map and College Scheduler apps, where institutions are seeing 3-11% persistence gains across the board. Interestingly, the Degree Map outcomes are particularly strong for struggling students (up to 19% persistence lifts for the highest risk students), and College Scheduler is even more impactful with stronger students, especially in STEM. It’s one of the reasons we’re combining these apps in our personalized pathways work in the coming year.
  • On-ground tutoring centers having an impact of 5.5% persistence gains and their online tutoring centers having 6.9% persistence bumps.
  • Greek life and residential life programs showing significant positive impact on persistence.
  • Finally, this New York Times story features the work of University of Arizona, El Paso Community College, and Sinclair College, where they found much more powerful graduation signal in courses that most students passed like writing and intro to biology. Now they are diving into work to nudge students to writing centers and/or possible course and pathway redesign.

We saw even more of these efforts and outcomes featured by partner colleges and universities at our annual Civitas Learning Summit a few weeks ago, where almost 300 leaders from across the US and UK came to Austin to learn from each other’s good work. We were joined by Vince Tinto, George Siemens, Sally Johnstone, and the Bill & Melinda Gates Postsecondary Director, Dan Greenstein, all reinforcing the idea that it’s time to put the data to work. The partner presentations were particularly inspiring—especially the students discussing how strategic outreach at the right time, in the right way, made all the difference in their decisions to stay and succeed. However, my key take away was the theme of the event: Take the Next Step. The Civitas Learning community seemed intent on challenging each other to get beyond data gazing and into using their infrastructures to power and test student success innovation at scale.

Our partners were arguing that it’s time to change our analytics-centric conversations. The better conversations are about how to best leverage student-facing apps to improve outcomes; they’re about how to organize

Dozens of partners sessions at Summit shared outcomes and success stories.

internally so we can quickly identify and effectively nudge the most in-need students to access vital college services like tutoring or financial aid; they’re about the importance of and the word choice in mindset messaging; they’re about remembering high-GPA students have psycho-social and life-and-logistics issues too, and how reaching out to them in the right way can significantly impact success rates; they’re about course and pathway redesign; they’re about balancing our focus on access, equity, and completion. They’re about change management and shifting a college systems, process and culture to actually use operational data.

In short, the lights are turning on and folks are happy about it; but, in the end, the illumination should not be the focus. It’s the issues, innovations, and impact that matter most. Moreover, shining a light often means we as educators need to do the work of theorizing and testing (quantitatively and qualitatively) why these signals are emerging in the data. As Vince Tinto said at our summit, “you can’t outsource insight.” We need to come together and synthesize, analyze, and then decide what to do about it!

Of course, if the lights go out, people really begin to care about power grids and light bulbs. And I don’t want to demean of the incredibly hard work of data engineering, derivative variable creation, model building, machine learning, model refresh, and daily scoring of students in our context. Doing this infrastructure work the right way is not easy—and we’ve chosen to do this challenging work the right way so that the more-important efforts of education innovators can be easier, especially for those on the front lines of learning—i.e., students, advisors, and faculty. In fact, in this core infrastructure development at Civitas Learning, we owe a lot to our pioneer institutions. They helped us lead through this often-frustrating, first-time build out. But as you can see by the outcomes listed above, the results of that hard work are coming together in exciting and impactful ways.

Analytics haven’t ‘gone away’ yet. But we can already see the shift happening—thankfully. It’s going to be exciting to watch the fading, if for no other reason than to see the new and more compelling innovations build on this infrastructure in the months and years to come, and to learn together about how to best inspire, support, and celebrate our striving students on their pathways to and through higher education.

Post a Comment

Your email address will not be published. Required fields are marked *

Top