Some Student Success Initiatives Aren’t Working (Here’s How to Make Sure Yours Are)
By Civitas Learning
It’s no secret that higher ed is changing. With 74% of students now considered “non-traditional” (U.S. Department of Education), meeting students’ unique needs is essential. Yet spending per student has dropped by 25% since 2007 (Education Next, 2018).
After working with hundreds of higher education student success and assessment leaders and helping evaluate more than 1,000 student success initiatives, we’ve learned there’s a better way to know which student success initiatives are working (and for whom), and how to use that insight to drive institutional improvement.
Institutions successfully moving the needle on student outcomes are doing something different – and it starts with comprehensively evaluating their student support initiatives and beginning to improve them iteratively and intelligently.
Step 1: Know what exists today
It’s important to begin with a thorough audit of what initiatives exist today. We have worked with many higher education leaders who aren’t quite sure what student success initiatives exist across their institution. Many can list the most obvious or visible, like the tutoring center, or those closely related to their own work; however, when they set out on a mission to understand the full scope of initiatives across all departments on campus, they’re often surprised by how many they uncover.
Initiatives can range from ongoing services and support programs to more sporadic, one-off resources and events. Examples might include counseling or mental health services, guest speaker events, or a student job fair.
When you begin your audit, you might start by conducting interviews with department heads, deans, career center leaders, students, advisors, faculty members, etc. Throughout the process, document what you learn. Make sure you uncover the following:
What is the initiative designed to do?
Who is the initiative designed for?
What department, person or team oversees the initiative?
Who is eligible to participate in the initiative?
Does it potentially overlap with another initiative?
Step 2: Understand how effectively an initiative fulfills its objective
Take a look at your initiative inventory and select a handful of initiatives that warrant further investigation. Some of our most successful customers choose to hone in on services that are either clearly duplicative, services that require evidence of outcomes based on funding sources, or services that require significant investment each year.
During this step, answer questions like:
Who is most taking advantage of this service?
How much do different student groups benefit from this service? (consider quantitative and qualitative evidence)
Are the benefits we see clearly linked to the outcome the service intends to achieve? If no, why not?
Once you have decided where to focus, and completed your preliminary discovery, it’s time to ensure you have enough information to take action.
During this step, consider more nuanced questions like:
Where overlap exists, is it intentional or simply duplicative?
Would there be a benefit to combining duplicative efforts after better understanding relative impact of each?
What is the “theory of change” of this program… why would we assume it achieves these outcomes?
If Student Learning Outcomes (SLOs) are the focus, do they have any logical or evidence-based relationship to persistence or are they geared toward some other outcome of value to the institution?
How has this initiative changed over time, and has that been in response to data of some kind?
Does the initiative have any special cultural, historical, or emotional significance to the campus community, and why?
Is there a relationship between what the initiative means to people and what it achieves?
Most importantly … what should we do now?
Some institutions have used the information they gather to change policy or seek additional funding for certain services, while others have sent highly targeted nudges to students most likely to benefit. For example, University of Missouri-Kansas City updated the required number of supplemental instruction visits, and Monroe Community College sent personalized messages to students shown most likely to benefit from their Center for Academic Reading.
The key here: with the right level and kind of intelligence, it becomes possible to focus your efforts and begin the process of iterative improvement.
Every student success initiative was created by someone with the best of intentions. As we seek to understand and improve the effectiveness of these interventions, we need to appreciate them in context. Why did they begin, how have they evolved and changed, what challenges do they face today, and upon what concepts are they based?
We find that a process like appreciative inquiry, aided by great measurement, helps ground our investigation of efficacy in strengths and creates a safe space for improvement. Almost every program is effective for someone… so how can we value the intention of the initiative, recognize and improve upon the pieces that work, and rethink what doesn’t?
Want to take the first step?
Download this helpful inventory template to guide you in your discovery.
Tina Donahoo is a Principal Strategic Consultant of Outcomes at Civitas Learning, where she works with colleges and universities on the use of advanced analytics for assessment and optimization of student success initiatives and financial aid allocation. Utilizing Civitas’ Impact software tool, Tina helps institutions perform rigorous efficacy measurement of student success initiatives to optimize finite resources, personalize student support and demonstrate the impact of student programs to donors, board members, institutional leaders, and the community.
Dr. Angela Baldasare, Senior Principal Strategic Consultant at Civitas Learning, is an experienced researcher, analyst, and consultant with a demonstrated history of success in data strategy and change management, helping organizations of all kinds use data to inform strategy and achieve measurable outcomes. Angela’s work at Civitas Learning is informed most recently by her eight years at the University of Arizona (UA). From 2014-2018 Angela served as the assistant provost for institutional research, leading the university’s student data strategy, including the use of Civitas Learning analytics, and helping to achieve record high retention and graduation rates for the institution as well as a significant rise in US News and World Report rankings.