As a Leader College with Achieving the Dream, Montgomery County Community College has immersed itself in data-informed initiatives for several years. It was also among the earlier joiners to the Civitas Learning community. Today, they are seeing positive outcomes from this work at all corners of the college.
Montgomery County Community College (MCCC) is using data teams called DIAL (Data-Insight-Action-Learning) Working Groups, and Civitas Learning’s signature app, Illume® to measurably improve student outcomes. What follows are examples of their DIAL Working Group’s use of analytics to understand the needs of their students and the impact of their college’s initiatives to support them.
DIAL Working Groups
To learn more about MCCC’s DIAL Working Group we visited with Dr. David Kowalski, recently-named executive director of Institutional Research. Kowalski comes to the work from his previous role as a psychology professor and educational researcher. He was attracted to the position because of the IR’s office changing role from reporting to analytics. “It was exciting to me to be part of an IR office that could play a more dynamic and integral role instead of being solely a reporting function,” he said.
“The DIAL Working Groups are interesting by design, in that the team members go across all departments – from advising to our online campus, from faculty to financial aid, business intelligence, enrollment management and marketing,” Kowalski said. “Working in educational research for the past several years I have found that, in general, teams that convene don’t make decisions at the end of the meeting. It’s very different with our DIAL Working Group – it’s an action-oriented group of individuals empowered to make decisions.”
Adding Courses for Faster Completion
One of the decisions they recently made was around the subject of course load. They used Illume to explore student populations of high likelihood to persist, but taking less than a full course load. “Our goal is to help students succeed in a timely manner, so we began to explore the data to find a way for them to progress at a rate most beneficial to them. For those the data showed were not at risk, we suggested adding a course, and there was a high level of academic success associated with it. We used Illume to identify the cut-off level,” said Kowalski.
“Illume allowed us to come into the course load conversation with a hypothesis,” said Angela Polec, DIAL working group member and executive director of marketing and communications for the college. “We were able to intervene without flying blind. We had the data to know which students to reach out to that would likely benefit the most without putting them at risk. That was exceptionally helpful.”
Analytics in iPASS Grant Work
A significant college-wide focus in analytics this year is with an iPASS grant (Integrated Planning and Advising Systems) from the Bill & Melinda Gates Foundation. “Our work with Civitas Learning is going to be very helpful in our iPASS grant work because we want to ensure students have a similar experience across the cohort, and then we want to measure and watch results in real-time,” said Kowlaski. “The first-year experience, intensive advising, working with faculty to provide early semester feedback – it will all figure into the mix.”
Celeste Schwartz, Vice President for Information Technology and College Services concurs. “When we think about the work with iPASS, there’s not a single meeting with our core or extended team when the data from Civitas is not important,” said Schwartz. “Our use of it has evolved so much over the last nine months – we’re really using the tool. But to make the most of it, as we are now, you really have to have the right team in place, and the right leadership to support it. I can’t imagine not having it now. Our work with the iPASS grant is challenging, but we are finding a lot of different ways to use it.” Schwartz credits much of the success to Kowalski’s leadership and research expertise, an asset in helping to direct the focus of the DIAL Working Group.
They are using the predictions in Illume to identify who among the students in the iPASS experiment would benefit most from the test of ‘invasive’ or attentive advising. “We are using random control trials in our work with the Foundation and the Community College Research Center (CCRC) to evaluate the efficacy of our attentive advising model. For our selection of students we are using the Civitas Learning predictions of likelihood to persist. As moderate and moderate-to-low likelihood to persist are potentially most receptive as well as most likely to benefit, we want to test with that group,” said Kowalski. “That we can identify these students on day one of the semester with Illume is critical, then we can just run with the intervention and watch as their likelihood to persist changes over time.”
Testing the Impact of Tutoring Services
In addition to advising work, MCCC has scaffolded support services including free tutoring for all students. To better understand the efficacy of that service, they employed Illume Impact to test persistence changes among those using the services as compared to those who did not.
Using Illume Impact and prediction-based propensity score matching (PPSM), they matched 4,500 pairs of students to measure the effectiveness of the tutoring center. They found a 6.2 percentage point increase in persistence overall for students who used the tutoring center from Summer 2014- Fall 2016. The lift was even greater for students starting their first term, at 9.8 percentage points. And, isolating for only STEM students, they found an 8.6 percentage point increase in persistence for students using the tutoring center.
Civitas Learning uses PPSM to allow institutions to, in the absence of randomization, use existing learning data to create control groups that most closely match the pilot or test group being studied. The more closely the control group is matched to the test group, the more statistically accurate the analysis becomes. The most recent iPASS literature lists PPSM as an accepted ‘rigorous’ standard.
Reaching Persistence Goals
To reach their academic and fiscal goals, MCCC knows they must achieve a 75 percent term-over-term persistence rate, and armed with the knowledge of the efficacy of their tutoring services, the importance of advising students to take a term credit loads associated with academic success, and the benefits of attentive advising, they are prepared to lean into the conversations that matter.
“Everyone from advising, to admissions to marketing is using Illume insights,” said Kowalski. It’s proving useful in helping with another trend in colleges of universities – high GPA departure. Supporting the evidence Civitas Learning uncovered in its Community Insights Report, it’s not just academic failure that cause students to leave. In fact, more students are leaving higher education with GPAs above 2.0 than below it. And, MCCC is no exception with 73.7 percent of the non-persisting students having GPAs at or above 2.0 and 37.2 percent of non-persisters had GPAs at 3.0 or above in the last year. “We reached out to a significant group of students who did not persist last year and asked them why,” said Kowalski. “It wasn’t academic. For the majority – about 70 percent – it was life logistics. And, based on our survey findings, it’s not typically just one thing, it’s a process. We asked the ones we didn’t see as being at risk if it would have made a difference if we knew, and reached out before they left. They almost overwhelmingly said yes.”
Kowlaski says use of Civitas Learning solutions now empowers his team identify individual students who are at risk before the students realize they need help, allowing his team to reach out before it’s too late. “We cannot count on the students to always be calibrated to their learning needs and to understand how to ask for the resources available them,” he said. “About 40% of the students we surveyed said they were overwhelmed with their courses, but less than half reached out for help.”
“The thing I am excited about is, with Illume our DIAL Working Group is empowered to do targeted outreach to understand what factors outside of GPA are making a difference, and act accordingly ,” Kowalski said. “That outreach is making a difference in the success of our students.”
David recently shared some of the DIAL Working Group’s findings with the Board of Trustees who were very receptive to the work of the DIAL Working Groups and interested in learning more. “I think back to our first Summit where we discovered the insight around time to enroll and risk for persistence,” said Schwartz. “We were so excited – that was so big. Now it’s just one of so many discoveries as this team continues to dive in. They come curious and say: ‘Let’s see what else this can do.’ The team pushes us. Now we have so many examples we can share out.”