We’ve all been there. In the meeting with the executive team and the reports are being presented for a first view. The numbers don’t match. One person sees the enrollment numbers and nods yes. Another looks and says, “These data are wrong. These numbers are not right and can’t be trusted!” Discussion and disagreement ensue and the clock spins.
“In the end, it may well be that neither number was wrong, the people reporting just were not speaking the same language,” explains Jesse Coraggio, Associate Vice President for Institutional Effectiveness, Research and Grants at St. Petersburg College. “One set may have included dual enrollment, or excluded online. The thing to remember in this work is transparency and collaboration. Once the data is out there where everyone can see it and engage with it. It has no power to surprise, and true information can surface.”
So how does a college get to this place of transparency?
Focus on the Culture
“Focus on the culture. That’s where we got the biggest bang for the buck,” said Coraggio. “It starts with values. When you are clear on the values, it makes the conversations easier. At St. Petersburg College, our key value is student success. We live it. Every conversation we have is about student success. If there is a disagreement in the conversation, we come back to what’s best for students.
St. Petersburg College charted a course the last four years that has many inquiring for a blueprint to emulate. “We get asked often about how to build a data culture,” said Coraggio. “We didn’t have an actual blueprint when we started; we had President Bill Law’s intense focus on student success and vision for creating a transparent culture of collaboration. Coraggio says the clarity of that vision has led to a college that is “rowing in the same direction,” and collectively improving the student experience.
The Power of Transparency
The college had committed to data transparency where any faculty or staff member can see budgeting information and other data across the college. “With transparency comes mutual respect. We’ve created a culture where it’s safe, even encouraged, to ask questions. Faculty, staff and administrators often question the data or direction, and in doing so, learn more collectively. We also value diversity. We are mindful to beyond the importance of a diverse population to also strive for a diversity of ideas and opinions,” he said.
“When we start new initiatives, it’s not a scenario where an administrator returns from a conference and says, ‘Here’s a great idea, let’s do this”’ in a top-down manner. We gather the team and ask them to look at the idea and the data that supports it, and we discuss if it’s a good fit for our students. If it is, we also discuss together how to implement it so everyone feels a level of empowerment and ownership,” he said.
Moving Away From a Culture of Blame
Coraggio is quick to caution that a data culture is not the same thing as a measurement culture. “You have to be careful not to just implement a measurement culture. We know what gets measured gets managed,” he said. “When you set a firm, rigid target and enforce high accountability, you can achieve that target, but generally in a culture of blame, and at the considerable expense of other valuable outcomes. In the standard performance metrics example, you see groups put a ton of money and effort into getting to a very specific result, but not really having the overarching outcomes they needed to improve the situation. Achieving the metric does not necessarily improve student success. Data should not be the end of the conversation; it is just the place to start.”
Originally joining St Petersburg College nine years ago as an assessment counselor, Coraggio is excited to see the changing model in the work he and the team at St. Petersburg College are doing within the IR office. “We see ourselves as a consulting office within the institution. In the old model, someone would come with a request and we would run the query and get back to them in about two weeks based on our work load. We realized that the way they posed the question may not actually get them the answer they needed. It was frustrating,” he said. Today, his IR team gets out of the office, and works directly with faculty and staff. “It has to be collaborative and interactive to make change. We go to the deans, provosts, etc., and have meaningful conversations, asking about what they need to know, and we help them determine what we can do to help them most.”
Real-time vs. Autopsy Data
For all of this to succeed, Coraggio says there must be real-time data. “So much of what we used to use in IR was autopsy data. It was delivered and disseminated too late to help any particular student. With real-time data in a transparent data culture we can all see what’s happening and act in the right way – at the right time – to truly help students during that semester before it’s too late. When you work with real-time data, you can see trends and nuances in a way you can’t with flat outcomes data. It makes it more actionable. If we’re really going to move the needle on student success we have to be working with real-time data,” said Coraggio.
The Road Ahead
“We have a long way to go before we get to where we want to be,” he said. “The funny thing about this work is, the more you succeed the more you realize how much you do not know. There’s so much to do. We’re running – but we’re all running in the right direction, and we’re running together.”
Coraggio recently presented at the Winter Summit, offering additional insights into the value of transparency in a data culture. View it here.
Jesse Coraggio is the Associate Vice President of Institutional Effectiveness, Research, and Grants at St. Petersburg College. Jesse joined St. Petersburg College in 2006, and has had increasing levels of responsibility since that time. He received his doctorate in Educational Measurement from the University of South Florida. His research interests include instrument development, psychometrics, setting performance standards, and various item response theory applications. His work has been published and he has presented over 30 papers in the area of assessment and measurement at conferences including the American Educational Research Association, the National Council on Measurement in Education, and the Association for Institutional Research. He is the former president of the Florida Association for Institutional Research.