Do college and career pathways help students succeed after high school? Which students benefit from dual credit? Are college and career readiness programs effectively reaching students from our most marginalized communities? Do we have the data to answer these questions?
Illinois has a range of innovative and thoughtful college and career readiness policies that are scaling across the state, but a lack of consistent evaluation means we are unable to answer the questions above. Given the burdens that system leaders face, it is imperative that we provide evidence that the policies being implemented will bring meaningful improvements for the learners they serve.
For example, recent research from the Illinois Workforce and Education Research Collaborative (IWERC) highlights statewide challenges and inequitable outcomes stemming from postsecondary education and chronic absenteeism. These studies raise the question: What strategies can adequately engage and prepare learners for long-term success?
Prior research suggests that strategies, such as those outlined in the 2016 Postsecondary and Workforce Readiness (PWR) Act, can help address these challenges. However, we need more evidence to confirm how supports, including College and Career Pathway Endorsements and transitional instruction, impact learners’ progressions through education and into employment.
This is especially important as we learn about the challenges districts face in scaling college and career readiness programs. In particular, we must assess whether students from marginalized groups have access to these programs and whether they lead to better outcomes. These interventions are complex and may take many years to evaluate end-to-end, but there are approaches to research and evaluation design that can help develop a more robust understanding of the combination of experiences and skills necessary to equip learners for success.
The third pillar of our Policy Agenda, Policy Evaluation and Continuous Improvement, outlines our priorities for building statewide data capacity and for evaluation approaches that empower local and state leaders with actionable information to improve programs and shape systems, thereby increasing student success.
Building the Pipeline: Our Policy Agenda in Action
Too many Illinois students still struggle with the transitions from high school to college and career. In this blog series, explore each pillar of our policy agenda and how systemic change can create equitable pathways for every student:
Pillar 1 | Leadership, Governance, and Vision: Centering Student Success in a Chaotic Context
Pillar 2 | Creating Conditions Where Students Can Thrive
Pillar 3 | From Policy to Progress: Charting a Path to Data-Informed Continuous Improvement for Student Success
Pillar 4 | Coming soon…
Pillar 3: Policy Evaluation & Continuous Improvement
The EdSystems team focuses on four challenges that are core to developing and deploying the needed research and data to improve students’ experiences:
- Lack of Evaluation: We do not yet know if the college and career readiness policy levers we have implemented are functioning as intended and having the desired impact on equity and outcomes.
- Data Access Gaps: Illinois lacks consistent, quality, and actionable implementation, participation, and outcome data both statewide and locally.
- Underutilization of Data: State leaders and community partners often underutilize implementation, participation, and outcome data to inform policy changes and local practices.
- Underequipped for Equity: Significant gaps remain between shared P-20 goals and local and statewide action, as leaders are unevenly equipped to operationalize equity in policy and practice.
Too often, data is used as a blunt tool to criticize system leaders, which makes them hesitant to establish a more robust and transparent data practice. Instead, we believe data should serve as a flashlight, illuminating areas both for celebration and for further investigation. We need leaders at all levels to be able to make sense of how their work impacts learners, but resources and practice are required to strengthen that “muscle.”
How do we gather and utilize the right data and tools to ensure our systems are driving the student outcomes that matter the most?
Our First Aim: Statewide Research Agenda and Governance
Our first aim in this pillar builds from our call to have a coordinated education and workforce ecosystem with a shared vision for economic mobility:
Illinois’ education and workforce agencies hold a shared and funded research agenda and governance structure to systematically examine the efficacy and equity of statewide policies and programs.
Illinois currently lacks a research agenda to guide the development of education-to-workforce programs, and its data ecosystem remains too siloed, with limited data integration across agencies to execute that agenda. Current tools and resources, though valuable, only represent discrete parts of our P-20 pipeline. Tools such as the Illinois School Report Card, Illinois Postsecondary Profiles, and Illinois High School to Career are very valuable, but provide only snapshots of how our systems are functioning and how learners progress into and through opportunities. Illinois needs to move beyond snapshot tools and develop the agenda, information systems, and governance that allow us to understand programs from K-12 into college and career.
Many Illinois laws already require some form of program evaluation. However, the capacity to integrate data across multiple systems often hinders the ability to meet such requirements. In fact, legal constraints are often perceived as hindering our ability to match data across agencies and use it effectively – despite the fact that most privacy laws make exceptions for research and evaluation, and that agencies and researchers have methods for ensuring privacy while still allowing effective evaluation. Governance can help break this morass by identifying the required research and developing legal frameworks that support better research and evaluation rather than hindering it.
As highlighted in our Crain’s Op-Ed in September, there is both an educational and a business imperative to adequately fund and manage shared data governance and capacity in Illinois. Bringing business to Illinois requires aligning our education and economic development systems, which in turn requires data to understand how to deliver programming that prepares students for future careers. Data processes must be inclusive, encompassing both the state agencies that will share data and the data users, such as researchers, community members, and educators. However, we also need business leaders at the table who can articulate industry needs and help define the outcomes we are trying to achieve.
The Illinois Community College Board’s annual Dual Credit Report, which includes valuable analysis of student access to and outcomes from dual credit coursework, is an example of the type of information we need to improve programming. That said, more research and evaluations are needed to understand how other college and career readiness strategies empower learners. These analyses will require more robust data resources, including student-level data that can be easily linked across different data systems, as well as governance and a shared vision for identifying the most important questions to answer. This shared research agenda should be informed by communities, and state leaders need to drive continuous improvement both statewide and locally.
Our Second Aim: Data-Informed Continuous Improvement Process
Beyond establishing the governance necessary to design and develop research priorities, our second aim centers on how this research and data should drive continuous improvement locally and statewide:
Community and state leaders systematically engage with quantitative and qualitative data to understand student experiences and implement continuous improvement processes to enhance equitable student access and outcomes.
A data-driven vision for change that aims to equip local and statewide leaders with the data and tools they need to diagnose challenges and identify opportunities to improve learner experiences and outcomes drives EdSystems’ work. In both our policy and practice portfolios, we regularly integrate data engagement into planning and continuous improvement processes, as exemplified by the annual regional meetings of the Illinois Education and Career Success Network. We have also created the Success Network Dashboard to provide new ways for communities to engage with data about their learners, visualizing trends over time, comparing with peer communities, and comparing the proportionality of student access and outcomes to their overall student body.
Since 2017, we have facilitated dozens of community-level data workshops to empower cross-stakeholder teams to use their data to inform their work. This has enabled peer-to-peer learning. For example, we recently highlighted a school district’s success in scaling access to dual credit with Latino students and sharing their practices with a neighboring district that identified that work as an area for growth. Our data work, whether with state agencies or community partners, helps us visualize progress, identify areas for improvement, share best practices, and center student input in continuous improvement. Providing data to communities and schools is critical for their continuous improvement efforts.
Recent progress toward this goal was made in the 104th General Assembly with the passage of SB2039 (now Public Act 104-0405). It aims to create a more robust higher education dashboard that will bring together student demography, institutional characteristics, student success factors, and 5-year comparison data. Alongside what is already available on the Illinois Postsecondary Profiles site, we are eager to see how this additional data resource can be utilized to support statewide policy conversations and inform institutional practice at our state’s postsecondary institutions.
Data access, like that provided by SB2039, should be layered like an onion, with high-quality publicly available data resources being the outer layer. These need to be consistent across agencies, easily digestible, and connected. The data used to generate these dashboards should be accessible as spreadsheets and data files, allowing for analysis with data science tools and integration into AI engines for more sophisticated use. For example, the Illinois State Board of Education should be commended for publishing the data file each year alongside the Interactive Report Card. Finally, agencies need consistent processes for requesting aggregated data that is not published on a consistent basis, and robust, efficient data request processes that allow researchers to access student- and individual-level data necessary for serious research efforts.
These data resources, used in consistent, inclusive processes across districts, institutions, and communities, can unlock strategies to help our students succeed.
What’s Next
At EdSystems, we use data across our portfolio to inform our policy and practice and drive the impact we hope to see in the state.
In our policy portfolio, we will continue to support the Success Network Policy Committee in engaging with statewide data, including at our upcoming December meeting and the January deep dive. Please join us as we review the latest Illinois Report Card release and other data assets. Our data team will also present at the annual Success Network Conference on building a shared data use practice, which will equip attendees with tools and practices for engaging with data in their collaborative contexts. We will highlight our work integrating data in the Rockford community, discuss our approaches to collecting data on pathway programs, and demonstrate how we use agency data to support improvements in programs such as transitional instruction and dual credit.
We plan to continue developing our data practice in the coming year. We are developing a data playbook to provide communities with a standard, inclusive process for data engagements, and we are working with school districts and community colleges on a strategy to integrate their data and, eventually, connect it to workforce data. Recently, we published an updated dashboard for our Scaling Transformative Advanced Manufacturing Pathways (STAMP) initiative, and we plan to work with regional partners to design additional reporting tools.
Beyond the work we already have planned, we are continually pursuing new opportunities to secure funding for the research and evaluation of key strategies, such as those established by the Postsecondary and Workforce Readiness Act.
In our final policy agenda blog post, we will reflect on how to build a system that is future-ready and that prepares learners to be resilient in the face of evolving education, job markets, and emerging technology.
