Preliminary findings from Kalamazoo Promise study released

KALAMAZOO--Preliminary findings from a study of the Kalamazoo Promise's impact on the local school system have been released by the Western Michigan University Evaluation Center.

The first round of findings are detailed in three working papers that have been posted on the center's Web site, www.wmich.edu/evalctr/promise. The papers, titled "The Kalamazoo Promise as a Catalyst for Change in an Urban School District," "Response from Community Groups" and "Key Findings from the 2007 Survey of High School Students," give a preliminary assessment of the celebrated scholarship program's impact on schools and the community at large.

The purpose of the evaluation, which is funded by the U.S. Department of Education, "is to determine whether the Kalamazoo Promise, through its transformative effect on the education culture within the school district, improves the progress of students through their K-12 experience and better prepares them for entering a postsecondary education program."

Many of the key findings are still pending and will be released later in the summer. But initial results show the scholarship program, which was announced in November 2005 and provides four years of tuition and fees at any public college or university in Michigan for students who have attended Kalamazoo Public Schools, is having an impact on students, schools and the community.

The 2007 survey of high school students shows the vast majority plan to use the scholarship. Perhaps most telling, more than three-quarters of the students said they wish to use the Promise to enroll in a community college or university. Nearly a third reported the existence of the Promise has motivated them to attend school more often. A third also said they are now working harder in school due to the Promise.

Almost half of the students said they think the Promise has led to teachers having higher expectations, while only 34 percent reported the promise had not changed their education goals and plans.

Dr. Gary Miron, Evaluation Center research associate and director of the Promise evaluation, said, "these are promising results, but it is important to note that these are self-reported data on short-term outcomes. Hard data on intermediate and long-term outcomes will be critical to determine the actual impact of the Promise on the public school system."

Interviews with a diverse array of key informants from the community show the Promise is also having an impact on community groups that support students and families. Changes among community organizations prompted by the Promise include increased support for students through new programming, such as tutoring or through expanding existing programs serving children, and an influx of tutors and mentors across different community organizations.

"In a few cases, entirely new groups formed to provide tutoring for KPS students," according to Stephanie Evergreen, co-author of the working paper on community response and Evaluation Center staff member.

Some examples include the nearly doubling of mentors at Big Brothers-Big Sisters in 2006 from the previous year, while Kalamazoo Communities in Schools, one of the non-profit organizations coordinating volunteers and services for many district schools, reported a 134 percent increase in volunteer service hours after the Promise was announced.

Some organizations said they had implemented changes specifically because of the Promise, but roughly an equal number reported little to no change. "In many cases, the Promise spurred community organizations to review their purpose and objectives to ensure that they are in line with student needs," the paper says.

Findings also show an abundance of anecdotal evidence of some change in students and parents after the scholarship program was announced. Community leaders said parents had told them "they now have a greater focus on their children's academic work and are enforcing more academic and social discipline," the paper says.

Other key indicators of the Promise's impact are still being assessed. Evaluators working on this project expect to see increases in the number of, and enrollment in, college prep courses; an increase in student retention, promotion and graduation rates; a rise in student performance on standardized tests; higher rates of application to colleges and universities; an increase in options and access to postsecondary colleges and universities; and a general "building of social capital for all students."

The logic model employed to assess the Promise's impact is not a prescriptive tool that suggests to educators and community actors what to do, Miron notes. Instead, it is assumed that the catalytic effect of the Promise will motivate and inspire various stakeholder groups to focus on a common goal.

"The Promise has potential as a comprehensive school reform model," Miron says. "While other such models are highly prescriptive, the systemic reform model we are testing assumes the district will find the right combination of reforms and changes. The Promise acts as an incentive, it raises hopes, and it removes obstacles for access to postsecondary education."

In this way, the Promise can have a catalytic effect that will motivate and inspire various stakeholder groups to focus on common goals: improve K-12 education and prepare more students for success in postsecondary education.

Miron says there has been great interest in the Promise evaluation, both locally and nationally, including from the U.S. Department of Education.