Three Evaluation Studies Prove the Efficacy of ISA's Model
ISA’s work has been the subject of three independent evaluation studies and one special sub-population analysis. Having met the What Works Clearinghouse standards, ISA’s Model has been approved for use in School Improvement Grants (SIG) as an evidence-based Whole School Reform model.
IMPAQ International conducted an impact study of the Institute for Student Achievement (ISA) high school model. The ISA model is grounded in seven principles that focus on creating and sustaining intellectually rigorous, caring, and personalized learning environments. The study used a quasi-experimental approach to estimate the impact of the model on attendance, credit accumulation, grade promotion, and graduation rates in 27 high schools in New York City, New York and Atlanta, Georgia. The study sample included students who were 9th graders in fall 2006, 2007, 2008 and 2009. ISA schools serve predominately poor and minority students who enter high school with below proficient levels of math and English language arts achievement. Findings from the study suggest that ISA schools study had positive effects on student outcomes in New York and Atlanta. In particular, ISA students outperformed their peers on the following outcomes:
- 11th grade attendance
- Earning enough credits to be promoted to the next grade
- Four-year credit accumulation
- On-track for graduation at the end of 9th grade
- High school graduation
ISA’s estimated impact on these outcomes was greater for schools rated as having a high level of implementation. Additionally, post-secondary outcome data collected from five ISA schools showed that ISA students attend four-year schools at higher rates and have greater persistence rates than students nationwide. These findings indicate that ISA has a positive impact on the transition to high school, progress through high school, and high school completion. The estimated impact on promotion from 9th to 10th grade is particularly important given that 9th grade is a critical transition year, especially for at-risk youth, and that promotion to 10th grade is highly predictive of high school success and graduation. The findings suggest that ISA is an effective school reform model that can make a difference for at-risk students in terms of their success in high school and beyond.
The Academy for Educational Development (AED) conducted a six-year study of two cohorts of 9th graders in New York City ISA schools through high school and into their early post-graduation careers. Using a quasi-experimental design, this study looked in depth at ISA’s impact on student, teacher, and school outcomes. Quantitative data came from student achievement records and from teacher and student surveys. Qualitative data were obtained from site visits, interviews of key school and ISA staff, and classroom observations. We also compared achievement outcomes for ISA students with a matched comparison group of students in large New York City high schools who had similar characteristics and achievement levels.
Data show that the schools in this study generally implemented ISA principles well. ISA teachers attributed many positive changes in their practice and in the school environment to ISA professional development and particularly to the ISA coaches. Our analyses indicate that the implementation of ISA had a large and positive effect on student achievement outcomes.
High School Achievement
- ISA students had greater rates of grade promotion and attendance than comparable peers.
- ISA students accumulated more course credits and failed fewer core subject courses than comparable peers.
High School Completion
- Comparison students were more likely than ISA students to pass state exit exams at the more rigorous “Regents” level (scoring 65 or higher) and to earn an advanced diploma.
- Compared to the peer group, fewer ISA students dropped out of high school and more graduated in four years.
- Most ISA seniors planned to attend college. Most students reported that they received the support that low-income, first-generation college attenders often require to navigate the process of applying for college and financial aid. For instance, they visited colleges and received help in selecting schools, writing essays, and completing college and financial aid applications.
College Enrollment and Persistence
- ISA students attended four-year colleges, rather than two-year institutions, at higher rates than did Black and Latino students nationwide. They were also more likely to attend college full-time. Both factors are important indicators of future success in college and beyond.
- ISA graduates enrolled in City University of New York (CUNY) programs were somewhat less likely to have to take remedial courses than students CUNY-wide.
- Most ISA graduates who enrolled in college persisted into the second year of college, a major dropping-out point. Rates of college persistence for ISA graduates were much higher than the national rate.
In sum, this study indicates that the ISA model has improved students’ performance in high school and persistence in college. Our findings suggest that small learning communities that emphasize individual attention, provide a college preparation curriculum, and foster a college-going culture can give students from low-income urban communities the support they need to achieve academic—and later economic—success.
This MDRC Evaluation Study provides rigorous evidence (for 12,130 participants in a series of naturally occurring randomized lotteries) that a large-scale high school reform initiative (New York City’s creation of 100+ small high schools of choice between 2002 and 2008) can markedly and consistently increase high school graduation rates (by 9.5 percentage points overall and for many different student subgroups) for a large population of educationally and economically disadvantaged students of color without increasing annual school operating costs. These findings are directly relevant to current debates by policymakers and practitioners about how to improve the educational prospects of disadvantaged students in the United States.
In 2014, ISA commissioned IMPAQ International (IMPAQ) to conduct an evaluation of the impact of the ISA school model on African American male students’ high school outcomes. The evaluation compared several measures – high school attendance, credit accumulation/grade promotion, dropout rate, and four-year graduation rates – across four cohorts of African American male students enrolled in ISA and comparison schools in New York City and two cohorts of students enrolled in ISA and comparison schools in Atlanta.
The results showed that African American male students in ISA schools consistently outperformed their non-ISA school peers on several high school outcome measures examined in the study. African American male students in ISA schools 1) had higher average daily attendance rates in grades 9-11, 2) were promoted to the next grade at significantly higher rates, 3) had lower dropout rates and 4) graduated at a higher rate than students at the comparison schools.