Small is Volatile

Summer 2005

By Wayne Au

  High-stakes standardized tests like the ones Bush is proposing can only mean big trouble for small schools.
Illustration: Roxanna Bikadoroff

It was hour two of Washington State's mandated standardized test, and Shannon, one of my students at this small public school for former high school dropouts, was wringing her hands over the English section. Brow creased and back hunched over her multiple-choice exam, Shannon was frustrated and angry. Even though it wasn't high stakes, a test like this one took an emotional toll, making her feel inadequate, insecure, and not very smart.

Of course nothing could have been further from the truth. Shannon was smart, but as a white working-class young woman from West Seattle, she was frequently distressed by a myriad of personal issues and everyday dramas. That's why she had dropped out of the big high schools, and that's why she was in my classroom. And even though she was often emotionally distraught over the latest traffic ticket or last weekend's fights, Shannon also exhibited flashes of intellectual ferocity and a real hunger to understand why the world was so messed up—standardized tests and the whole system of education included.

Before the hour was up, Shannon craned her neck toward me and mournfully confessed, "Mr. Au, I'm sorry. But I just can't do it." The look of disappointment on Shannon's face killed me. Even though I had stressed again and again that this test did not measure her worth or intelligence, Shannon could not avoid the matrix of judgment brought down upon her by the state tests. I had told her that the only thing riding on this test was our school's image within the district and that we weren't that worried about it. But Shannon didn't want to let me down personally, as her teacher and an adult who cared for her and provided emotional support.

But support for students like Shannon is shrinking, and the screws of high-stakes "accountability" are tightening on high school students. On Jan. 12, President Bush announced his new "high school initiative" to expand high-stakes testing through the 11th grade under No Child Left Behind (NCLB). Under this new initiative, high school students would be tested in two high school grades in reading and mathematics. Their scores would be calculated into a school's adequate yearly progress (AYP) goals for test score improvement. Ironically, even while the Bush administration is pushing to cut programs that assist low-income students and students of color, the new "high school initiative" allocates $250 million of the FY2006 budget to fund these new tests.

When I heard Bush's announcement, I thought about Shannon. What would have happened if her tests actually were high stakes, if her scores kept her from graduating or kept our small, alternative public school from meeting our AYP requirements under NCLB? My second thought was of all the other high school dropouts nationwide, where it is not uncommon for only 30 percent of the African-American and Latino ninth graders to make it through high school and graduate. The Bush administration touts these tests as the cure-all for underperformance and academic failure. But in my experience, the dropouts like Shannon, and hundreds of others I have worked with, were more like the collateral damage of underfunded schools and assessment systems built upon high-stakes tests.

Big Trouble for Little Schools

High-stakes standardized tests like the ones Bush is proposing can only mean big trouble for small schools. All other arguments about testing aside, small schools are extremely "volatile" when it comes to measuring their progress statistically through standardized test scores. To be volatile in a statistical sense means that you may be subject to wild swings in test scores from year-to-year, grade-to-grade, and school-to-school.

In their oft-cited article "Volatility in School Test Scores: Implications for Test-Based Accountability Systems," education researchers Thomas Kane and Douglas Staiger found that in North Carolina, the smallest schools had 50 percent more variability in test scores than the largest schools. In layperson's terms, this means small schools had swings in their scores that were 50 percent "wider" than large schools.

If you take a moment to think about it, the logic of this test score volatility makes absolute sense. Say you have a small high school or school-within-a-school of 200 students (50 per grade). In one year, you may have recruited 20 students who achieve higher test scores relative to the rest of their grade. Because your high school is small, and these 20 students (fully 10 percent of your total small school population) did well on a test, your school will show strong gains in test scores and AYP. Kudos for you and your school.

But say that in your next year, you recruited 20 more students who perform poorly on standardized tests. Suddenly, this new group of students' test scores, because they represent 10 percent of your student population, will have a drastically negative impact on your school's overall test scores. Whoops! You've shown a drop in scores, didn't meet your AYP, and are now placed on the NCLB school watch list where, if you don't improve in three years, you could be privatized or reconstituted.

Perhaps one of the twisted implications of small school volatility has to do with its impact on diversity. From the angle of high-stakes tests scores, small schools are better off having homogenous populations. If a small group of either high-scoring or low-scoring students can have such a drastic impact on your school's high-stakes test scores, and the future of your small school depends on the standards set by AYP, then it serves a small school's interest to keep low-scoring students out. Because we know that statistically black, Latino, and low-income students perform poorly on the high-stakes tests relative to white and middle-class populations, there is an incentive for small schools to keep poor students of color out of their schools for fear of having scores that don't meet AYP.

A similar logic extends to other, non-racial, subgroups that are counted for AYP. If, for instance, you have a significant number of bilingual or English as a second language (ESL) students, then you have to show AYP on test scores for that subgroup. If you don't have any bilingual or ESL students, then you have no reason to show improvement in that category because the subgroup simply does not exist for you. The more subgroups you have, the more ways there are for you to fail to meet AYP. In an article in Education Week, "Subgroup Reporting and School Segregation," authors Amy Ellen Schwartz, Leanna Stiefel, and Colin Chellman note, "Ironically, be-cause of the way the law is written, the schools and districts that could end up being most heavily penalized are those that are the most heavily integrated." Looking at it from the other direction, homogeneously tracked or privileged small schools can insulate themselves from NCLB's "close the gap" mandate by avoiding student populations with test-score gaps in the first place.

Small schools may be able to sneak under the subgroup reporting radar, however. If you are a small school and you have just enough diversity, but not enough to be required to report subgroups under NCLB, then it is possible that you might avoid having to report test-score data for any group at all. Many states have set their magic number at around 30 to 40 students in any one subgroup to be counted for AYP, but predominately rural states like South Dakota cannot work with subgroup reporting numbers that high, since many of their schools are so small that they have no "official" test scores to report for AYP.

High-Stakes Battle

Many of us who have taken up small schools reform have done so for the best of reasons. I know that I loved working in a small school. I knew my students' home and life situations, I knew their academic histories, and best of all, I knew them personally. Small schools hold the promise of building community and allowing us to create institutions of learning that are not as alienating and inhumane as the large factory-school prototype.

But NCLB and its focus on high-stakes testing and AYP puts a stranglehold on small schools' abilities to work effectively with kids, especially if we build our small schools around diverse student populations. Issues like social justice, equity, and opportunities to learn don't count for much on the tests. Thus we are faced with having to wedge our schools into a behemoth assessment and "accountability" system that is structured for standardization, not creativity and social justice.

Many people have already been resisting high-stakes testing in high schools. A coalition of more than 45 education, civil rights, child advocacy, disability, and religious organizations including the NAACP, the Children's Defense Fund, the National Education Association, and the National Alliance of Black School Educators recently sent a letter to Congress protesting Bush's new plan to increase high school testing. This same group of organizations released a "Joint Organizational State-ment" in October 2004 calling for substantial changes to NCLB. Addition-ally, small schools in New York waged a fierce battle against the Regents tests, touting a portfolio assessment system that was far more rigorous than the tests [See "Standardizing Small," page 15.] Even though policy-makers ignored the power of portfolios, this spirited and organized resistance shows that communities are standing up to the tests.

To add fuel to our fires, and maybe to "empirically" prove what many of us already knew, a recent study by the Northwest Evaluation Association found that the high-stakes tests weren't really working anyway. This study, which used data from more than 320,000 students in 23 states, found that test-score gains have slowed greatly, and improvements may be attributed to students getting used to taking tests as they grow up in an educational era dominated by NCLB. Additionally, this study found that test-score gaps between students of color and white students were still widening, instead of closing.

NCLB is not for my former student, Shannon, hunched over her abusive tests feeling insecure and ashamed. It is not for programs that work to keep kids in school, that actually support the material needs of working-class students and students of color. It is not for a humane system of public education that many small schools are fighting for.

When it comes to small schools, NCLB, its tests, and AYP amount to attempting open-heart surgery with a broad axe: wrong tool, wrong job, killing the patient in the process.

Wayne Au ( is currently pursuing his Ph.D. in curriculum and instruction at the University of Wisconsin-Madison. He is a Rethinking Schools editor.

Summer 2005