ALEXANDRIA, VA – OCTOBER 8-10
To hear the mainstream discussion, it seems clear that public education in the United States is a failed enterprise. The only sensible road ahead, according to this view, is defined by test-based accountability, the privatization of public education through charter schools, and the replacement of school staff. Curiously, there is little credible research supporting value-added assessment of teachers, the efficacy of charter schools, or turnaround efforts involving wholesale staff turnover. What the research suggests is that charter schools are typically not an improvement over traditional schools, that value-added assessment is beset with technical and practical challenges, and that turning around schools and districts depends on the complex and subtle interplay, often over years, of several essential supports.
This debate is being played out in the context of the most difficult economic situation since the Great Depression, in which, according to Representative Jim Moran, the Congressman representing Alexandria, things are likely to get worse before they get better.
These issues dominated the agenda of the National Superintendents Roundtable during its Fall, 2010 meeting, which concluded with a sense that while the status quo in American schools is unsustainable, the reform agenda needs to be as complex as the challenges facing American schools.
Death and Life of the Great American School System
Diane Ravitch, professor of education at New York University, long supported the conventional wisdom of more testing and charter schools. But as the evidence of the ineffectiveness of these approaches mounted, she became convinced that school improvement was not well served by the existing consensus. In a lengthy keynote presentation to the Roundtable, Ravitch laid out her misgivings.
The message, including the message from films such as “Waiting for Superman” has become “alarmingly familiar,” she reported. “American public education is a failed enterprise. The problem is not money… Test scores are low because there are so many bad teachers, whose jobs are protected by powerful unions… The only hope for the future of our society…is escape from public schools.”
American anxieties over threats to our place in the world are at the root of much of this thinking, she noted. There is a “broad apprehension that the nation is falling behind in global competition. If the economy is a shambles, if poverty persists…if American kids are not as serious about their studies as their peers in other nations, the school must be to blame…It’s not globalization, or deindustrialization, or poverty or our coarse popular culture, or predatory financial practices that bear responsibility. It’s the public schools, their teachers, and their unions.”
But, she points out, only 17 percent of the nation’s 5,000 charter schools are superior to a matched traditional public school, while 37 percent are worse. Many charter schools are “mired in unsavory real estate deals,” while some charter principals have been indicted for embezzlement and others have been paid $300,000 annually or more. Meanwhile, the central thesis of the today’s discussion is skewed: “the proposition that teachers are the most important factor in learning is false.” Teachers are the most important factors in schools, but teacher quality accounts for between 7.5 and 10 percent of student test score gains. Finally, the argument frequently advanced that 70 percent of American eighth-grade students cannot read at grade level is “perhaps the greatest distortion.” It relies on numbers drawn from the National Assessment of Educational Progress (NAEP), which actually doesn’t measure grade-level achievement. The three levels of NAEP (Advanced — equating to an A++; Proficient–equating to an A; and Basic – equivalent to a C) are set very high. “It would be more fitting to worry about students who are “below basic,” about 25% of the national sample, not 70 percent.
Ravitch argued that while such emotional appeals are effective, most adults in the United States have been well served by public schools. For such people, “these arguments require a willing suspension of disbelief.” This helps explain why although most people are overwhelmingly dissatisfied with school quality in the abstract, according to Gallup polls, they like their own schools a lot. “Seventy-seven percent award their own child’s school an A or a B, the highest level of approval since the question was first asked in 1985.”
Some charter schools seem to be producing remarkable results, Ravitch acknowledged, but gave them short shrift as a model for widespread emulation:
- Geoffrey Canada’s Harlem Children’s Zone runs two charter schools and surrounds them with a broad array of social and medical services. With assets of $200 million Canada has no shortage of funds and he is paid $400,000 annually. Even with all these resources, she pointed out, 60 percent of students in one of Canada’s charters (and 50 percent in the other) were not proficient in reading, according to 2010 state assessments. “Apparently even the best of charters, with the finest services, can’t completely negate the effects of poverty.”
- The SEED charter boarding school in Washington, DC? It enjoys remarkable graduation and college acceptance rates, but spends $35,000 per child. “Those who claim better education for the neediest students won’t require more money cannot use SEED or Harlem Children’s Zone to support their argument.”
- What about the Locke High School in Los Angeles, part of the Green Dot charter chain? With an infusion of $15 million of mostly private funding, Green Dot produced a safer and cleaner campus, but “no more than tiny improvements in its students’ abysmal test scores.”
“Leadership,” concluded Ravitch, “doesn’t require beating up on your teachers.” She recommended following the example of Finland, typically the Western nation with the highest international assessment results: invest in the preparation, support, and retention of excellent teachers, establish a demanding national curriculum, and greatly improve social welfare programs for children and families.
Federal Education Policy
“I can’t tell you anything that’s particularly encouraging,” said Representative Jim Moran, who represents Virginia’s 8th Congressional District and serves as a senior member of the House of Representatives Appropriations Committee and on the House Democratic Steering and Policy Committee. Economically, the nation is in a lot of trouble, he noted, and suggested that things are likely to get worse before they get better.
Moran ticked off a litany of bad news:
- Employment declined the month before the Roundtable met — mostly due to laying off of 77,000 temporary Census workers, along with 66,000 public service employees.
- Housing prices are in a slump, the very source of the property taxes that provide local funds for schools.
- An additional 230,000 local public employees (most of them teachers) may be on the chopping block in the next twelve months. “How many communities will fire a policeman or firefighter before they fire teachers?” asked Moran rhetorically. “Not many.”
- For three decades following the 1940s, median family income increased about 20 percent per decade, some of it driven by the entry of women into the workforce. Since the 1980s, it has increased at a rate of about one percent, with the result of “a hollowing out of the middle class.”
- Gross tax revenue in the United States represents about 14% of GDP in 2010; from the Eisenhower years on, tax revenue accounted for between 20-23% of GDP.
- In the midst of this great recession, most voters do not have children in public schools, so political support for school programs is weak. Meanwhile, superintendents are not involved in this struggle, while teachers’ unions and school boards have stood on the sidelines.
- All of these factors help explain why Congress “just shrugged,” when presented with the estimate of 230,000 lost jobs, showed no interest in increasing the federal share of P.L. 94-142 to 40% (as the legislation has long required), and lost the opportunity in 2010 to extend the Elementary and Secondary Education Act.
“The people likely to win the midterm elections do not believe education is a federal responsibility,” noted Moran soberly, and the “people most likely to vote are property owners without children in public schools and with the lowest stake in public education.”
Moran called for extending ESEA (NCLB) with provisions that encouraged value-added assessment of teachers, while taking social and economic factors into account. He had faint praise for the Race to the Top program of the Obama administration. “It’s a nice idea, but not a national program. It’s just a political incentive program. If you won, good for you, but if not, tough luck.” He suggested that what is required in the extension of ESEA is a more “holistic approach — not just a school nurse — but a more collaborative and cooperative spirit between the schools, social service agencies, health care facilities and the juvenile justice system.”
Moran’s parting words were a cautionary tale: “We’re going to have a terrific split in income and in the skills of young people. I don’t know what to do about that in this environment. I think honestly that things are going to get much worse than they are, but we cannot permit a survival of the fittest mentality to dominate our education system.”
The grim presentation from Representative Moran was rounded out by a presentation from Dan Domenech, Executive Director of the American Association of School Administrators. The Obama administration, he observed, has “very astutely” driven change with the programmatic emphases in Race to the Top and the American Recovery and Reinvestment Act (the stimulus fund). States had to move toward teacher assessment and charters “just to be in the running to apply for the money.”
Unfortunately, said Domenech, $100 billion intended for schools as part of the stimulus package never reached local districts. “The funds were hijacked by state governments, with the result that most districts today have a budget that is lower than it was two years ago.” As a result, all over the country reports surface of summer school being dropped, after-school programs being eliminated, Advanced Placement and International Baccalaureate programs losing ground, and even four-day weeks replacing five-day weeks.
Domenech bemoaned the administration’s focus on the lowest five percent of the schools in the country, arguing that the existence of these struggling schools defined 100% of American schools as failures.
And Now a Few Words from Ben
Christopher Lowell, actor, historian and former school teacher, who specializes in presenting Benjamin Franklin, sometimes referred to as the “first American,” brought Franklin to life for the Roundtable and provided a break from the cascade of problems and challenges discussed at the meeting.
Franklin professed to be “shocked to find a ‘public system’ of education everywhere in the United States. In my time, only elites were educated–and they had to pay for it.”
While Franklin may not have been familiar with the concept of public education, he did a great deal during his life to advance public purposes and enlighten the public. His “Proposals Relating to the Education of Youth in Pensilvania” (1749) essentially outlined what we know today as the land-grant university system — the marriage of knowledge and practicality in education. Franklin founded the first public library in the Americas, one of the first universities in the United States, and the first medical library.
Reflecting on 40 years in public life, in the United States and abroad, Franklin concluded that in developing young people’s talents it was essential to pay attention to:
- Curiosity: Without questions, little learning takes place.
- Practicality: How do things work?
- Desire to be of service: The noblest idea in the world was to be of use to my fellow man.
- Love of learning: I was curious about everything and willing to experiment with everything.
Remembered as both a skilled writer and a prolific doer, Franklin counseled: “Either write things worthy [of] reading; or do things worth the writing.” That vision still resonates throughout the United States.
Franklin concluded his presentation to the Roundtable by recalling what a woman cried out to him as he left the Constitutional Convention in Philadelphia. “ ‘Well, Dr. Franklin,’ she called out, ‘what have we got?‘ I replied: ‘A republic, if we can keep it.’ ”
What does the research tell us?
Two first-rate research presentations rounded out the Roundtable’s formal agenda. The first came from Penny Bender Sebring, founding co-director of the Consortium on Chicago School Research and co-author of Organizing Schools for Improvement: Lessons from Chicago. The second presentation was by Henry Braun, Boisi Chair in Education and Public Policy, Boston College, who had chaired a National Research Council committee that examined value-added approaches to teacher assessment. Neither presentation provided much support for the conventional wisdom.
Organizing Schools for Improvement
Bender Sebring began her report on 15 years of school improvement efforts in Chicago by reviewing the history of reform efforts in that city that dated back to 1988, when a major decentralization effort was launched, with parallel requirements for parent councils for each school. When the Daley administration took office, to some extent control of the schools was centralized once again, under the leadership of Arne Duncan, now U.S. Secretary of Education.
The good news: Throughout this time, Bender Sebring and her colleagues in the Consortium have studied the effects of these changes. “I am convinced a comprehensive approach is required,” she reported. “It doesn’t have much to do with merit pay, charter schools, or teacher compensation systems.” Citing an analogy frequently brought up in the Consortium, she reported that genuine and deep-rooted reform is complex: “It is like baking a cake. If one of the ingredients is missing, the cake is ruined.” Schools, she maintained, are complex organisms, with multiple interacting subsystems, each involving a mix of human and social factors that shape what is going on. The five essential supports are:
- School leadership that is strategic and focused on instruction. This requires district leadership that is a driver for change and, more specifically, school principals serving as catalysts for systemic improvement. The principal builds “agency” for change at the school and community level encouraging element number two:
- Parent and community ties. It is essential to build new relations with local communities and parents to repair the “long-standing disconnect between urban schools and the children and families” they serve. Active outreach is critical.
- Professional capacity. This involves strengthening the processes supporting faculty learning while enhancing the faculty’s professional capabilities through deliberate focus on the quality of new staff.
- A student-centered learning climate. An environment needs to be nurtured in which students feel safe, while being supported to engage and succeed in more ambitious intellectual activity.
- Instructional guidance. Cultivating school wide supports around curriculum and instruction to promote ambitious goals of academic achievement for every child.
Over 20 years of this research effort, reported Bender Sebring, the Consortium found that if schools adhered to these five principles they’d improve. The real value of these supports rested on their combined strength. Schools that were strong in three to five of the supports were ten times more likely to improve in reading and math than the other schools. Conversely, a persistent weakness in one of the supports undermined the school’s efforts.
If Bill Clinton’s first presidential campaign was governed by the phrase “It’s the economy, stupid,” these reformers would substitute something similar for school improvement: “It’s the instruction, stupid.” Here the sense is that the primary responsibility of school principals is a continuing focus on improving instructional work throughout the school, with the principal serving as “an instructional leader,” because he or she is an expert on teaching and learning, spends most time in school classrooms, aiming relentlessly to improve instruction.
The bad news: Still, she acknowledged, the second half of the Consortium study produced less optimistic results. “We could find improving schools throughout Chicago,” reported Bender Sebring. But “stagnating schools far more often than not were found in low-income communities. In truly disadvantaged Chicago communities, 70 percent of families were below the poverty level, compared to just 7 percent in integrated schools. She noted “tremendous heterogeneity even in low-income populations, with household incomes ranging from $9,480 to $37,750. In schools serving high proportions of truly disadvantaged students, 46 percent of these schools made little or no progress. “Just 15 percent of truly disadvantaged school improved substantially over the life of the study, compared to 42 percent of racially integrated schools.”
Looking beyond race and ethnicity, the study explored “bonding social capital” (a collective sense of efficacy in the community, religious participation, and crime statistics), “bridging social capital” (contacts with people outside the community), and the percentage of students living in extraordinarily difficult circumstances, with histories of abuse or neglect. The results were troubling. Only 5-8 percent of individuals in communities with stagnating schools reported contacts outside the community, regular church attendance, or a sense that their destiny was in their own hands. By contrast 33-39 percent of individuals in communities with improving schools felt a stronger sense of bonding and bridging social capital. Similar ranges could be determined with respect to crime rates (4 percent versus 36 percent) and the incidence of abuse and neglect (2 percent versus 40 percent). “There are almost no good schools for children suffering abuse and neglect,” said Bender Sebring. “These are children suffering terribly traumatic lives.”
Bender Sebring concluded that disadvantaged communities need robust essential supports. Communities can respond to this need, she suggested, by investing in strengthening the capacity of schools and their ability to work together; adopting the five essential supports as the tasks around which schools should plan; encouraging professional development around the five essential supports; and bolstering partnerships with other agencies serving students.
Getting value out of value-added
Braun divided his presentation into two parts. In the first, he acted as a reporter describing the results of the National Research Council committee on value-added measurement that he chaired. In the second, he described his personal conclusions about value-added measurement, views he was at pains to say, that were not necessarily shared by the committee.
Braun, a distinguished statistician, took as his text the question, “Value-added: is it the silver bullet, or is it just a bullet?”
Random assignment of subjects in an experiment is the gold standard of research, he noted. But students are not assigned to classrooms randomly. Still, “the public buys into the construct that if good teachers are critical to student learning, why can’t evidence of student achievement be used as an indicator of school and teacher quality?”
The research literature on this is very complex, he noted, so the National Research Council appointed a committee to look into the question. The committee pulled together several commissioned papers and held a two-day conference on the subject in 2007, before issuing a report earlier this year. The report pointed to several measurement and analytic challenges, including:
- Need for very high standards for assessments used to make high-stakes decisions.
- Possibility of changing educators’ behavior with unintended consequences.
- The difficulty of attributing student growth to specific teachers in team-teaching situations.
- The reality that tests scores reflect a narrower set of educational goals than most parents and educators hold for students.
- All test scores are susceptible to measurement error.
- Need for equal interval scales that permit consistent ranking of schools, teachers and value-add (so that a 10-point jump from 30 to 40 equals a 10-point jump from 60 to 70), a requirement most tests do not meet.
- The difficulty of vertically linking tests from grade to grade to compare growth.
- Bias introduced by non-random assignment of students to teachers and teachers to schools.
- Challenges to precision and stability in the face of small class sample sizes, leading to year-to-year fluctuations in estimates of teacher effects.
- Finally, challenges to data quality, including missing or faulty data.
These are hardly trivial issues. Turning then to his personal views on the matter, Braun noted, “Given what I know and my knowledge that a lot of what is being said is inaccurate, it is wrong for me not to speak.”
On pay for performance, Braun noted that evaluations of the concept produce contradictory results, although the latest study from Vanderbilt indicates that even bonuses as high as $15,000 are ineffective. What value-added is really measuring, he said, is not the effectiveness of teachers, but the effect of being in a particular classroom. “Suddenly this becomes just a teacher issue as though nothing else in the class mattered.”
Although few studies show that value-added measurement correlates well with observational assessments, there is no a priori reason to believe that value-added measurement is better than observational techniques.
Braun argued that the status quo in schools is not sustainable, especially with high dropout rates. “There are staggering opportunity costs associated with the high dropout rates in the United States. He called for using value-added measures not for accountability but for “triage,” that is to say to identify classrooms and schools needing more support. “We should then go from vam (value-added measures) to vans,” he quipped — that is to say “we should get in a car if need be and go visit the schools needing assistance.”
If, he cautioned, policymakers continue to insist on value-added measurement, they should be aware of “Campbell’s law”: “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
In a concluding session presided over by Mort Sherman, Alexandria superintendent, it was agreed that:
- The Roundtable did not want to lose the special value of its meetings as safe harbors in which superintendents could consider the major issues facing American schools. The Roundtable provides something unique and valuable as a professional learning community for its members.
- It would be highly desirable to develop a statement of Roundtable principles defining the organization’s sense of what is important in American education.
- The members also requested assistance in drafting talking points on major issues (for use with both national and state legislators), and potential opinion pieces for local newspapers.