Cost Effectiveness

PEMBERTON, NJ (JUNE 1-2)

By 2011, about 85% of school districts were anticipating budget cuts within the next two years and school administrators everywhere were eager to find the best bang for the buck. At the same time, state and local officials were grappling with implementation of the Bush administration’s No Child Left Behind Act and the Obama administration’s Race to the Top program, while debate about the level of resources needed to accomplish these programs’ ambitious goals raged in legislatures across the country.

It is against that broad backdrop that the bottom fell out of school budgets in 2008 as the economy entered the Great Recession, housing prices (and with them local tax revenues) plummeted, and unemployment soared. The  Roundtable’s summer meeting, hosted by Pemberton’s Michael Gorman and Pennsville’s Mark Jones and meeting at Pemberton High School, took up the issues of cost effectiveness and cost-benefit analysis. It also touched briefly on legislative stalemate in Washington, D.C. and explored recent developments in the Garden State with the chief of staff to the New Jersey Commissioner of Education.

Cost Effectiveness and Cost-Benefit Analysis

Henry Levin of Teachers College, Columbia University helped the Roundtable move the cost-effectiveness conversation along during it’s Summer meeting, June 1-2. Levin, the William Heard Kilpatrick Professor of Economics and Education at Teachers College, is a specialist in the economics of education and human resources and has published fourteen books and almost 300 articles on these and related subjects.

How do you, as superintendent, judge which is the best bet for improving reading at the elementary level: investing in a new reading series or a library literacy program? What about a major new investment in preschool or kindergarten programs? Do they have any effect?  This is where cost effectiveness and cost-benefit analyses come into the picture, according to Levin.

Cost effectiveness analysis (CEA) lets analysts compare alternatives by taking both the costs and the effects into account. It is always oriented around educational objectives. Do you want to improve high school graduation rates? Which is more cost effective — preschool reading programs, smaller elementary school classes, or encouraging all students to take Algebra in Grade 8?

Cost effectiveness gets at questions such as these, said Levin (center, below, speaking with Jerry Kohn former Harrisburg superintendent and Steve Price. Hazelwood, Missouri).

Henry Levin, middle, speaks with Jerry Kohn, former Harrisburg, Pa superintendent, and Ste Price, Hazelwood, Missouri.

How does that differ from cost-benefit analysis (CBA)? CBA also compares the costs and outcomes of alternatives, but the results are expressed not in educational outcomes but in dollars and cents — and frequently in terms of outcomes years after school is long past. According to Levin and his colleagues, “Not only is it possible to ascertain which educational investment has the largest benefits relative to costs, but it is also possible to compare these results with investments in health, transportation, physical capital, such as plant and equipment, and other sectors. In this way, governmental units can use CBA to compare the desirability of alternative educational investments, as well as to determine the balance between investing in education and other sectors.”

Both approaches are closely related. The major difference is that CBA measures benefits in financial terms, while CEA does so in educational outcomes.

Perry Preschool Program

Perhaps the most influential example of these analyses revolved around the famous Perry Preschool Program, funded in Michigan in the 1960s. The study compared 123 African-American children, with 58 randomly assigned to a preschool program and 65 who did not participate and followed these small children for 40 years.

The Perry Preschool Analysis revealed stunning long-term economic returns to investments in high-quality preschool programs. When these participants, who entered the program at age three or four were examined at the ages of 27 (and then again at age 40), the results were little less than extraordinary.

Participants earned more money than non-participants; they were more likely to have graduated from high school, to be employed, and to own their own homes. They were much less likely to be on public assistance, to use drugs, or to become involved with the juvenile justice or the adult correctional systems. They were also much more likely to own their own car, and to own a savings account. In short, in just about every area where families, society, and policymakers would hope to find positive results, they found positive results. And in just about every area where negative results should be avoided, program participants tended to avoid them.

The analysis above is a cost-effectiveness analysis. It evaluated the outcomes over time in terms of educational and social results. But the Perry analysts went beyond cost effectiveness to add a cost-benefit discussion to the conversation. They emerged with a remarkable figure. They estimated that, for every dollar invested in a preschool effort such as the Perry program, society received an economic return of $17.01 over 40 years. Investing $15,166 returned $258,888 to society per participant, in constant dollars (see figure below). The return is in the form of reductions on expenditures for education, public assistance, and the criminal justice system and net additions to state and local revenues in the form of higher taxes on participants’ earnings. Fully 93% of the public return is attributable to more positive outcomes for male participants, due largely to their decreased involvement with the criminal justice system. Similar analyses in New Jersey, Ohio, Pennsylvania and other states also indicate promising returns on public investments in preschool programs.

Public schools are just beginning to get involved with cost-effectiveness and cost-benefit analyses. One of the best examples was recently reported from Virginia Beach, Virginia, a district with 70,000 students. Tired of a public discussion implying that schools are nothing but a drain on taxpayers’ resources, superintendent James Merrill commissioned a cost-benefit analysis from an economist at North Carolina State University, Michael Walden. The economist concluded that for every dollar spent on Virginia Beach schools, the local economy receives back $1.53. And for every million dollars spent on school construction, the community creates 13 jobs. That’s because high school graduates earn $800,000 more in a lifetime than dropouts and return more to city, county, and state coffers in taxes. High school graduates are also more likely than dropouts to be healthy (and cost less to care for) and to attend college (and earn higher salaries), while being less likely to draw on state and local resources for public health needs, social welfare expenditures, or expenses of the criminal justice system.

Complex Methodology

“We’re spending a trillion dollars on K-12 and higher education in the United States,” said Levin. The question is, “Can we use our resources better?” Unfortunately, he noted, the people answering that question were the people complaining about expenditures in letters to the editor and at school board meetings. “We need a more balanced set of answers.”

In 2007, said Levin, after an exhaustive examination of 200 studies, he was able to identify five programs with solid, empirical, grounded research behind them that improved high school graduation rates. The five included the Perry Preschool program. The other four?  First Things First (a comprehensive reform of small learning communities); substantial class size reduction (in Grades K-3, with class sizes reduced from 25 to 15); child-parent center programs in Chicago (involving preschools, parental outreach, and health and nutrition services); and teacher salary increases (of 10% for teachers in all years, K-12).

None of these programs are cheap, cautioned Levin. Imagine 100 students in danger of dropping out. About 50 0f them will graduate anyway. But since educators don’t know which 50, they have to “treat” every student with the same program. “Say, you succeed in getting 60 to graduate.  In reality the entire expenditure has been directed at getting just 10 extra graduates. You can’t divide the costs among the 100 students — since 50 of them were destined to graduate anyway.”

So getting the costs right is essential to sound analysis.  The methodologies are complex. “These ten students didn’t drop out. They stayed in school. So now you have the costs of educating them in 9th, 10th, 11th, and 12th grades. Plus some of them will go to college. What are the additional costs associated with that?” All of these expenses have to be accounted for.

On the other side of the ledger are the returns. High school graduates live longer — on the order of nine years longer than high school dropouts. So they earn more over a lifetime and pay more taxes. In addition, as with the Perry Preschool analysis, high school graduates demand less by way of expenditures on public health, public assistance, and the criminal justice system.

What does it all add up to, asked Levin. “For every additional high school graduate you can produce, the taxpayer gets a net present-rate of return of $128,000.” That is to say, even after subtracting all the costs, taxpayers still come out ahead.

Why aren’t these approaches used more often (with the exception of places like Virginia Beach)?  Administrators aren’t trained in these methods. And in fact, these procedures, which presume a zero-based budgeting approach don’t match budgeting practices in most districts, which tend to be grounded in incremental increases.

Discussion after the presentation zeroed in on the practical and political challenges of implementing these approaches. Such approaches need to be planned ahead of time. When budgets are being savagely cut, administrators have trouble stepping back to say, “Wait, we need a cost-benefit analysis.” In a highly politicized environment, there is little demand from public officials for such analyses. “The problem,” said one superintendent, “is that when the community is consumed with the need to reduce budgets right now, it is difficult to justify expenditures on the basis of returns 20 years down the line.”

Stalemate at the Federal level

In an impressive short presentation, Noelle Ellerson of the American Association of School Administrators briefed the Roundtable on developments in Washington. There was not a lot of good news to report, she noted. Although both the House and Senate have produced legislation to extend ESEA (No Child Left Behind) and move the pendulum back toward more state and local control, neither bill is likely to arrive at the President’s desk. The House committee has produced a streamlined bill (with just two Titles) that was voted out of committee on a highly partisan, party-line vote. The outlook for floor action is slim to non-existent. The Senate committee has proceed in a bipartisan fashion, but it is clear that committee will move a bill to the Senate floor only after the House has acted.

Meanwhile, posturing around the deficit and the national debt means that discretionary domestic and defense expenditures (including education) face substantial reductions in January 2013 under a procedure known as sequestration. Just about everyone agrees that sequestration needs to be avoided at all costs, but how that will play out in an election year, with one-third of the Senate, the President, and the entire House of Representatives up for election, remains to be seen.

Developments in the Garden State

David Hespe, chief of staff to the New Jersey Commissioner of Education, served as Commissioner of Education in the Garden State under former Governor Christine Todd Whitman. “In philosophy, vision, and structure,” he said, “there is no comparison between the Department of Education I entered in 1997 and the one that exists today. They are very different places.”

David Hespe, chief of staff to NJ Commissioner of Education, outlines plans for the Garden State.

He went on to describe a department that is becoming more focused on school performance and “outcomes,” as distinct from the entity he led, which was part of a hierarchy in education that “focused on making sure that everyone did what they were supposed to do.” He thought this change was typical of the changes experienced by state education agencies around the country.

“In 1997 we thought our job was compliance,” noted Hespe. “When it came to school improvement, we told people what to do. If you made a list of low-performing schools in 1997 and compared it to the same list today, there’d be very little change. So, you’d have to say that in terms of our efforts in the state since 1997, the compliance approach has not been very successful.”

But, he said, a good symbol of the changes afoot could be found in New Jersey’s request for a waiver under No Child Left Behind. Today, he said, the state, like states everywhere, is focusing on schools, not on districts. It is now moving away from telling districts what to do to an emphasis on doing more alongside the most troubled schools. “We want to bring in expertise in data, leadership, organizational development, curriculum and evaluation. We want to be partners. There are 2,500 schools in New Jersey and, instead of worrying about them all, we’re focused on the 200 schools that need the most help.”

Hespe described a new state education agency worrying about four key initiatives: an ESEA waiver from the U.S. Department of Education; educator effectiveness; college and career readiness; and educational transformation. In this effort, he said, his agency is trying hard to listen to what local officials have to say.

He acknowledged that the New Jersey Department of Education is experiencing difficulty finding the right people to put on its own bus, to use an image from Jim Collins’ Good to Great. State department employees face a salary gap of $141,000 and it’s hard to find outstanding people willing to go to work for the state at that level of salary, he said. Challenged on whether Governor Christie’s decision to cap salaries for school superintendents had not been a boon for surrounding states (who are hiring experienced superintendents from New Jersey), Hespe insisted that the situation presented the Garden State with an opportunity:  districts are hiring ambitious young administrators from elsewhere. Is the rancor between educators and the governor that seems so divisive to people outside the state interfering with good government in New Jersey? Hespe thought not, dismissing the issue as a clash between the personal styles of the governor and the head of the state teachers’ association.

Good, Fast, and Cheap — Pick Two

The adage holds that if you want something that’s good, fast, and cheap you can only have two, noted Michael Gorman during a debriefing of the meeting. Projects that are good and fast are very rarely cheap. Those that are good and cheap are not, on the whole, fast. It almost goes without saying that things that are fast and cheap are almost always never very good. Sometimes lightening strikes and you get all three, but for the most part you have to be satisfied with two.

The consensus of the group seemed to be that too much education change anticipates good results, but wants the results to be fast and cheap. Even when policymakers put money into new initiatives, they want fast results, even if it is not it is not clear the initiatives promise good outcomes.

It’s all reminiscent of an insight from systems thinker Peter Senge. Leaders, he has observed, need to see the big picture, because very often the fastest way out of a problem is also the easiest way to get enmeshed in it again.

Comments are closed.