Assessment

PORTLAND, OR – JULY 16-18

“I come to these Roundtable meetings to have my assumptions challenged,” says Steven Ladd, superintendent in Elk Grove, California.  Timothy Grieves of Sioux City, Iowa simply says, “This where I come to get my batteries recharged.”  Each got what he was looking for at the summer meeting in Portland, Oregon, hosted by Eugene superintendent George Russell in July.

The agenda was full and challenging, presenting the views of those who support value-added assessment of teachers and those skeptical about the approach. It ranged across a host of questions.  Is it possible to evaluate teacher effectiveness based on the value they add to students’ learning?  How do we square the demand for “value-added assessment” with some experts’ belief that it is likely to be impractical? Is most current reform thinking a mistaken attempt to fix 19th- and 20th-century notions of teaching and learning?  If so, what would new educational models for the 21st century look like?  How can central offices best support principals and teachers in their efforts to improve student achievement?  What is the essential purpose of schooling?  That is to say, we all know what we do, and most of us have a pretty good sense of how we do it, but most people (and many educators) have trouble explaining why they do it.  Finally, what does the Roundtable have to contribute to the national debate about these issues?

These concerns dominated the Roundtable’s three-day July meeting, and prefigured a preliminary discussion about what the Roundtable’s voice might be in the national discussion of teacher quality, assessment, and accountability.

Value-Added Assessment

Dan Goldhaber, director of the Center for Education Research and Data, University of Washington, supports the idea of using value-added methods as part of teacher evaluation, but his data-heavy presentation underscores the difficulty of putting into practice what seems so simple in theory.  One of the hopes of proponents of value-added assessments is that administrators and policy makers will be able to identify which teachers add the greatest value to student outcomes, potentially offering a mechanism to weed out ineffective teachers, reward the most competent, and assign the best to students and schools facing the greatest challenges.

Traditional metrics for assessing teachers leave a lot to be desired, Goldhaber notes. Most standard evaluation devices — possession of a teaching credential, years of experience, holding tenure, in-service assessments, and financial rewards for continuing education — fall far short of the mark.  Few tenured teachers are dismissed, he argued.  Although one survey of eight districts revealed that administrators believed between 2-7% of tenured teachers should be out of teaching, typically less than 0.5% are dismissed each year. Despite the surface appeal of the value-added concept, a lot stands between the ideal and the reality.

Goldhaber’s presentation suggests:

  • While it is true that “teacher quality is the most important schooling factor influencing student achievement,” Goldhaber’s work confirms earlier research: Schooling accounts for just 20% of achievement variation; out-of-school factors far outweigh school variables in explaining achievement outcomes.  Teacher quality itself accounts for about 8% of student achievement variation, according to Goldhaber’s 2002 Hoover Institution Report.
  • Another way of understanding the significance of teacher quality is to say that it accounts for about 40% of in-school explanatory power.  Just 3% of that is made up of “easily measurable aspects of teacher quality”; 97% consists of “less tangible aspects.”  Teacher quality is “primarily unobservable.”
  • A single year’s data on teacher effectiveness is likely to include a lot of noise; a more accurate picture of individual teacher effectiveness requires multiple years of assessment data – at least three and perhaps as many as five years.

Other potential troublesome issues include ignoring non-tested areas (history, the arts, and the civic purposes of education); narrowing of instruction to tested areas; corruption of the assessment system by cheating and hiding low-growth students; the imprecision of cut-point scores; the challenge of teachers who are competent in one teaching area, but not another; apportioning credit among multiple teachers; the reality that a teacher can be effective in one school situation and not in another; and the real possibility of discouraging teacher collaboration.

Goldhaber’s presentation led to an animated discussion among Roundtable members.  They acknowledged that his data had challenged some of their assumptions, but the potential discouragement of teacher collaboration troubled many. Superintendents need to worry about an entire system and how changes in one area reverberate with surprising power in unexpected ways elsewhere. The discussion indicated that the need is not simply to do a better job of identifying problem teachers, but also of finding ways to encourage teachers to seek out others with expertise.  School systems currently have trouble doing this and the creation and cultivation of professional learning communities might go a long way toward meeting this need.

Fixing Old Notions of Teaching and Learning

Tom Carroll’s outlook on the teaching challenges facing American schools could not be more different.  He directly took up the issue of the importance of teacher collaboration. Carroll is president of the National Commission on Teaching and America’s Future, which has issued several landmark reports on teaching in the last 20 years.

“Once in a generation we have the opportunity to redefine public education,” declared Carroll.  “We are at such a point now.  The problem is we are busy trying to fix a 19th- or 20th-century notion of teaching, when what we should be doing is creating a new vision of learning for the 21st century.”

The challenge, he said, is moving from an emphasis on teaching and on leadership to an emphasis on learning – for everyone within the school, administrators, teachers and students. “Today, teachers are on their own. Each individual teacher is busy developing his or her abilities.  There is no effort to build common resources for the entire teaching community.”

Carroll argued that a teaching organization (what we have now) uses batch processing in the search for efficient transmission of knowledge. What we need, he suggested are “learning organizations that facilitate co-created learning environments and make use of ubiquitous learning resources that can be customized to meet individual needs.”  In such an environment everyone wins because every learner can thrive.

For decades, he pointed out, it has been known that 50% of new teachers leave the profession in the first five years.  They leave because they are poorly prepared, have little support, and cannot find a rewarding career path.

Educators and organizations such as the Roundtable have a “professional responsibility to create 21st century learning organizations, learning spaces, and learning places.”  Just as law and medicine have moved from Perry Mason and Dr. Kildare in the 1950s to legal and medical teams in the 21st century, so too education must make the transition to teams if it is to meet its responsibilities to parents, students and taxpayers.  So far, we have succeeded only in moving from the stand-alone teacher of the 20th century to the stand-alone teacher of the 21st, said Carroll.

A New Model

Carroll called for a “learning ecology” for schools to replace the current emphasis on teaching.  We are all familiar with two different kinds of teams, he said.  There are “independent teams” – e.g., Olympic track or skiing teams) in which the members work together but perform independently and even compete against each other.  Then there are “interdependent teams,” that is teams made up of individuals dependent on each other for the team’s success (hockey or basketball teams).  These are the teams in which, in an old coach’s saw, “there is no “I” in the word “team.”

There’s no “I” in the word “teacher” either was Carroll’s message. When reform or teacher quality becomes all about the individual teacher, learning suffers.  “Teachers have a collective responsibility for student learning,” noted Carroll, who pointed to the need to create a 21st-century view of learning that considered teachers as an interdependent learning community focused on four “learning age competencies”:

  • Core Competencies: in language arts, reasoning, information literacy, mathematics, sciences, and social sciences necessary for adult success;
  • Creative Competencies: including creative expression in the arts, critical thinking, innovation, collaborative problem solving, and resourcefulness;
  • Communication Competencies: with languages, digital media; social networking, and content creation technologies; and
  • Cultural Competencies: including cultural understanding, personal and communal responsibility, adaptability and resilience, an ability to engage in productive teamwork, and active citizen participation.

He called up an excellent on-line video of the Richard J. Murphy School in Boston to illustrate the importance of collaborative teamwork.  Murphy emphasizes: individual learning plans for all students; a distributed leadership model in which teachers govern and manage the school; and an environment that encourages teachers to take risks.  The video clearly demonstrates teachers focusing on what they need to learn, consulting with colleagues for assistance, and seeing each other as resources in the construction of learning teams.  The teachers see themselves as learners within the school.

The Murphy School is not the only site pioneering professional learning communities for teachers.  Panels featuring Eugene’s George Russell, Decatur’s Gloria Davis, and a team of teachers and administrators led by Dan Jamison, Superintendent of Schools in Sherwood, Oregon described their efforts to reshape teaching and learning.  Sherwood has adapted the CLASS project (Creative Leadership Achieves Student Success), an Oregon-wide experiment involving a dozen districts to expand career paths, improve performance evaluation, provide outstanding professional development, and explore new teacher compensation models.

If the CLASS approach is to work, according to the Sherwood participants, trust is essential, leadership is defined as a culture that begins in the classroom, and relationships between and among teachers are critical.  CLASS requires a comprehensive approach, one that relies on active research to guide practice and collaborative teams.

In the face of these on-the-ground experiments, value-added approaches come across as yesterday’s solution applied to the problems of today and tomorrow.  “What does the Murphy example say about value-added assessment?” asked Carroll. “It’s not encouraging.  Value is added through team learning, not through individual accomplishment or competition between teachers.”

Central Office Support

As Roundtable members discussed these presentations a number of questions bubbled to the surface, including the appropriate role of the central office.  Michael A. Copland of the University of Washington’s Center for the Study of Teaching and Policy presented the Wallace Foundation’s latest research on precisely that point.

In an interactive session with the Roundtable, Copland emphasized that school boards must be supportive of this work; administrators must be in schools; and central office staff need to change their behavior so that central staffers are listening to and respectful of teachers’ voices.

The Wallace research completed by Copland and his colleagues, explored how central offices in several urban school districts were able to reshape themselves to advance learning by transforming their relationship with schools.

It reached four major conclusions. The new approach to reform:

  • Focuses meaningfully on improving teaching and learning. Improving efficiency in delivering basic services to schools is not the point; staff must be able to demonstrate in concrete terms how their work relates to improving teaching and learning.
  • Engages the entire central office in reform.  It’s not just a matter of involving the curriculum and instruction office. Everyone in the central office staff, no matter the department, unit or function, participates in the transformation
  • Requires central office administrators to fundamentally remake their work practices and relationships with schools.  This isn’t about structural change or modifying reporting relationships.  A transformation strategy requires people in the central office to remake their daily work and relationships with schools.
  • Constitutes an important focus for reform in its own right.  It’s common for central offices to reshape work practices to implement particular programs – for example, small schools initiatives or site-based management.  That’s not enough.  Central office transformation involves ongoing work on central office practice that transcends particular programs or initiatives.

This work is challenging and requires time to come to fruition, noted Copland.  It calls for learning-focused partnerships with principals; leadership support to nurture central office-principal partnerships; reorganizing and “re-culturing” each central office unit; stewardship of the overall central-office-transformation process; and use of evidence through the central office to support continued improvement of work practices and relationship with schools.

That’s a demanding agenda, all by itself, said Copland, noting that Atlanta required ten years of “staying the course” to reshape its central office.

The Golden Circle

In a remarkable session led by Douglas Hesbol, superintendent of Laraway Schools in Illinois, Roundtable members challenged themselves to make these presentations their own and act on them back in their districts.  Hesbol opened the discussion by introducing the Roundtable members to a fascinating presentation by Simon Sinek at a recent TED conference (see below).

TED, which started out in 1984 as a conference bringing together people from the worlds of technology, entertainment, and design, holds two conference annually, one in California and one in Oxford, England. It describes itself as a small nonprofit devoted to “Ideas Worth Spreading.”  Sinek, trained as an anthropologist and active in advertising throughout his career, published Start with Why.

Focusing on the Apple Corporation and the Wright Brothers, Sinek introduced the TED audience (and the Roundtable) to the “Golden Circle.” At the outer edges of the circle, we describe what we do.  Boring deeper into the circle, we describe how we do it.  And most entities that fail to prosper are motivated by the “what” and the “how.”  It’s in the inner core of the Golden Circle that we find the meaning of what we’re doing – why we are doing what we are doing.

People don’t buy Apple products because they need a computer. They can buy a computer anywhere, from anyone. It’s not even that Apple products are typically marvels of design.  People buy Apple products because Apple has convinced them it shares their values and is committed to using technology to reshape modern life for the better.  Similarly, the Wright Brothers, without significant backing succeeded in mastering the mysteries of flight because they loved the idea of flying – not because they enjoyed substantial financial backing as some of their competitors did.

“People,” said Sinek, “don’t buy what you do or how you do it, they buy why you do it.”  The goal, he asserts, is not to do business with people who need what you have; it is to do business with people who believe in what you believe.

Public schools, of course, have to do business with everyone who shows up at the door.  Still, the idea of getting to the question of “why” we are in business resonated deeply with Roundtable participants. Remember Martin Luther King, Jr., urged Sinek.  King “gave the ‘I Have a Dream’ speech, not the ‘I Have a Plan’ speech.”  In other words this visionary leader beseeched his audience to buy into why civil rights were important, not what they were or how to obtain them.

Don’t confuse leaders and leadership, was another Sinek message.  “There are leaders and there are those who lead,” he said.  “Leaders hold positions of power or authority; those who lead, inspire us.”

Under Hesbol’s questioning, the presentation from Sinek helped Roundtable members think about the other presentations in a new light.  Members worried that educational reform has been tinkering around the edges.  The exposure to conflicting concepts of teacher quality was powerful; challenges to our assumptions were worthwhile. Of course we need to develop greater urgency around this issue while somehow convincing national policy makers that teaching is not an individual enterprise but a team effort in which interdependencies are at the very core of what a good school does.  Above all we need to remind people of why educators do what they do. It’s not just to help the country produce better widgets.  It’s more than international competitiveness. Most educators are in business to produce better people for a better future in a better world.

The Roundtable’s Voice

Where does the Roundtable fit in all this?  What does it have to say?  How can it say it? A fascinating discussion led by Mort Sherman, Alexandria, Virginia superintendent closed out the meeting.

Several themes dominated the conversation.  Several were bread and butter concerns of superintendents everywhere.  The delay in amending No Child Left Behind troubled some.

Questions about “highly qualified teachers” bothered others.  Some wanted to challenge the federal government on not living up to its promise to fund 40% of the excess costs of providing educational services to students with disabilities.  Getting the public to understand that American schools do a good job with students who remain within the system over time, but experience difficulty with highly transient student populations, was another note. The lack of political will to provide sufficient funds to do the job also came into the debate. The Obama administration’s “Blueprint” was touched on, with a sense that it needed a lot of work, but the Roundtable should support several parts of it, including those that modify No Child Left Behind.  Conceivably the Roundtable should function as a think tank of sorts, bringing data to bear and issuing white papers.

The discussion nibbled around the question of international comparisons.  It touched on both outcomes (we don’t seem to be doing as good a job as Asian and Scandinavian countries) and process (we can learn from people beyond our own borders).

Then the key questions at the heart of the meeting came to the fore.  How do we move from a teaching system to a learning system?  How do we build the transparency, break down the silos, and develop the competencies required to create a 21st-century system? How do we engineer the shift of vision that is required to transform schools from institutions that sort students to institutions that educate all students?

It was clear that members want to find a unique voice for the Roundtable, while simultaneously protecting the original vision of the organization – a safe harbor in which superintendents could learn, grow, improve their practice, and think about the shape of education as they would like to see it.  That’s why Sinek’s focus on the “why” of education held such appeal to the members.  That’s why the Roundtable promises to recharge members’ batteries while questioning the assumptions they bring to the table.

Sherman concluded the discussion with another TED video and promised to revisit these issues in Alexandria during the Roundtable’s October meeting.  This video presented pianist and conductor Benjamin Zander in a lively presentation about classical music and its potential for universal appeal.

Two shoe salesmen went to Africa to explore a potential new market, noted Zander.  The first cabled back that the task was hopeless:  these people don’t wear shoes.  The second cabled the home office:  “Fantastic opportunity.  They don’t have any shoes!”

A similar fantastic opportunity presents itself in classical music, he suggested (and now that we think about it, in education as well).

As a conductor, said Zander, you are very powerful and all eyes are on you. But you don’t make a sound. “You depend for your power,” he said, “on your ability to make other people powerful.”  That insight came to Zander after 20 years leading orchestras.  But once he realized it “it changed everything…My job was to awaken possibility in other people…You find out if that’s working when you see that the eyes of the people in front of you are shining…. The definition of success is not about wealth or power, but about how many shining eyes are around you.”

It’s an apt metaphor for superintendents.  Their power depends on their ability to make other people powerful…and the measure of their success will be found not in test scores or international comparisons, but in how many shining eyes surround them.

Comments are closed