While student test scores are part of the solution, scores alone are not enough to gauge how well a teacher is performing, according to the Gates-funded Measures of Effective Teaching Project.
Released Tuesday, the final report from the MET Project says a three-prong approach, multiple classroom observations, student surveys and student growth as measured by state test scores, provides a good picture of how effective a teacher is. The project found that an accurate observation rating for a teacher requires two or more lessons, each scored by a different certified observer.
The report will likely resonate in Georgia, which is in the midst of rolling out a new teacher evaluation system funded by the state’s Race to the Top grant. Georgia is spending millions on its new evaluation system, which will consider student performance in rating whether a teacher is exemplary, proficient, developing/needs improvement or unsatisfactory.
The MET findings examined the performance of students of the 3,000 teachers from Charlotte-Mecklenberg, Dallas, Denver, Hillsborough County, Fla., Memphis, Pittsburgh and New York City who volunteered to be part of the project.
Through a randomized experiment, the research sought to answer the question: Are seemingly more effective teachers really better than other teachers at improving student learning, or do they simply have better students?
Ultimately, the only way to resolve that question was by randomly assigning students to teachers to see if teachers previously identified as more effective actually caused those students to learn more. That is what we did for a subset of MET project teachers.
Based on data we collected during the 2009–10 school year, we produced estimates of teaching effectiveness for each teacher. We adjusted our estimates to account for student differences in prior test scores, demographics, and other traits. We then randomly assigned a classroom of students to each participating teacher for 2010–11.
In fact, we learned that the adjusted measures did identify teachers who produced higher (and lower) average student achievement gains following random assignment in 2010–11. The data show that we can identify groups of teachers who are more effective in helping students learn. Moreover, the magnitude of the achievement gains that teachers generated was consistent with expectations.
In addition, we found that more effective teachers not only caused students to perform better on state tests, but they also caused students to score higher on other, more cognitively challenging assessments in math and English
On average, the 2009–10 composite measure of effective teaching accurately predicted 2010–11 student performance. The research confirmed that, as a group, teachers previously identified as more effective caused students to learn more. Groups of teachers who had been identified as less effective caused students to learn less. We can say they “caused” more (or less) student learning because when we randomly assigned teachers to students during the second year, we could be confident that any subsequent differences in achievement were being driven by the teachers, not by the unmeasured characteristics of their students. In addition, the magnitude of the gains they caused was consistent with our expectations.
A practical concern of many readers of this blog is the time and expense associated with classroom observations. The new MET report — with its endorsement of multiple observers including someone from outside the school to counter bias — will probably inflate those concerns.
The study addressed the time constraints:
Our analysis from Hillsborough County showed observations based on the first 15 minutes of lessons were about 60 percent as reliable as full lesson observations, while requiring one-third as much observer time.
Therefore, one way to increase reliability is to expose a given teacher’s practice to multiple perspectives. Having three different observers each observe for 15 minutes may be a more economical way to improve reliability than having one additional observer sit in for 45 minutes. Our results also suggest that it is important to have at least one or two full-length observations, given that some aspects of teaching scored on the Framework for Teaching (Danielson’s instrument) were frequently not observed during the first 15 minutes of class.
American Federation of Teachers President Randi Weingarten issued this statement in response to Tuesday’s release of the final MET report:
The Gates Foundation’s study makes clear that evaluation of teachers must start with genuine feedback, which means the days of haphazard or check-list observation of teachers must end. Just dropping by a teacher’s classroom and writing up an evaluation must be replaced with a more serious process that actually helps improve teacher practice and student learning.
The MET study of thousands of teachers reaffirms that teacher evaluation is both an art and a science that requires time, tools, training and trust — ingredients that teachers and principals should have but too often don’t. This study underscores the fact that teacher evaluations must be about improving teaching, not just a mere snapshot assessment.
The MET findings reinforce the importance of evaluating teachers based on a balance of multiple measures of teaching effectiveness, in contrast to the limitations of focusing on student test scores, value-added scores or any other single measure.
To read more responses to the MET report, check out this Ed Week story.
–From Maureen Downey, for the AJC Get Schooled blog