Publication Type

In Search of High-Quality Evaluation Feedback: An Administrator Training Field Experiment

2019

Starting in 2011, Boston Public Schools (BPS) implemented major reforms to its teacher evaluation system with a focus on promoting teacher development. We administered independent district-wide surveys in 2014 and 2015 to capture BPS teachers’ perceptions of the evaluation feedback they receive. Teachers generally reported that evaluators were fair and accurate, but that they struggled to provide high-quality feedback. We conduct a randomized controlled trial to evaluate the district’s efforts to improve this feedback through an intensive training program for evaluators. We find little evidence the program affected evaluators’ feedback, teacher retention, or student achievement. Our results suggest that improving the quality of evaluation feedback may require more fundamental changes to the design and implementation of teacher evaluation systems.

Suggested Citation:

Suggested citation

Kraft, M.A., & Christian, A. (2019). In Search of High-Quality Evaluation Feedback: An Administrator Training Field Experiment. EdWorkingPapers.com. https://doi.org/10.26300/ydke-mt05

Share

Publication Type

The Effect of Teacher-Family Communication on Student Engagement: Evidence from a Randomized Field Experiment

2013

In this study, we seek to evaluate the efficacy of teacher communication with parents and students as a means of increasing student engagement. We estimate the causal effect of teacher communication by conducting a randomized field experiment in which 6th and 9th grade students were assigned to receive a daily phone call home and a text/written message during a mandatory summer school program. We find that frequent teacher-family communication immediately increased student engagement as measured by homework completion rates, on-task behavior and class participation. On average, teacher-family communication increased the odds a student completed their homework by 42% and decreased instances in which teachers had to redirect students’ attention to the task at hand by 25%. Class participation rates among 6th grade students increased by 49%, while communication appeared to have a small negative effect on 9th grade students’ willingness to participate. Drawing upon surveys and interviews with participating teachers and students, we identify three primary mechanisms through which communication likely affected engagement: stronger teacher-student relationships, expanded parental involvement, and increased student motivation.

Suggested Citation:

Suggested citation

Kraft, M.A., & Dougherty, S.M. (2013). The Effect of Teacher-Family Communication on Student Engagement: Evidence from a Randomized Field Experiment. Journal of Research on Educational Effectiveness. https://doi.org/10.1080/19345747.2012.743636

Share

Publication Type

Sustaining a Sense of Success: The Importance of Teacher Working Conditions During the COVID-19 Pandemic

2020

COVID-19 shuttered schools across the United States, upending traditional approaches to education. We examine teachers’ experiences during emergency remote teaching in the spring of 2020 using responses to a working conditions survey from a sample of 7,841 teachers across 206 schools and 9 states. Teachers reported a range of challenges related to engaging students in remote learning and balancing their professional and personal responsibilities. Teachers in high-poverty and majority Black schools perceived these challenges to be the most severe, suggesting the pandemic further increased existing educational inequities. Using data from both pre-post and retrospective surveys, we find that the pandemic and pivot to emergency remote teaching resulted in a sudden, large drop in teachers’ sense of success. We also demonstrate how supportive working conditions in schools played a critical role in helping teachers to sustain their sense of success. Teachers who could depend on their district and school-based leadership for strong communication, targeted training, meaningful collaboration, fair expectations, and recognition of their efforts were least likely to experience declines in their sense of success.

Suggested Citation:

Suggested citation

Kraft, M.A., Simon, N.S., & Lyon, M.A. (2020). Sustaining a Sense of Success: The Importance of Teacher Working Conditions During the COVID-19 Pandemic. EdWorkingPapers.com. https://doi.org/10.26300/35nj-v890

Share

Publication Type

Teacher layoffs, teacher quality and student achievement: Evidence from a discretionary layoff policy

2015

Most teacher layoffs during the Great Recession were implemented following inverse-seniority policies. In this paper, I examine the implementation of a discretionary layoff policy in Charlotte Mecklenburg Schools. Administrators did not uniformly lay off the most or least senior teachers but instead selected teachers who were previously retired, late-hired, unlicensed, low-performing, or nontenured. Using quasi-experimental variation within schools across grades, I then estimate the differential effects of teacher layoffs on student achievement based on teacher seniority and effectiveness. Mathematics achievement in grades that lost an effective teacher, as measured by principal evaluations or value-added scores, decreased 0.05 to 0.11 standard deviations more than in grades that lost an ineffective teacher. In contrast, teacher seniority has limited predictive power on the effects of layoffs. Simulation analyses show that the district selected teachers who were, on average, less effective than those teachers identified under an inverse-seniority policy, and also reduced job losses.

Suggested Citation:

Suggested citation

Kraft, M. (2015). Teacher layoffs, teacher quality and student achievement: Evidence from a discretionary layoff policy. Education Finance and Policy. https://doi.org/10.1162/EDFP_a_00171

Share

Publication Type

The Effect of Teacher Coaching on Instruction and Achievement: A Meta-Analysis of the Causal Evidence

2018

Teacher coaching has emerged as a promising alternative to traditional models of professional development. We review the empirical literature on teacher coaching and conduct meta-analyses to estimate the mean effect of coaching programs on teachers’ instructional practice and students’ academic achievement. Combining results across 60 studies that employ causal research designs, we find pooled effect sizes of 0.49 standard deviations (SD) on instruction and 0.18 SD on achievement. Much of this evidence comes from literacy coaching programs for pre-kindergarten and elementary school teachers. Although these findings affirm the potential of coaching as a development tool, further analyses illustrate the challenges of taking coaching programs to scale while maintaining effectiveness. Average effects from effectiveness trials of larger programs are only a fraction of the effects found in efficacy trials of smaller programs. We conclude by discussing ways to address scale-up implementation challenges and providing guidance for future causal studies.

Suggested Citation:

Suggested citation

Kraft, M., Blazar, D., & Dylan Hogan. (2018). The Effect of Teacher Coaching on Instruction and Achievement: A Meta-Analysis of the Causal Evidence. Review of Educational Research.

Share

Publication Type

School organizational contexts, teacher turnover, and student achievement: Evidence from panel data

2016

We study the relationship between school organizational contexts, teacher turnover, and student achievement in New York City (NYC) middle schools. Using factor analysis, we construct measures of four distinct dimensions of school climate captured on the annual NYC School Survey. We identify credible estimates by isolating variation in organizational contexts within schools over time. We find that improvements in school leadership especially, as well as in academic expectations, teacher relationships, and school safety are all independently associated with corresponding reductions in teacher turnover. Increases in school safety and academic expectations also correspond with student achievement gains. These results are robust to a range of threats to validity suggesting that our findings are consistent with an underlying causal relationship.

Suggested Citation:

Suggested citation

Kraft, M., Marinell, W., & Yee, D. (2016). School organizational contexts, teacher turnover, and student achievement: Evidence from panel data. American Educational Research Journal. https://doi.org/10.3102/0002831216667478

Share

Publication Type

The Effect of Evaluation on Teacher Performance

2012

Teacher performance evaluation has become a dominant theme in school reform efforts. Yet, whether evaluation changes the performance of teachers, the focus of this paper, is unknown. Instead, evaluation has largely been studied as an input to selective dismissal decisions. We study mid-career teachers for whom we observe an objective measure of productivity -- value-added to student achievement -- before, during, and after evaluation. We find teachers are more productive in post-evaluation years, with the largest improvements among teachers performing relatively poorly ex-ante. The results suggest teachers can gain information from evaluation and subsequently develop new skills, increase long-run effort, or both.

Suggested Citation:

Suggested citation

Taylor, E.S., & Tyler, J.H. (2012). The Effect of Evaluation on Teacher Performance. American Economic Review.

Share

Publication Type

Effectiveness Research for Teacher Education

2021

Despite calls for more evidence regarding the effectiveness of teacher education practices, causal research in the field remains rare. One reason is that we lack designs and measurement approaches that appropriately meet the challenges of causal inference in the context of teacher education programs. This article provides a framework for how to fill this gap. We first outline the difficulties of doing causal research in teacher education. We then describe a set of replicable practices for developing measures of key teaching outcomes, and propose causal research designs suited to the needs of the field. Finally, we identify community-wide initiatives that are necessary to advance effectiveness research in teacher education at scale.

Suggested Citation:

Suggested citation

Hill, H.C., Mancenido, Z., & Loeb, S. (2021). Effectiveness Research for Teacher Education. EdWorkingPapers.com. https://doi.org/10.26300/zhhb-j781

Share

Publication Type

Identifying Effective Classroom Practices Using Student Achievement Data

2010

Research continues to find large differences in student achievement gains across teachers’ classrooms. The variability in teacher effectiveness raises the stakes on identifying effective teachers and teaching practices. This paper combines data from classroom observations of teaching practices and measures of teachers’ ability to improve student achievement as one contribution to these questions. We find that observation measures of teaching effectiveness are substantively related to student achievement growth and that some observed teaching practices predict achievement more than other practices. Our results provide information for both individual teacher development efforts, and the design of teacher evaluation systems.

Suggested Citation:

Suggested citation

Kane, T.J., Taylor, E.S., Tyler, J.H., & Wooten, A.L. (2010). Identifying Effective Classroom Practices Using Student Achievement Data. Journal of Human Resources. https://doi.org/10.3368/jhr.46.3.587

Share

Publication Type

Teacher Effects on Complex Cognitive Skills and Social-Emotional Competencies

2019

I exploit the random assignment of class rosters in the MET Project to estimate teacher effects on students’ performance on complex open-ended tasks in math and reading, as well as their growth mindset, grit, and effort in class. I find large teacher effects across this expanded set of outcomes, but weak relationships between these effects and performance measures used in current teacher evaluation systems including value-added to state standardized tests. These findings suggest teacher effectiveness is multidimensional, and high-stakes evaluation decisions are only weakly informed by the degree to which teachers are developing students’ complex cognitive skills and social-emotional competencies.

Suggested Citation:

Suggested citation

Kraft, M. (2019). Teacher Effects on Complex Cognitive Skills and Social-Emotional Competencies. Journal of Human Resources.

Share