Research Publications

More Than Shortages: The Unequal Distribution of Substitute Teaching

By Jing Liu, Susanna Loeb, and Ying Shi. (2020)

Classroom teachers in the US are absent on average approximately six percent of a school year. Despite the prevalence of teacher absences, surprisingly little research has assessed the key source of replacement instruction: substitute teachers. Using detailed administrative and survey data from a large urban school district, we document the prevalence, predictors, and variation of substitute coverage across schools. Less advantaged schools systematically exhibit lower rates of substitute coverage compared with peer institutions. Observed school, teacher, and absence characteristics account for only part of this school variation. In contrast, substitute teachers’ preferences for specific schools, mainly driven by student behavior and support from teachers and school administrators, explain a sizable share of the unequal distribution of coverage rates above and beyond standard measures in administrative data.

More →

The Short- and Long-Run Impacts of Secondary School Absences

By Jing Liu, Monica Lee, Seth Gershenson. (2020)

We provide novel evidence on the causal impact of student absences in middle and high school on state test scores, course grades, and educational attainment using a rich administrative dataset that includes the date and class period of each absence. Our identification strategy addresses potential endogeneity due to time-varying student-level shocks by exploiting the fact that in a given year, there exists within-student, between-class variation in absences. We also leverage information on the timing of absences to show that absences that occur after the annual window for state standardized testing do not appear to affect test scores, which provides a further check of our identification strategy. We find that absences in middle and high school harm contemporaneous student achievement and longer-term educational attainment: On average, missing 10 math classes reduces math test scores by 7% of a standard deviation, math course grades by 19% of a standard deviation, the probability of on-time graduation by 8%, and the probability of immediate college enrollment by 7%. Similar results hold for absences in English Language Arts classes. These results suggest that absences in middle school and high school are just as harmful, if not more so, than absences in elementary school. Moreover, the timing of absences during the school year matters, as both the occurrence and the impact of absences are dynamic phenomena.

More →

The Longitudinal Effects of School Improvement Grants

By Min Sun, Alec Kennedy, and Susanna Loeb. (2020)

School Improvement Grants (SIG) represent one type of governments’ capacity-building investment to spur sustainable changes in America’s persistently under-performing public schools. This study examines both short- and long-run effects of the first two cohorts of SIG schools from two states and two urban districts across the country. Using dynamic event analyses, we observe that SIG showed larger effects in the second and third years of the intervention than the first year on 3-8th grade student test scores—a pattern of gradually increase over the course the intervention. These positive effects are largely sustained three or four years after the funding ended. In high schools, the SIG effects on 4-year graduation rates were steadily increasing throughout the period of six or seven years after the initial start of the intervention. These patterns of SIG effects mostly apply to each of the four locations, but the magnitude of effects varies across locations, suggesting differential implementations. Moreover, SIG effects on students of color or low-socioeconomic students are similar to, and sometimes a bit larger than, the overall SIG effects. We also conduct a variety of sensitivity and robustness checks. Lastly, we discuss the policy implications of our findings on states’ continuing efforts of transforming public organizations and building their long-term capacity for better performance.

More →

Engaging Teachers: Measuring the Impact of Teachers on Student Attendance in Secondary School

By Jing Liu, Susanna Loeb. (2019). The Journal of Human Resources

Teachers’ impact on student long-run success is only partially explained by their contributions to students’ short-run academic performance. For this study, we explore a second dimension of teacher effectiveness by creating measures of teachers’ contributions to student class-attendance. We find systematic variation in teacher effectiveness at reducing unexcused class absences at the middle and high school level. These differences across teachers are as stable as those for student achievement, but teacher effectiveness on attendance only weakly correlates with their effects on achievement. We link these measures of teacher effectiveness to students’ long-run outcomes. A high value-added to attendance teacher has a stronger impact on students’ likelihood of finishing high school than does a high value-added to achievement teacher. Moreover, high value-added to attendance teachers can motivate students to pursue higher academic goals as measured by Advanced Placement course taking. These positive effects are particularly salient for low-achieving and low-attendance students.

More →

Differing Views of Equity: How Prospective Educators Perceive Their Role in Closing Achievement Gaps

By Emily K. Penner, Jane Rochmes, Jing Liu, Sabrina M. Solanki, Susanna Loeb. (2019). The Russell Sage Foundation Journal of the Social Sciences

Hiring is an opportunity for school districts to find educators with values and beliefs that align with district goals. Yet beliefs are difficult to measure. We use administrative data from more than ten thousand applications to certificated positions in an urban California school district in which applicants submitted essays about closing achievement gaps. Using structural topic modeling (STM) to code these essays, we examine whether applicants systematically differ in their use of these themes and whether themes predict hiring outcomes. Relative to white applicants, Hispanic and African American applicants are more likely to identify structural causes of inequities and discuss educators’ responsibilities for addressing inequality. Similar differences in themes emerge between applicants to schools with different student populations. Techniques like STM can decipher hard-to-measure beliefs from administrative data, providing valuable information for hiring and decision making.

More →

Stress in Boom Times: Understanding Teachers' Economic Anxiety in a High Cost Urban District

By Elise Dizon-Ross, Susanna Loeb, Emily Penner, Jane Rochmes. (2019). AERA Open

Despite growing concern over teachers' ability to live comfortably where they work, we know little about the systematic impacts of affordability on teachers' well-being, particularly in highcost urban areas. We use novel survey data from San Francisco to identify the patterns and prevalence of economic anxiety among teachers and how this anxiety relates to teachers' attitudes and behaviors. We find that San Francisco teachers have far higher levels of economic anxiety on average than a national sample of employed adults and that young teachers are particularly anxious. Furthermore, anxiety relates to job performance and well-being economically anxious teachers tend to have more negative attitudes about their jobs, have worse attendance, and are more likely to be chronically absent.

More →

One Step at a Time: The Effects of an Early Literacy Text-Messaging Program for Parents of Preschoolers

By Ben York, Susanna Loeb, and Chris Doss. (2018). The Journal of Human Resources

Large systematic differences in young children’s home learning experiences have long-term economic consequences. Many parenting programs place significant demands on parents’ time and inundate parents with information. This study evaluates the effects of READY4K!, an eight-month-long text-messaging intervention for parents of preschoolers that targets the behavioral barriers to engaged parenting. We find that READY4K! increased parental involvement at home and school by 0.15 to 0.29 standard deviations, leading to child gains in early literacy of about 0.11 standard deviations. The results point to the salience of behavioral barriers to parenting and the potential for low-cost interventions to reduce these barriers.

More →

More than Just a Nudge: Supporting Kindergarten Parents with Differentiated and Personalized Text-Messages

By Christopher Doss, Erin Fahle, Susanna Loeb, Ben York. (2018). The Journal of Human Resources

Recent studies show that texting-based interventions can produce educational benefits in children across a range of ages. We study effects of a text-based program for parents of kindergarten children, distinguishing a general program from one adding differentiation and personalization based on each child’s developmental level. Children in the differentiated and personalized program were 63 percent more likely to read at a higher level (p<0.001) compared to the general group; and their parents reported engaging more in literacy activities. Effects were driven by children further from average levels of baseline development indicating that the effects likely stemmed from text content.

More →

Educators As "Equity Warriors"

By Jane Rochmes, Emily Penner, Susanna Loeb. (2017)

Despite the multifaceted causes of educational disparities, schools' responsibility for reducing inequality undergirds American ideals. Educators operate as street-level bureaucrats to accomplish this equalizing work. Surprisingly, we know little about how teachers think about inequality or enact strategies to combat it, even though their execution of this mandate is almost certainly mediated through pre-existing understandings. This paper provides a framework (applicable to other sectors) to examine educators' beliefs about inequality and their role in advancing equity. To progress, teachers need to believe that doing so is valuable, feel empowered to overcome disadvantage, and be willing to make tradeoffs in pursuit of equity goals. Our framework highlights the salience of individual differences between lowand highachievers as a form of inequality that may divert teachers' focus from the structural inequality that is central to policy and sociological concern. We test this framework empirically using novel survey data from over 1,500 teachers collected in a diverse urban school district. Although most surveyed teachers believe addressing inequality is important and feel empowered to do so, many seemingly equity-minded educators do not endorse strategies aligned with closing racial and socioeconomic inequality'indicating an important barrier to reducing inequality

More →

Resource- and Approach-Driven Multi-Dimensional Change: Three-Year Effects of School Improvement Grants

By Min Sun, Emily Penner, Susanna Loeb. (2017). American Educational Research Journal

Hoping to spur dramatic school turnaround, the federal government channeled resources to the country’s lowest-performing schools through School Improvement Grants (SIG). However, prior research on SIG effectiveness is limited and focuses primarily on student achievement. This study uses a difference-in-differences strategy to estimate program impacts on multiple dimensions across the 3-year duration of the SIG award in one urban school district. Following 2 years of modest improvement, we find pronounced, positive effects of SIG interventions on student achievement in Year 3, consistent with prior literature indicating that improvements from comprehensive school turnarounds emerge gradually. We also identify improvements indicating the process through which change occurred, including reduced unexcused absences, increased family preference for SIG schools, improved retention of effective teachers, and greater development of teacher professional capacity.

More →

For schools and teachers to help students develop knowledge and skills, students need to show up to class. Yet absenteeism is prevalent, especially in secondary schools. This study uses a rich data set tracking class attendance by day for over 50,000 middle and high school students from an urban district in academic years 2007–2008 through 2012–2013. Our results extend and modify the extant findings on absenteeism that have been based almost exclusively on full-day absenteeism, missing class-by-class absences. Notably, part-day absenteeism is responsible for as many classes missed as full-day absenteeism, raising chronic absenteeism from 9% to 24% of secondary-grades students. Incorporating part-day absences sharply increases the chronic absenteeism gap between underrepresented minority students and their peers. Both full- and part-day absenteeism show a discrete jump at the point of transition from middle school to high school, but full-day absenteeism then declines whereas part-day absenteeism remains high in Grades 10 and 11 and increases again in Grade 12. Whereas 55% of full-day absences are unexcused, 92% of part-day absences are unexcused. Absenteeism from individual classes varies considerably by time of day but less by class subject matter.

More →

Using student test scores to measure principal performance

By Jason Grissom, Demetra Kalogrides, Susanna Loeb. (2015). Education Evaluation and Policy Analysis

Researchers have devoted substantial attention to the use of student test score data to measure teacher performance. In response to recent policy interest in using student achievement data to measure the contributions of school administrators as well, this paper investigates the capacity of longitudinal achievement data to uncover principal effects. Building on prior research, it develops multiple models for capturing the contributions of principals to student test score growth, examines the properties of each model, and compares the results of the models empirically. It then assesses the degree to which the estimates from each model are consistent with measures of principal performance that come from sources other than student test scores, such as school district evaluations. It finds that the approach that attributes the school's effectiveness to the principal more closely aligns with non-test-based measures than do approaches that more convincingly separate the effect of the principal from the effects of other school inputs.

More →

In this policy brief Heather Hough and Susanna Loeb examine the effect of the Quality Teacher and Education Act of 2008 (QTEA) on teacher recruitment, retention, and overall teacher quality in the San Francisco Unified School District (SFUSD). They provide evidence that a salary increase can improve a school district’s attractiveness within their local teacher labor market and increase both the size and quality of the teacher applicant pool. They also provide evidence that targeted salary increases can increase the quality of new-hires. QTEA salary increases did not affect teacher retention, however, perhaps because the implementation of QTEA coincided with a major economic downturn that made many workers, including teachers, reluctant to leave their jobs.

More →

The Quality Teacher and Education Act: Second Year Report

By Heather Hough, David Plank, and Susanna Loeb. (2012)

This report presents findings from the second year of a three-year study on the implementation and effect of the Quality Teacher and Education Act (QTEA) in San Francisco Unified School District (SFUSD). In 2008, SFUSD enacted the Quality Teacher and Education Act (QTEA), which made changes to teacher compensation, support, and accountability. QTEA is part of SFUSD’s larger human capital system that is designed to attract, develop, motivate, and retain effective teachers with the overarching goal of improving student achievement.

More →

The Quality Teacher and Education Act: First Year Report

By Heather Hough, Susanna Loeb, and David Plank. (2011)

This report presents findings from the first year of a three-year study on the implementation and effect of the Quality Teacher and Education Act (QTEA) in San Francisco Unified School District (SFUSD). In 2008, SFUSD enacted the Quality Teacher and Education Act (QTEA), which made changes to teacher compensation, support, and accountability. QTEA is part of SFUSD’s larger human capital system that is designed to attract, develop, motivate, and retain effective teachers with the overarching goal of improving student achievement.

More →

New thinking about instructional leadership

By Susanna Loeb, Eileen Horng. (2010). Phi Delta Kappan

School leaders matter for school success. Numerous studies spanning the past three decades link high-quality leadership with positive school outcomes. Recognition of the importance of school leadership has led to increased attention to recruiting and preparing school leaders. Many new principal preparation and development programs emphasize the role of principals as instructional leaders. This emphasis on instructional leadership was driven in large part by the effective schools movement of the 1970s and 1980s and has since been renewed because of increasing demands that school leaders be held accountable for student performance (Hallinger 2005). However, while broad agreement exists on the importance of instructional leadership, there is less consensus on what instructional leadership actually is. Some construe instructional leadership as synonymous with classroom observations and direct teaching of students and teachers. Informed by observations and interviews in hundreds of schools, we call for a different view of instructional leadership, one that includes broader personnel practices and resource allocation practices as central to instructional improvement.

More →