American Institutes for Research/CALDER
CALDER Policy Brief No. 10-0918-1
Principals are widely seen as a key influence on the educational environment in the schools they lead. Despite belief in the importance of principals, research has only recently focused on the import of principals for student success while taking advantage of longitudinal student-level data to address empirical challenges. To this end, several studies consider the overall effectiveness of principals as measured by value-added; this metric suggests that principals do influence student test scores, but the estimated magnitude of the influence is sensitive to statistical modeling choices. Other research suggests positive associations between principals’ time use, organizational management skills, and experience and student test achievement. Principal quality also appears to play an important role in teacher turnover and differential retention of effective teachers. That said, many empirical challenges remain, and there is relatively little research on the effects of principals on student test achievement and other student outcomes or whether principal effectiveness is associated with the path to the principalship (e.g., prior schooling experiences or training).
Do principals matter for student achievement? There are many reasons to think the answer is “yes.” For instance, principals could affect students directly by interacting with students and teachers. Principals are in fact widely viewed as “instructional leaders” who are responsible for facilitating instruction and student learning in the classroom (Hallinger & Murphy, 1985; Blasé & Blasé, 2004; Bottoms & O’Neill, 2001; Glickman, Gordon, & Ross-Gordon, 2014; Hoy & Hoy, 2003). In contrast, other researchers have noted that principals are responsible for the organizational management of schools (March, 1978; Heck, 1992; van de Grift & Houtveen, 1999; Balu, Horng, & Loeb, 2010; Grissom & Loeb, 2011), including tasks like managing day-to-day operations necessary to keep schools running smoothly and indirectly affect student achievement. In addition to interacting with students and teachers, principals may also change the composition of teachers who work at the school by hiring, counseling out, or influencing retention.
Yet, despite belief in the importance of principals, research evidence is less clear on which factors are associated with improved student outcomes. This lack of evidence is problematic for policy and raises questions such as, “How should principals be selected from the general population of education staff?” and “What types of job skills are associated with success?” Answers to these questions are important for informing the way we train prospective principals and how we support principals who are currently employed.
There are many early studies on principals which I do not discuss in detail; instead, there are several literature reviews and meta-analyses that highlight concerns with this research. In general, these studies focus on particular tasks or characteristics of principals that are likely to have a direct impact on student achievement (e.g., time spent on conflict resolution, principal’s education level), and few of these findings are consistent across this early research. This may not be surprising because there are many empirical challenges in relating principals to student achievement. One possibility is that the principal role is difficult to model empirically because they are responsible for a myriad of tasks within the school (Brewer, 1993). Research by Horng, Klasik, and Loeb (2010) sheds light on these roles and allocations of principal time, which they classify into six categories: Administration, Organization Management, Instructional Program, Day-to-Day Instruction, Internal Relations, and External Relations. The researchers find that principals tend to spend the most time on administrative tasks, about 30% of the school day; 20% on organization management; and 20% on Internal and External Relations (the remaining 30% is spent on other, miscellaneous tasks). Given the complexity of the principal role, directly modeling the associations between principal tasks and student achievement seems particularly daunting. For instance, it is likely that schools have unique needs so that tasks that are important for one school may be less relevant for another.
Another possibility is that there are unique empirical challenges when studying principals that are not present in other education topics, such as research on teachers. As noted by numerous researchers (Grissom et al., 2015; Chiang et al., 2016), a key empirical issue is separating the effects of schools from the effects of principals. Moreover, principal studies tend to have much smaller sample sizes than other studies, which could lead to results that are statistically insignificant but important in magnitude. Additionally, very few papers attempt to control for nonrandom selection of students and teachers to higher performing principals or omitted variables that are related to both student achievement and principal factors. In particular, virtually no studies use randomized controlled trials (RCTs) or quasi-experiments to evaluate principal effectiveness, and few control for prior student achievement.
In what follows, I discuss the recent research that has attempted to address some of these concerns by estimating effects using more sophisticated techniques: controlling for observable characteristics of students and setting, estimating gains in test scores instead of levels, and leveraging panel data to consider “within-school” effects. I begin by discussing research on the distribution of principal effectiveness and then discuss studies that identify particular factors related to student achievement.
Estimating the Distribution of Principal Effectiveness
Recently, researchers have estimated principal value-added models to investigate the overall impacts of principals on students (instead of specifying specific characteristics or tasks), and to quantify the distribution of principal effectiveness. These value-added models attribute the improvements in student achievement between current scores and previous scores to principals while taking into account the observable characteristics of students, classes, and schools. It is important to note that relatively little research has focused on the unique empirical challenges for principal value-added models, and estimating principal effectiveness is challenging for several reasons. A primary concern is the difficulty of separating principal effects from school effects, which is important because many school factors are not under the control of principals (Grissom et al., 2015; Chiang et al., 2016). Several studies use “within-school” variation to compare principal performance in a similar setting, although this approach has several drawbacks. Another problem, noted by Clark, Martorell, and Rockoff (2009), is that students are repeatedly served by the same principal, leading to complicated, dynamic contributions of principals over time. Lastly, principal turnover is preceded by declines in student achievement, so newly hired principals may appear more effective than they are due to mean reversion (Miller, 2013).
With these challenges in mind, several studies have used a variety of settings and approaches to estimate principal value-added. Branch et al. (2009) use data from Texas and consider both across- and within-school estimates of value-added. They find substantial variation in principal effectiveness, with a 1–standard deviation increase in principal value-added associated with a 0.11–standard deviation increase in math scores when using within-school variation. Dhuey and Smith (2014) consider a unique setting in British Colombia where principals are routinely rotated between schools, and they use this variation in principal placement to identify within-school estimates of principal effectiveness. They find somewhat comparable distributions of principal quality, with a 1–standard deviation improvement in principal value-added associated with an increase in student achievement of 0.289 to 0.408 standard deviations in reading and math between Grades 4 and 7, suggesting an average improvement of 0.096 to 0.136 per grade. Other work by Dhuey and Smith (2018) uses North Carolina data to estimate within-school principal fixed effects. They find substantial variation in effectiveness, and a large portion appears to be related to “match” effects; that is, principals perform better in some schools rather than others. This is related to work by Cannon, Figlio, and Sass (2012), who also find that principal value-added tends to decline when principals change school settings.
Grissom et al. (2015) and Chiang et al. (2016) compare principal value-added estimates across different specifications. Grissom et al. (2015) find that the variance of principal effectiveness depends on the chosen specification, with narrower within-school variation estimates relative to across-school estimates. Estimates of principal value-added are positively correlated, although not particularly closely (correlations range from 0.4 to 0.6 across models). Interestingly, they also find that both across- and within-school estimates of principal effectiveness are associated with principal evaluations (average rating, school climate, accountability grade). Chiang et al. (2016) consider a relatively more narrow question: Are the school value-added models, typically used for accountability purposes, predictive of principal value-added estimates? In contrast to relatively strong correlations across specifications, they find that school value-added does not predict within-school, principal value-added (regression coefficient of 0.07, and not statistically significant).
Principal Factors That Predict Student Achievement
One thread of research relates principal practice and skills directly to student achievement instead of estimating value-added models. In terms of practice, Eberts and Stone (1988) use a national sample of elementary school students and find that instructional leadership and conflict resolution are positively associated with student achievement. In contrast, Brewer (1993) finds that teacher selection and goal setting are important predictors of student achievement gains (as well as higher principal salaries). More recent research by Grissom, Loeb, and Master (2013) suggests that principals who spend time in coaching and evaluation tend to have better student outcomes. In addition to time spent on tasks, principal skill also appears to be important. Grissom and Loeb (2011) use assessments of principal skills, as perceived by principals and assistant principals, to estimate associations with student achievement and find that organization management skills are associated with improved student outcomes.
Several studies have considered the influence of principals on teacher turnover. There is evidence that teacher perceptions and evaluation ratings indicating low principal quality tend to predict teacher turnover (Ladd, 2011; Boyd et al., 2011; Grissom, 2011). Subsequent work by Grissom and Bartanen (2018) suggests that effective principals tend to have lower rates of turnover among high-performing teachers and higher rates of turnover among low performers. This has important implications for principal effectiveness and may suggest a mechanism for improvements in principal effectiveness over time.
Other research focuses on the characteristics and training of principals. Perhaps most common are studies focusing on the returns to principal experience. For instance, Clark et al. (2009) find increasing returns to experience, as do Branch et al. (2009). That said, other work on principal transitions highlights reasons to be cautious when estimating returns to experience: Miller (2013) finds that schools have declining performance prior to hiring a new principal and the subsequent increases in achievement afterward could reflect mean reversion and not improvements in principal performance. As such, many studies exclude the first and last year of service in a particular school, though Miller’s results suggest that mean reversion in test scores could take several years.
In contrast to experience, few other principal characteristics appear to matter for student achievement. Clark et al. (2009) find little evidence that the level of degree attainment or prestige of the degree-granting institution are associated with student outcomes; interestingly, this mirrors some findings from the teacher value-added literature on degree attainment and credentials (e.g., Chingos & Peterson, 2011; Goldhaber & Brewer, 2000). Very few studies have considered principal preparation, but one exception is Grissom, Mitani, and Woo (2018). They compare principal preparation programs in Tennessee, and while they find differences in principal effectiveness across programs, these differences depend on the measure used to evaluate principal effectiveness. Relatedly, Jacob et al. (2015) study an RCT for prospective principals that aimed to improve principal leadership. While the program led to more principals feeling more effective, there was no impact on student achievement, and teacher assessments suggested no difference in instructional climate between treatment and control group principals.
There are many reasons to be cautious about the evidence on principals. Estimates of principal value-added generally suggest important differences in principal effectiveness, but they vary widely in their estimates across specifications and the evidence does not point to a single, preferred specification. One reason could be that the empirical challenges discussed above are difficult to address. Indeed, there has not been as much focus on testing the bias of principal value-added relative to the teacher literature (see Chetty, Friedman, & Rockoff, 2014; Kane & Staiger, 2008). Consequently, researchers such as Grissom et al. (2015) suggest caution in choosing between value-added model specifications.
It is worth noting that many of these modeling challenges are not unique to value-added but also apply to studies on principal practice. For instance, Hallinger and Heck (2011) discuss the possibility that measures of principal leadership are influenced by the school setting, which suggests problems of omitted variable bias. Future research in this area could consider within-school variation in these settings, similar to some value-added estimates, while noting the stringent requirements.
The lack of rigorous evidence is problematic for interventions that are intended to improve principal effectiveness. For instance, preparation programs have little information to use in selecting the most qualified principal candidates according to characteristics that predict student achievement. Research by Winograd, Garcia, and Dasenbrock (2008) suggests that principal preparation programs in New Mexico are not highly selective when choosing candidates; instead, principals are chosen based on availability. Similarly, there is relatively little research on whether a principal’s path to the principalship is predictive of effectiveness in terms of prior experience teaching or serving as an assistant principal. Indeed, we know very little in general about what the path to the principalship looks like. Additionally, principal training cannot be tied to teaching practices that are effective for student test scores; instead, states may require very little training. For instance, Indiana requires as little as 20 hours of graduate coursework to become a principal in that state. Lastly, there is little guidance for professional development or support for principals who are currently employed.
Another missing element of the research on principal effectiveness is the lack of effects on non-test outcomes for students. In particular, very few studies consider whether principals have a measurable effect on long-run, postsecondary outcomes such as college enrollment or employment. This is important if principals impact noncognitive learning via the school environment, as these skills are predictive of later life outcomes (e.g., Heckman, Stixrud, & Urzua, 2006). Other studies have found that educational interventions have impacts that are difficult to measure using test scores, such as work by Dynarski et al. (2013) showing that test score effects fade out over time but still provide accurate predictions of later life outcomes. There is also some concern in research on school choice; for some studies, test scores do not reflect improvement in other outcomes (graduation, postsecondary enrollment), and other studies find positive test score effects but no improvement in other outcomes (Hitt, McShane, & Wolf, 2018).
Lastly, principal effectiveness likely depends on the amount of discretion that principals have on school operations. Given that teachers have been consistently shown to be the most important schooling attribute influencing student outcomes, principals may be able to improve student outcomes if they are able to hire effective teachers or dismiss ineffective teachers. If principals cannot change the composition of teachers within the school, then a potent mechanism for improving student outcomes is closed. That said, it is worth noting that providing information about teacher quality to principals appears to have important impacts on personnel decisions (Rockoff et al., 2012), and there is some evidence that principals could use applicant selection instruments to pick more effective teachers (Goldhaber, Grout, & Huntington-Klein, 2017). Similarly, recent practices require a large portion of principal time to be allocated to evaluating teachers; if this activity is not useful, then it will likely divert time from other practices that could improve student achievement.
There are many potential policy levers that could be used to influence principal effectiveness. All 50 states have adopted standards that are required in order to serve as a school principal, and many of these standards differ across states and many practices differ. For example, 37 states require that principals have a master’s degree and 3 years of teaching experience, and 38 require field experience. In addition to traditional preparation programs, 39 states allow for alternative requirements depending on applicant qualifications, which vary widely in their requirements. Clearly, there are very different approaches used to determine who should be eligible for a principalship, but the lack of evidence on the relationship between principals’ training and prior experiences and their impacts on schools and students means that policy decisions, such as the implementation of standards, are largely being made in an empirical vacuum.
There are similar concerns about what interventions should be used to support inservice principals. In practice, one could advocate for more professional development or more spending on administrative support; however, there is no evidence on which skills professional development for principals should target to improve student outcomes or whether additional administrative support would allow principals to pursue other, more effective uses of their time. That said, there are a few promising associations between principal characteristics and student outcomes that may help inform future policy. Time use is important; more time spent on instructional leadership, conflict resolution, teacher selection, and goal setting is associated with improved student outcomes. Principal skills in organization management may also matter for student achievement. Several studies also suggest that there are returns to experience, and thus, potential costs to principal turnover. Lastly, there is a growing body of work that suggests principals play an important role in teacher turnover, and providing principals with information either through test score–based measures or observational ratings could be an important policy lever (e.g., Goldring et al., 2015).
Aaronson, D., Barrow, L. and Sander, W., 2007. Teachers and student achievement in the Chicago public high schools. Journal of labor Economics, 25(1), pp.95-135.
Balu, R., Horng, E.L. and Loeb, S., 2010. Strategic personnel management: How school principals recruit, retain, develop and remove teachers. School Leadership Research, Working Paper, pp.10-6.
Blase, J. and Blase, J., 2004. Handbook of instructional leadership. How successful principals promote teaching and learning, 2.
Bottoms, G. and O’neill, K., 2001. Preparing a new breed of school principals: It’s time for action. Atlanta, GA: Southern Regional Education Board.
Boyd, D., Grossman, P., Ing, M., Lankford, H., Loeb, S. and Wyckoff, J., 2011. The influence of school administrators on teacher retention decisions. American Educational Research Journal, 48(2), pp.303-333.
Branch, G., Hanushek, E., & Rivkin, S. (2009). Estimating principal effectiveness (Working Paper 32). Washington, DC: National Center for Analysis of Longitudinal Data in Education Research.
Brewer, D. J. (1993). Principals and student outcomes: Evidence from U.S. high schools. Economics of Education Review, 12(4), 281–292.
Cannon, S., Figlio, D., & Sass, T. (2012). Principal quality and the persistence of school policies. Unpublished manuscript.
Chamberlain, G.E., 2013. Predictive effects of teachers and schools on test scores, college attendance, and earnings. Proceedings of the National Academy of Sciences, p.201315746.
Chetty, R., Friedman, J.N. and Rockoff, J.E., 2014. Measuring the impacts of teachers I: Evaluating bias in teacher value-added estimates. American Economic Review, 104(9), pp.2593-2632.
Chiang, H., Lipscomb, S., & Gill, B. (2016). Is school value added indicative of principal quality? Education Finance and Policy, 11(3), 283–309.
Chingos, M. M., & Peterson, P. E. (2011). It’s easier to pick a good teacher than to train one: Familiar and new results on the correlates of teacher effectiveness. Economics of Education Review, 30(3), 449–465.
Clark, D., Martorell, P., & Rockoff, J. (2009). School principals and school performance (Working Paper 38). Washington, DC: National Center for Analysis of longitudinal data in Education research.
Coelli, M., & Green, D. A. (2012). Leadership effects: School principals and student outcomes. Economics of Education Review, 31(1), 92–109.
Dhuey, E. and Smith, J., 2014. How important are school principals in the production of student achievement?. Canadian Journal of Economics/Revue canadienne d’économique, 47(2), pp.634-663.
Dhuey, E., & Smith, J. (2018). How school principals influence student learning. Empirical Economics, 54(2), 851–882.
Dynarski, S., Hyman, J. and Schanzenbach, D.W., 2013. Experimental evidence on the effect of childhood investments on postsecondary attainment and degree completion. Journal of Policy Analysis and Management, 32(4), pp.692-717.
Eberts, R. W., & Stone, J. A. (1988). Student achievement in public schools: Do principals make a difference? Economics of Education Review, 7(3), 291–299.
Glickman, C.D., Gordon, S.P. and Ross-Gordon, J.M., 2005. The basic guide to supervision and instructional leadership. Pearson/Allyn & Bacon.
Goldhaber, D. D., & Brewer, D. J. (2000). Does teacher certification matter? High school teacher certification status and student achievement. Educational Evaluation and Policy Analysis, 22(2), 129–145.
Goldhaber, D. and Hansen, M., 2013. Is it just a bad class? Assessing the long‐term stability of estimated teacher performance. Economica, 80(319), pp.589-612.
Goldhaber, D., Grout, C. and Huntington-Klein, N., 2017. Screen twice, cut once: Assessing the predictive validity of applicant selection tools. Education Finance and Policy, 12(2), pp.197-223.
Goldring, E., Grissom, J. A., Rubin, M., Neumerski, C. M., Cannata, M., Drake, T., & Schuermann, P. (2015). Make room value added: Principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher, 44(2), 96–104.
Grissom, J.A., 2011. Can good principals keep teachers in disadvantaged schools? Linking principal effectiveness to teacher satisfaction and turnover in hard-to-staff environments. Teachers College Record, 113(11), pp.2552-2585.
Grissom, J. A., & Loeb, S. (2011). Triangulating principal effectiveness: How perspectives of parents, teachers, and assistant principals identify the central importance of managerial skills. American Educational Research Journal, 48(5), 1091–1123.
Grissom, J. A., Kalogrides, D., & Loeb, S. (2015). Using student test scores to measure principal performance. Educational Evaluation and Policy Analysis, 37(1), 3–28.
Grissom, J. A., Loeb, S., & Master, B. (2013). Effective instructional time use for school leaders: Longitudinal evidence from observations of principals. Educational Researcher, 42(8), 433–444.
Grissom, J.A. and Bartanen, B., 2018. Strategic Retention: Principal Effectiveness and Teacher Turnover in Multiple-Measure Teacher Evaluation Systems. American Educational Research Journal, p.0002831218797931.
Grissom, J.A., Mitani, H. and Woo, D.S., 2018. Principal Preparation Programs and Principal Outcomes. Educational Administration Quarterly, p.0013161X18785865.
Hallinger, P. and Murphy, J., 1987. Assessing and developing principal instructional leadership. Educational leadership, 45(1), pp.54-61.
Hallinger, P., & Heck, R. H. 1996. Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32(1), 5–44.
Hallinger, P. and Heck, R.H., 2011. Conceptual and methodological issues in studying school leadership effects as a reciprocal process. School Effectiveness and School Improvement, 22(2), pp.149-173.
Heck, R.H., 1992. Principals’ instructional leadership and school performance: Implications for policy development. Educational evaluation and policy analysis, 14(1), pp.21-34.
Heckman, J.J., Stixrud, J. and Urzua, S., 2006. The effects of cognitive and noncognitive abilities on labor market outcomes and social behavior. Journal of Labor economics, 24(3), pp.411-482.
Hitt, C., McShane, M.Q. and Wolf, P.J., 2018. Do impacts on test scores even matter? Lessons from long-run outcomes in school choice research. American Enterprise Institute. Retrieved from http://www. aei. org/publication/do-impacts-on-test-scores-even-matterlessons-from-long-run-outcomes-in-school-choice-research.
Horng, E. L., Klasik, D., & Loeb, S. (2010). Principal’s time use and school effectiveness. American Journal of Education, 116(4), 491–523.
Hoy, A.W., Hoy, W.K. and Hoy, A.W., 2003. Instructional leadership: A learning-centered guide.
Jacob, R., Goddard, R., Kim, M., Miller, R. and Goddard, Y., 2015. Exploring the causal impact of the McREL Balanced Leadership Program on leadership, principal efficacy, instructional climate, educator turnover, and student achievement. Educational Evaluation and Policy Analysis, 37(3), pp.314-332.
Ladd, H.F., 2011. Teachers’ perceptions of their working conditions: How predictive of planned and actual teacher movement?. Educational Evaluation and Policy Analysis, 33(2), pp.235-261.
Leithwood, K. and Jantzi, D., 2005. A review of transformational school leadership research 1996–2005. Leadership and policy in schools, 4(3), pp.177-199.
March, J.G., 1978. American public school administration: A short analysis. The School Review, 86(2), pp.217-250.
Marzano, R.J., Waters, T. and McNulty, B.A., 2005. School leadership that works: From research to results. ASCD.
Miller, A. (2013). Principal turnover and student achievement. Economics of Education Review, 36, 60–72.
Robinson, V.M., Lloyd, C.A. and Rowe, K.J., 2008. The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational administration quarterly, 44(5), pp.635-674.
Rockoff, J.E., Staiger, D.O., Kane, T.J. and Taylor, E.S., 2012. Information and employee evaluation: Evidence from a randomized intervention in public schools. American Economic Review, 102(7), pp.3184-3213.
van de Grift, W., & Houtveen, A. A. M. (1999). Educational leadership and pupil achievement in primary education. School Effectiveness and School Improvement, 10(4), 373–389.
Winograd, P., Garcia, V., & Dasenbrock, R. (2008). Strong Leaders for New Mexico Schools: A
report to the Legislative Education Study Committee. Senate Joint Memorial 3: Reports
Witziers, B., Bosker, R. J., & Krüger, M. L. (2003). Educational leadership and student achievement: The elusive search for an association. Educational Administration Quarterly, 39(3), 398–425.
 Researchers, including Eberts and Stone (1988), have noted that using student outcomes directly is important because other measures of principal effectiveness, such as evaluations of principal practice, focus on skills that have no relation to improving education for students. See also Ginsberg and Thompson (1992).
 See Hallinger and Heck (1996); Leithwood and Jantzi (2005); Marzano, Waters, and McNulty (2005);
and Robinson, Lloyd, and Rowe (2008).
 Horng, Klasik, and Loeb (2010) also consider associations to student achievement, which we describe below.
 Another possibility is that early studies tended to focus on the direct effects of principals on students instead of indirect effects. To this end, a meta-analysis by Witziers, Bosker, and Kruger (2003) provide a meta-analysis that finds that direct effects have only small positive effects. This could suggest that, if principals have a large impact on student achievement, the mechanism is indirect and principal effectiveness may be important but difficult to predict with observable factors.
 For example, previous research suggests that time spent on teacher selection is associated with improved student outcomes, but most principals are not responsible for hiring most of their teaching staff; instead, they inherit teachers selected by previous principals. How much responsibility should principals have over the performance of teachers they didn’t hire?
 For instance, these studies limit their analysis to a subset of principals whose performance can be compared in the same school and rely on the overlap between principals who change schools.
 Related research by Coelli and Green (2012), also from British Columbia, estimates the variance of high school principal effectiveness directly. They find that it takes several years for principals to have a large effect on students.
 They also attempt to address concerns of spurious correlation between models by estimating on different samples of students, although this requires additional assumptions about the persistence of principal effectiveness.
 These studies represent some of the earliest work on principals that focus on controlling for the characteristics of students and schools, including measures of prior student achievement.
 This topic is the focus of a recent CALDER cross-state project.
 Many studies have considered the relationship between student outcomes and teacher value-added. For example, see Aaronson et al., 2007; Goldhaber and Hansen, 2013; Chamberlain, 2013; and Chetty et al., 2014.
 As reported by the Education Commission of the States, https://www.ecs.org/50-state-comparison-school-leader-certification-and-preparation-programs, retrieved 9/26/18.
 For example, Utah makes exceptions for individuals with “exceptional professional experience,” while Virginia allows for exceptions based on concentrations of graduate coursework in “school law, evaluation of instruction, and other areas of study required by the employing Virginia school superintendent.”