LOADING

Type to search

Using Longitudinal Student Mobility Across Schools and Districts to Identify At-Risk Students

Using Longitudinal Student Mobility Across Schools and Districts to Identify At-Risk Students

Dan Goldhaber
American Institutes for Research/CALDER
University of Washington/CEDR

Cory Koedel
CALDER
University of Missouri

Umut Ӧzek
American Institutes for Research/CALDER

Eric Parsons
University of Missouri

CALDER Policy Brief No. 25-0421

We use comprehensive administrative data from three states to document the relationships between geographic mobility and student outcomes during K-12 schooling. We focus specifically on nonstructural mobility events—which we define as school or district changes that do not occur naturally as the result of planned transitions between schools—and on longitudinal measures that capture these events cumulatively for students. We show that the number of nonstructural moves experienced by a student is a powerful indicator of low academic performance and graduation rates. Longitudinal information on student mobility is unlikely to be readily available to local practitioners—i.e., individual districts, schools, or teachers. However, due to recent investments in longitudinal data systems in most states, this information can be made available at low cost by state education agencies.

Introduction

We use administrative data panels in three states—Massachusetts, Missouri, and Washington—to document the link between educational outcomes and longitudinal measures of student mobility across schools and districts. We define two types of student mobility. The first, “structural mobility,” is when a student’s natural progression through the schooling system is such that a cross-school or cross-district move occurs due to school district structure (e.g., at the transition point from elementary to middle school). The second, “nonstructural mobility,” is when a student moves for another reason (e.g., a residential move due to economic instability—e.g., see Been et al., 2011; Desmond & Gershenson, 2016). Our goal is to examine the extent to which repeated nonstructural mobility during primary and secondary education is useful for identifying at-risk students.

We show that nonstructural mobility is strongly associated with at-risk status, as measured by low academic performance and graduation rates. For example, students in all three states who switch schools (non-structurally) three times during grades 3-12 score about 20 percentile points lower in the distribution of test performance in high school and are 9-27 percentage points less likely to graduate from high school, on average, compared to students without any nonstructural moves. Our findings are in line with existing evidence showing that mobile students are more likely to be at-risk than other students (Anderson and Leventhal, 2016; Barrat & Berliner, 2013; Courtney et al., 2004; Cowen, 2017; Cutuli et al., 2013; Ingersoll et al., 1989; Mehana and Reynolds, 2004; Rumberger and Larson, 1998).

Our unique contribution is to establish that the relationships between student mobility and outcomes are much stronger when the cumulative history of mobility is considered. In this way, our work mirrors findings by Michelmore and Dynarski (2017), who perform a conceptually similar analysis of the information conveyed by a student’s history of free and reduced-price lunch (FRL) eligibility to identify persistent poverty. Like their work, our findings point to the value of using longitudinal data to provide a more complete picture of student risk status than is available from cross-sectional data. Moreover, we show that longitudinal mobility captures unique information that is not captured by persistent poverty. In addition, longitudinal mobility is an appealing alternative to FRL-based measures in light of the USDA’s recent adoption of the Community Eligibility Provision (CEP) in the administration of the school lunch program. The CEP allows eligible districts and schools to designate all students as eligible for free meals, degrading the value of FRL data for identifying persistent poverty at the individual student level.

The evidence we present in this note is purely descriptive. The methods we employ are not suited to isolate the causal impacts of nonstructural mobility on students’ academic outcomes, but causal inference is not necessary for longitudinal information on student mobility to be put to good policy use. In particular, we show that longitudinal measures of mobility can be used by policymakers, administrators, and teachers to better identify at-risk students. A concrete policy recommendation based on our findings is that state education agencies construct panel measures of student mobility to help educators identify students who are likely in need of additional supports. Repeated mobility information over the span of K-12 schooling is unlikely to be readily available to individual districts, schools, or teachers, which highlights a valuable role that states can play as information providers by leveraging their broader data systems.[1]

Data and Measures

Our analysis is based on state administrative microdata covering nearly all students enrolled in public schools in Massachusetts, Missouri, and Washington. In each state we track two cohorts of students, who were enrolled in the third grade in either the 2006-07 or 2007-08 school years, from grades 3-12. Students who exit the public school system at any point prior to the 12th grade, either because they enroll in private schools or leave the state, are excluded from the data.[2] Our samples include 113,938 unique students from Massachusetts, 101,629 unique students from Missouri, and 120,294 unique students from Washington.

We follow these students as they progress through public schools and track their school and district transitions from the 3rd through 12th grades. We distinguish between structural school and district transitions, which are associated with normal promotion from one grade to the next, and nonstructural transitions that are unexpected given school and district grade configurations. Specifically, we define nonstructural transitions as follows. First, we identify the highest grade in each school in each year (e.g., grade-5 in a K-5 school). We define all school transitions in years other than those following a school’s terminal grade as nonstructural moves. If the move also entails a district switch, then this also counts as a nonstructural district move. If a school transition occurs immediately after a school’s terminal grade, the grade is not the highest grade served by the district, and the student moves to a new district, we also treat this transition as a nonstructural school and district move.[3]

We believe our definition of nonstructural moves is conservative; i.e., it likely understates the total number of moves associated with disruptions. In particular, school transitions that take place after a school’s terminal grade to another school in the same district are not counted as nonstructural moves, despite the fact that some of these moves may be to schools other than the “next-in-line” zoned school. Unfortunately, we are unable to cleanly identify such moves because we do not observe student addresses (and hence cannot identify zoned schools).[4]

To assess the extent to which longitudinal mobility is an indicator of students being at risk, we document the relationships between student mobility and three primary performance measures – achievement in third grade, achievement in high school, and high school graduation. We measure achievement in the third grade and high school by percentiles of the statewide test distribution.[5] The third-grade tests are common statewide exams in math and English language arts (ELA). The high school exams we use in each state are the MCAS ELA exam in Massachusetts, the English II end-of-course exam in Missouri, and the HSPE ELA exam in Washington. For the graduation outcome, a student is considered a graduate if she is observed graduating high school within one year of her expected graduation year based on normal grade progression from the third grade forward. To be included in the non-graduate sample, a student must be observed with an enrollment record indicating a dropout. Students who simply disappear from the cohort without a graduation or dropout record are treated as state leavers and omitted from the graduation portion of the analysis.

Table 1 provides descriptive information about structural and nonstructural student mobility in each state overall and broken out by student race-ethnicity and FRL status. Across the three states, students experience an average of 1.7-2.0 structural school moves during grades 3-12 and 0.66-0.95 nonstructural school moves. For district moves, structural moves are uncommon (students average just 0.01-0.12 structural district moves across states, reflecting the fact that most students attend districts that contain the full 3-12 grade span), but the average number of nonstructural moves is non-negligible, ranging from 0.37-0.58 across states. Consistent with the idea that nonstructural mobility serves as a general indicator of disadvantage, underrepresented-minority (Black and Hispanic) and FRL-eligible students are much more likely to experience non-structural moves than their socioeconomically more-advantaged peers. [6]

Figure 1 provides complementary distributional information about nonstructural mobility in the form of student shares in each state with 0, 1, 2, 3, and 4+ nonstructural moves between grades 3-12. About 50-60 percent of students across the sample states have zero nonstructural school moves, which leaves 40-50 percent with at least one move. And we find that 15-25 percent of students across the three states experience two or more nonstructural school moves.

The rates of nonstructural district mobility are lower; still, 8-15 percent of students experience two or more nonstructural district moves. Appendix Table A1 provides the tabular data underlying Figure 1, along with similar distributional information broken out by student race-ethnicity and poverty status.

Note that the move counts and shares in Table 1 and Figure 1 understate total nonstructural mobility because as noted above, we restrict our sample to students who remain in our sample states from grades 3-12. By definition, this excludes students who cross state boundaries, who are also nonstructural movers (the total number of moves for students who leave state databases are unobserved in our data). This constraint on the sample may also impact the gradients we show below connecting the number of nonstructural moves to student outcomes. If nonstructural movers who cross state boundaries are more (less) likely to be at-risk than nonstructural movers who remain within their states, then the gradients we show below will be understated (overstated).

Using Longitudinal Nonstructural Mobility to Identify At-Risk Students

Figure 2 shows percentile ranks on the high school test for students who had 0, 1, 2, 3, or 4+ nonstructural moves during grades 3-12. Panel A shows the results for school moves, and Panel B for district moves. The charts show that the relationship between achievement rank and mobility events is roughly linear in all three states, and as measured by mobility at the school or district level. This points to the value of tracking mobility longitudinally—e.g., knowing a student moved twice during grades 3-12 indicates roughly double the disadvantage of a student who moved just once (relative to a student with zero moves).[7]

Next, Figure 3 replicates the same trends for graduation rates. The trends are somewhat flatter here, especially for lower numbers of mobility events, likely reflecting the fact that graduation is a less differentiated outcome than test scores. There are also clear differences in the gradients across states, unlike in Figure 2. The gradient is flattest in MA, followed by MO, and is steepest in WA. This pattern aligns with evidence from Austin et al. (2020), who show that graduation rates are lowest among initially low-performing students in WA, followed by MO, and then MA using similar data. That is, graduation appears to be a more discriminate outcome in the states with the steeper gradients in Figure 3. This is consistent with interpreting the number of nonstructural mobility events as an indicator of at-risk status—the relationship becomes more apparent as the outcome metric becomes more discriminate.

We again note that these relationships are descriptive and should not be interpreted causally. As a way of illustrating this, Figure 4 replicates the mobility-event trends using students’ test percentile ranks in the third grade. The third-grade tests are taken prior to our mobility metrics between grades 3 and 12. The time inconsistency does not entirely preclude causal impacts of mobility on achievement because unmeasured mobility prior to the third grade is likely correlated with later-grade mobility. However, the similar trends for the third-grade and high-school tests in Figures 2 and 4 suggest that what we are largely capturing is not causal; rather, nonstructural mobility is serving as a general indicator of at-risk status. This interpretation is consistent with available research that aims to estimate plausibly causal effects of mobility, which finds negative but small mobility effects (e.g., Brummet, 2014; Grigg, 2012; Hanushek et al., 2004; Xu et al., 2009) or mixed effects depending on type of move (e.g., structural versus nonstructural moves, short- versus long-distance residential moves) as in Cordes et al. (2019) and Schwartz et al. (2017).

Appendix Tables A2-A4 provide tabular data corresponding to Figures 2-4, with additional subgroup splits by race-ethnicity and FRL status measured in the 3rd grade, like in Table 1. Qualitatively, the relationships between mobility and performance outcomes are similar within all race-ethnicity and poverty groups. There are modest fluctuations in the magnitude of the mobility relationship depending on the outcome measured and student group, and the relationships are generally less steep for students from more disadvantaged backgrounds. The flatter relationships are likely the product of weaker positive selection into the low-mobility categories among Black, Hispanic, and FRL students relative to Asian, White, and non-FRL students.[8]

Finally, in Appendix Tables A5 and A6, we examine the relationship between persistent poverty and longitudinal mobility to determine whether longitudinal mobility predicts variation in student at-risk status above and beyond persistent poverty. Appendix Table A5 presents (1) the distribution of nonstructural school and district mobility and (2) average high school test scores and high school graduation rates by student mobility conditional on persistent poverty. Appendix Table A6 repeats the same exercise using different definitions of partially persistent poverty based on the number of years the student was FRL-eligible between grades 3 and 12.[9] Both tables show a strong link between nonstructural mobility and student outcomes holding the level of (measured) persistent poverty constant. For example, among persistent FRL students, those who do not change schools non-structurally score 9 to 11 percentiles higher on high school tests and are 8 to 21 percentage points more likely to graduate from high school compared to those who change schools non-structurally three times.

Conclusion

The key contribution we make is showing that longitudinal data on nonstructural student moves between schools and districts can help identify at-risk students. These findings have implications for both research and policy similar to those articulated by Michelmore and Dynarski (2017) in their related study on longitudinal poverty status based on FRL data. Importantly, the longitudinal mobility metrics are complementary to persistent FRL designations as they identify dimensions of at-risk status not captured by FRL information alone. They are appealing to use not only because they supplement FRL-based information, but also because they are not affected by district and school adoptions of the Community Eligibility Provision.

The findings also indicate that research relying on cross-sectional mobility information to proxy for student at-risk status, which is the current norm, will not fully capture differences across student circumstances captured by longitudinal mobility. To the extent that these differences are correlated with specific interventions and/or teacher and school assignments, they could cause bias in causal studies if left uncontrolled.

From a policy perspective, the main takeaway from our study is that state longitudinal data systems contain information that policymakers and practitioners can use to better target resources toward high-need students. It is unlikely that individual districts, schools, and teachers will have the ability to effectively collect data on longitudinal mobility. However, state education agencies can produce this information at low marginal cost given recent investments in state longitudinal data systems. Our findings point to a clear role of state education agencies in providing this type of information to local education actors in an effort to better serve at-risk students.


[1] In states with existing or emerging “early warning systems,” repeated student mobility is a strong candidate for inclusion. The National Forum on Education Statistics (2018) reports that mobility is used by most existing systems, although information on whether longitudinal mobility is considered in most systems is not provided.

[2] We set the initial cohorts based on all students who have either 3rd grade math or reading/ELA test scores. Although most students in our initial third-grade cohorts remain in-state in the public school system through high school (about 80 percent), our samples of state stayers are more advantaged and higher achieving, on average, than the full population (Austin et al., 2020).

[3] There are, for instance, K-5, K-6, and K-8 school districts in each of the three states.

[4] There is also some measurement error in the mobility data stemming from district restructurings and mergers. These can create excess structural and nonstructural moves and will cause very modest attenuation bias in our estimates (because most students do not experience restructurings or mergers). If states package information on longitudinal student mobility to districts and schools per our policy recommendation, data on restructurings and mergers could be used to remove the small amount of measurement error in the data created by these events.

[5] Percentile ranks are cohort-specific and based on the full entry cohort with values imputed for students with missing exam scores (to account for sample attrition over time). Details of the imputation procedure are available from the authors upon request.

[6] We also observe discrepancies in the number of structural moves across racial groups and groups based on FRL eligibility. This could be driven by the fact that students from disadvantaged backgrounds (e.g., racial minorities, FRL eligible students) are more likely to attend schools with wider grade bands in the three states we examine.

[7] Michelmore and Dynarksi (2017) find a similar near-linear relationship between exam scores and the number of grades spent in economic disadvantage.

[8] Put another way, within mobility categories there is less variation across student groups in the high-mobility categories than the low-mobility categories.

[9] We define persistent poverty in Table A5 as being FRL eligible in all years during grades 3-12; Table A6 shows results using a variety of “partial persistence” poverty metrics.

Appendix- Supplementary Tables