Show me the data: 25 years of constrained and unconstrained skills
Four nationally representative datasets with information about constrained and unconstrained reading, math, general knowledge, and nonacademic skills.
Note: Unconstrained Kids unpacks, translates, and integrates academic research and data about constrained and unconstrained skills for people that run, fund, and assist organizations that teach and serve kids. This post describes a group of four nationally representative datasets I used to create a series of charts to illustrate proficiency patterns of a representative group of unconstrained nonacademic (executive) skills. For reference, see this working list of constrained and unconstrained skills. Like everything on this Substack, this post is a work-in-progress. I will make updates as needed. Questions, comments, and suggestions are welcome.
Last updated: March 15, 2025
Three big ideas
The best way to see differences between developmental patterns of constrained and unconstrained skills is to track kids over time. I use three longitudinal datasets that collectively cover the period from 1998 through 2016.
The three longitudinal datasets come from federally-funded studies that involved nationally representative samples of kids. A nationally representative sample enables us to make generalizations about kids across the United States.
The richest dataset for reading and math skills runs from 1998-2007. I use more recent point-in-time (cross-sectional) data to compare patterns of skill proficiency in the late 2000s to current patterns.
Longitudinal data
Constrained skill theory contends that typically developing kids achieve mastery of constrained skills. These skills involve relatively limited amounts of information which can be learned and mastered within bounded periods of time. By contrast, unconstrained skills, which involve the acquisition, integration, and mastery of a broader set of information, take more time for proficiency. Constrained skills are readily taught and assessed in formal learning environments (the classroom). Unconstrained skills are not easily taught or assessed and are acquired through formal and informal learning environments (“caught and taught”). This implies that at some point we should expect to see greater differences between kids on unconstrained skills than constrained skills.
This is a job for data.
Paris and Luo (2010) suggest that the best way to see differences in developmental patterns of constrained and unconstrained skills is to track students over time. To accomplish this task, I use data about U.S. students from three longitudinal datasets. Each of these studies included large numbers of children. Moreover, the studies were designed to be demographically representative of the entire country.
Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K). Tracked roughly 22,000 students from grades K-8. Ran from fall 1998 through spring 2007.
Early Childhood Longitudinal Study, Kindergarten Class of 2010-11 (ECLS-K:2011). Tracked about 18,000 students from grades K-5. Ran from fall 2010 through spring 2016.
High School Longitudinal Study of 2009 (HSLS:09). Tracked about 23,000 students from grade 9 to 11. Conducted in SY 2009-10 and SY 2011-12
The source for each of these datasets is the National Center for Education Statistics, which is part of the U.S. Department of Education’s Institute for Education Sciences. Data about reading and math in each dataset was assessed and reported in a manner that supports analysis of skill proficiency (from more to less constrained).
My “workhorse” dataset is the ECLS-K. It includes detailed data about reading and math skill development over a 9-year period (K-8). It also uniquely provides information about general knowledge development for grades K-1. The ECLS-K:2011 does not offer this level of detail about reading, math, or general knowledge. However, it does offer insight into the development of nonacademic skills (working memory and cognitive flexibility) for grades K-5. The HSLS:09 round out the picture with insight into the development of math skills in the high school years.
Cross-sectional data
There have been many policy, practice, and curricular changes in education over the past 25 years. Most of the data in these studies were collected over 15 years ago. It’s possible that any insights we might obtain are less relevant today. As a check, I lean upon a fifth nationally representative dataset:
National Assessment of Educational Progress Long-Term Trend Assessment (LTT). I use data from the NAEP LTT for 9- and 13-year-olds. There were roughly 7,400 9-year-olds and 8,700 13-year-olds. LTT is assessed during the school year: in October through December for 13-year-olds, January through March for 9-year-olds.
Like the three other datasets, the NAEP LTT also comes from the National Center for Education Statistics. This long-term trend version of the NAEP dates to the early 1970s. I use NAEP LTT data from before and after the outbreak of the COVID-19 epidemic. Performance levels on the NAEP LTT can readily be ordered from more to less constrained. By contrast, achievement levels on the main NAEP are not as amenable. Despite the name (“long-term trend”) the NAEP LTT assesses different groups of kids at a single point in time (a cross-section). Thus, we are not able to track children’s progress over time. Nonetheless, we can use the LTT to compare more recent patterns of skill development with those from 15+ years ago.
Twenty-five years of constrained and unconstrained data
Collectively, these four datasets provide a picture of constrained and unconstrained skill development from kindergarten through high school over a 25-year period. The data are analyzed alternatively by racial groups, parental education level, household income, or socioeconomic status. This is a work in progress. Despite my best efforts, some of this information may be incomplete or inaccurate. I will make revisions and updates as needed.
But wait, there’s more
If you’d like to see more data about constrained and unconstrained skills, check out these other posts on Unconstrained Kids:
Works cited
Paris, S. G., & Luo, S. W. (2010). Confounded statistical analyses hinder interpretation of the NELP report. Educational Researcher, 39(4), 316-322.