Prematurity is associated with diverse developmental abnormalities, yet few studies relate cognitive and neurostructural deficits to a dimensional measure of prematurity. Leveraging a large sample of children, adolescents, and young adults (age 8-22 years) studied as part of the Philadelphia Neurodevelopmental Cohort, we examined how variation in gestational age impacted cognition and brain structure later in development. Participants included 72 preterm youth born before 37 weeks’ gestation and 206 youth who were born at term (37 weeks or later). Using a previously-validated factor analysis, cognitive performance was assessed in three domains: (1) executive function and complex reasoning, (2) social cognition, and (3) episodic memory. All participants completed T1-weighted neuroimaging at 3 T to measure brain volume. Structural covariance networks were delineated using non-negative matrix factorization, an advanced multivariate analysis technique. Lower gestational age was associated with both deficits in executive function and reduced volume within 11 of 26 structural covariance networks, which included orbitofrontal, temporal, and parietal cortices as well as subcortical regions including the hippocampus. Notably, the relationship between lower gestational age and executive dysfunction was accounted for in part by structural network deficits. Together, these findings emphasize the durable impact of prematurity on cognition and brain structure, which persists across development.