Slowed information processing speed is among the earliest markers of cognitive impairment in multiple sclerosis (MS) and has been associated with white matter (WM) structural integrity. Localization of WM tracts associated with slowing, but not significant impairment, on specific cognitive tasks in pediatric and young age onset MS can facilitate early and effective therapeutic intervention. Diffusion tensor imaging data were collected on 25 MS patients and 24 controls who also underwent the Symbol Digit Modalities Test (SDMT) and the computer-based Cogstate simple and choice reaction time tests. Fractional anisotropy (FA), mean (MD), radial (RD) and axial (AD) diffusivities were correlated voxel-wise with processing speed measures. All DTI metrics of several white matter tracts were significantly different between groups (p < 0.05). Notably, higher MD, RD, and AD, but not FA, in the corpus callosum correlated with lower scores on both SDMT and simple reaction time. Additionally, all diffusivity metrics in the left corticospinal tract correlated negatively with SDMT scores, whereas only MD in the right superior fronto-occipital fasciculus correlated with simple reaction time. In conclusion, subtle slowing of processing speed is correlated with WM damage in the visual-motor processing pathways in patients with young age of MS onset.