Emerging evidence indicates abnormal retinal micro-vasculature in schizophrenia (SCZ) and bipolar disorder (BD) and its relation to cognitive functions. However, the association of these abnormalities with the cognitive deficits in these disorders has not been examined till date. Hence, we explored this aspect in patients with SCZ, BD, and healthy volunteers (HV). We examined 34 with SCZ, 39 with BD, and 45 HV. Retinal images were acquired using nonmydriatic fundus camera. The retinal images were analyzed, and average diameters of retinal arterioles and venules were calculated. Working memory was assessed using computerized one-back test from Cogstate® battery. There was significant difference between groups in retinal venules and arterioles caliber (p < 0.001). Both SCZ and BD patients had wider venules and narrower arterioles. They had significantly lower working memory accuracy (p = 0.008) and higher log mean speed (p < 0.001). There was significant positive correlation between one-back test accuracy and retinal arteriolar caliber (r = 0.22; p = 0.01) and between log mean speed score and retinal venular caliber (r = 0.20; p = 0.02). Findings suggest association between working memory and retinal vascular caliber, a potential pointer towards understanding the vascular pathology in cognitive deficits in SCZ and BD. Future studies need to examine whether retinal vascular could be a biomarker for SCZ and BD.