We aimed to characterize the nature and magnitude of cognitive decline in a group of adults with amnestic mild cognitive impairment (aMCI) with high and low levels of amyloid-β (Aβ) in relation to healthy older adults with low Aβ levels. Healthy older adults and adults with aMCI enrolled in the Australian Imaging, Biomarker, and Lifestyle study, completed the CogState brief battery at baseline and 18 months, and underwent positron emission tomography neuroimaging for Aβ at baseline. In this study, we included adults with MCI who had been classified as having high and low levels of Aβ and healthy older adults who had been classified as having low levels of Aβ. Linear model analyses adjusted for baseline cognitive function indicated that relative to healthy older adults with low Aβ, adults with aMCI and high Aβ showed greater decline in working memory and in verbal and visual episodic memory at 18 months. Adults with aMCI and low Aβ also showed greater decline in working memory; however they did not evidence any decline in episodic memory at 18 months. The results of our study suggests that relative to healthy older adults and adults with aMCI with low Aβ, adults with aMCI and high levels of Aβ showed faster rates of decline on measures of episodic memory over 18 months, and this was approximately twice that observed previously for healthy older adults with high Aβ levels.