Given the aging populations in many countries throughout the world, there is an increasing interest in lifestyle factors and interventions that will enhance the cognitive vitality of older adults and reduce the risk for age-related neurological disorders, such as Alzheimer's disease. In this review, we evaluate the hypothesis that physical activity and exercise might serve to protect, and also enhance, cognitive and brain function across the adult lifespan. To this end, we critically review three separate literatures that have examined the influence of physical activity and exercise on cognition, brain function and brain structure of adults, including epidemiological or prospective observational studies, randomized human clinical interventions and non-human animal studies. We suggest that this literature supports the claim that physical activity enhances cognitive and brain function, and protects against the development of neurodegenerative diseases. We discuss future directions to address currently unresolved questions, such as interactions between multiple lifestyle factors on offsetting or protecting against cognitive and neural decline, and conclude that physical activity is an inexpensive treatment that could have substantial preventative and restorative properties for cognitive and brain function.