With the development of extreme high contrast ground-based adaptive optics instruments and space missions aimed at detecting and characterizing Jupiter- and terrestrial-mass planets, it is critical that each target star be thoroughly vetted to determine whether it is a viable target given both the instrumental design and scientific goals of the program. With this in mind, we have conducted a high contrast imaging survey of mature AFGKM stars with the PALAO/PHARO instrument on the Palomar 200 inch telescope. The survey reached sensitivities sufficient to detect brown dwarf companions at separations of > 50 AU. The results of this survey will be utilized both by future direct imaging projects such as GPI, SPHERE and P1640 and indirect detection missions such as SIM Lite. Out of 84 targets, all but one have no close-in (0.45-1) companions and 64 (76%) have no stars at all within the 25 field-of-view. The sensitivity contrasts in the Ks passband ranged from 4.5 to 10 for this set of observations. These stars were selected as the best nearby targets for habitable planet searches owing to their long-lived habitable zones (> 1 billion years). We report two stars, GJ 454 and GJ 1020, with previously unpublished proper motion companions. In both cases, the companions are stellar in nature and are most likely M dwarfs based on their absolute magnitudes and colors. Based on our mass sensitivities and level of completeness, we can place an upper limit of ~17% on the presence of brown dwarf companions with masses >40 MJ at separations of 1 arcsecond. We also discuss the importance of including statistics on those stars with no detected companions in their field of view for the sake of future companion searches and an overall understanding of the population of low-mass objects around nearby stars.