We identified a highly-polarized, steep-spectrum radio source in a deep image with the Australian Square Kilometre Array Pathfinder (ASKAP) telescope at 888 MHz. After considering and rejecting a stellar origin for this source, we discovered a new millisecond pulsar (MSP) using observations from the Parkes radio telescope. This pulsar has period 2.77 ms and dispersion measure 228.27 pc/cm**3. Although this pulsar does not yet appear to be particularly remarkable, the short spin period, wide profile and high dispersion measure do make it relatively hard to discover through traditional blind periodicity searches. Over the course of several weeks we see changes in the barycentric period of this pulsar that are consistent with orbital motion in a binary system, but the properties of any binary need to be confirmed by further observations. While even a deep ASKAP survey may not identify large numbers of new MSPs compared to the existing population, it would be competitive with existing all-sky surveys and could discover interesting new MSPs at high Galactic latitude without the need for computationally-expensive all-sky periodicity searches.