The radio pulsar B1951+32 and the supernova remnant CTB 80 provide a rich laboratory for the study of neutron stars and supernova remnants. Here, we present ground-based optical and near-infrared observations of them, along with X-ray observations with Chandra and a re-analysis of archival data obtained with the Hubble Space Telescope. The X-ray observations reveal a cometary pulsar wind nebula which appears to be confined by a bow shock produced by high-velocity motion of the pulsar, making PSR B1951+32 a rare pulsar exhibiting both an H alpha bow shock and a shocked X-ray pulsar wind nebula. The distribution of H alpha and radio continuum emission is indicative of a contact discontinuity of the shocked pulsar winds and shocked ambient medium at ~0.05 pc. On the other hand, the optical synchrotron knot of PSR B1951+32 likely has a flat spectrum in the optical and near-infrared wavebands, and our astrometry is consistent with only one of the two reported optical counterpart candidates for the pulsar.