We present the stellar and ionized gas kinematics of 13 bright peculiar Virgo cluster galaxies observed with the DensePak Integral Field Unit at the WIYN 3.5-meter telescope, to seek kinematic evidence that these galaxies have experienced gravitational interactions or gas stripping. 2-Dimensional maps of the stellar velocity $V$, and stellar velocity dispersion $sigma$ and the ionized gas velocity (H$beta$ and/or [ion{O}{3}]) are presented for galaxies in the sample. The stellar rotation curves and velocity dispersion profiles are determined for 13 galaxies, and the ionized gas rotation curves are determined for 6 galaxies. Misalignments between the optical and kinematical major axis are found in several galaxies. While in some cases this is due to a bar, in other cases it seems associated with a gravitational interaction or ongoing ram pressure stripping. Non-circular gas motions are found in nine galaxies, with various causes including bars, nuclear outflows, or gravitational disturbances. Several galaxies have signatures of kinematically distinct stellar components, which are likely signatures of accretion or mergers. We compute for all galaxies the angular momentum parameter $lambda_{rm R}$. An evaluation of the galaxies in the $lambda_{rm R}$-ellipticity plane shows that all but 2 of the galaxies have significant support from random stellar motions, and have likely experienced gravitational interactions. This includes some galaxies with very small bulges and truncated/compact H$alpha$ morphologies, indicating that such galaxies cannot be fully explained by simple ram pressure stripping, but must have had significant gravitational encounters. Most of the sample galaxies show evidence for ICM-ISM stripping as well as gravitational interactions, indicating that the evolution of a significant fraction of cluster galaxies is likely strongly impacted by both effects.