We present optical and near-infrared adaptive optics (AO) imaging and spectroscopy of 13 ultracool (>M6) companions to late-type stars (K7-M4.5), most of which have recently been identified as candidate members of nearby young moving groups (YMGs; 8-120 Myr) in the literature. The inferred masses of the companions (~10-100 Mjup) are highly sensitive to the ages of the primary stars so we critically examine the kinematic and spectroscopic properties of each system to distinguish bona fide YMG members from old field interlopers. 2MASS J02155892-0929121 C is a new M7 substellar companion (40-60 Mjup) with clear spectroscopic signs of low gravity and hence youth. The primary, possibly a member of the ~40 Myr Tuc-Hor moving group, is visually resolved into three components, making it a young low-mass quadruple system in a compact (<100 AU) configuration. In addition, Li 1 $lambda$6708 absorption in the intermediate-gravity M7.5 companion 2MASS J15594729+4403595 B provides unambiguous evidence that it is young (<200 Myr) and resides below the hydrogen burning limit. Three new close-separation (<1) companions (2MASS J06475229-2523304 B, PYC J11519+0731 B, and GJ 4378 Ab) orbit stars previously reported as candidate YMG members, but instead are likely old (>1 Gyr) tidally-locked spectroscopic binaries without convincing kinematic associations with any known moving group. The high rate of false positives in the form of old active stars with YMG-like kinematics underscores the importance of radial velocity and parallax measurements to validate candidate young stars identified via proper motion and activity selection alone. Finally, we spectroscopically confirm the cool temperature and substellar nature of HD 23514 B, a recently discovered M8 benchmark brown dwarf orbiting the dustiest-known member of the Pleiades. [Abridged]