We utilize for the first time HST ACS imaging to examine the structural properties of galaxies in the rest-frame U-V versus V-J diagram (i.e., the UVJ diagram) using a sample at 0.6<z<0.9 that reaches a low stellar mass limit (log M/Msun>10.25). The use of the UVJ diagram as a tool to distinguish quiescent galaxies from star forming galaxies (SFGs) is becoming more common due to its ability to separate red quiescent galaxies from reddened SFGs. Quiescent galaxies occupy a small and distinct region of UVJ color space and we find most of them to have concentrated profiles with high Sersic indices (n>2.5) and smooth structure characteristic of early-type systems. SFGs populate a broad, but well-defined sequence of UVJ colors and are comprised of objects with a mix of Sersic indices. Interestingly, most UVJ-selected SFGs with high Sersic indices also display structure due to dust and star formation typical of the n<2.5 SFGs and late-type systems. Finally, we find that the position of a SFG on the sequence of UVJ colors is determined to a large degree by the mass of the galaxy and its inclination. Systems that are closer to edge-on generally display redder colors and lower [OII]3727 luminosity per unit mass as a consequence of the reddening due to dust within the disks. We conclude that the two main features seen in UVJ color space correspond closely to the traditional morphological classes of early and late-type galaxies.