Much of the information we hope to extract from the gravitational-waves signatures of compact binaries is only obtainable when we can accurately constrain the inclination of the source. In this paper, we discuss in detail a degeneracy between the measurement of the binary distance and inclination which limits our ability to accurately measure the inclination using gravitational waves alone. This degeneracy is exacerbated by the expected distribution of events in the universe, which leads us to prefer face-on systems at a greater distance. We use a simplified model that only considers the binary distance and orientation, and show that this gives comparable results to the full parameter estimates obtained from the binary neutron star merger GW170817. For the advanced LIGO-Virgo network, it is only signals which are close to edge-on, with an inclination greater than $sim 75^{circ}$ that will be distinguishable from face-on systems. For extended networks which have good sensitivity to both gravitational wave polarizations, for face-on systems we will only be able to constrain the inclination of a signal with SNR 20 to be $45^{circ}$ or less, and even for loud signals, with SNR of 100, the inclination of a face-on signal will only be constrained to $30^{circ}$. For black hole mergers observed at cosmological distances, in the absence of higher modes or orbital precession, the strong degeneracy between inclination and distance dominates the uncertainty in measurement of redshift and hence the masses of the black holes.