A tremendous international effort is currently dedicated to observing the so-called $B$-modes of the Cosmic Microwave Background (CMB) polarisation. At the unprecedented sensitivity level that the new generation of CMB experiments aims to reach, every uncontrolled instrumental systematic effect will potentially result in an analysis bias that is larger than the much sought-after CMB $B$-mode signal. The absolute calibration of the polarisation angle is particularly important in this sense, as any associated error will end up in a leakage from the much larger $E$ modes into $B$ modes. The Crab nebula (Tau A), with its bright microwave synchrotron emission, is one of the few objects in the sky that can be used as absolute polarisation calibrators. In this paper we review the best current constraints on its polarisation angle from 23 to 353 GHz, at typical angular scales for CMB observations, from WMAP, XPOL, Planck and NIKA data. These polarisation angle measurements are compatible with a constant angle of $-88.19,^circpm0.33,^circ$. We study the uncertainty on this mean angle, making different considerations on how to combine the individual measurement errors. For each of the cases, we study the potential impact on the CMB $B$-mode spectrum and on the recovered $r$ parameter, through a likelihood analysis. We find that current constraints on the Crab polarisation angle, assuming it is constant through microwave frequencies, allow to calibrate experiments with an accuracy enabling the measurement of $rsim0.01$. On the other hand, even under the most optimistic assumptions, current constraints will lead to an important limitation for the detection of $rsim10^{-3}$. New realistic measurement of the Crab nebula can change this situation, by strengthening the assumption of the consistency across microwave frequencies and reducing the combined error.