Thin highly epitaxial BiFeO$_3$ films were prepared on SrTiO$_3$ (100) substrates by reactive magnetron co-sputtering. Detailed MOKE measurements on BiFeO$_3$/Co-Fe bilayers were performed to investigate the exchange bias as a function of the films thicknesses and Co-Fe stoichiometries. We found a maximum exchange bias of H$_{mathrm{eb}}$=92 Oe and a coercive field of H$_{mathrm{c}}$=89 Oe for a 12.5 nm thick BiFeO$_3$ film with a 2 nm thick Co layer. The unidirectional anisotropy is clearly visible in in-plane rotational MOKE measurements. AMR measurements reveal a strongly increasing coercivity with decreasing temperature, but no significant change in the exchange bias field.