For the non-conforming Crouzeix-Raviart boundary elements from [Heuer, Sayas: Crouzeix-Raviart boundary elements, Numer. Math. 112, 2009], we develop and analyze a posteriori error estimators based on the $h-h/2$ methodology. We discuss the optimal rate of convergence for uniform mesh refinement, and present a numerical experiment with singular data where our adaptive algorithm recovers the optimal rate while uniform mesh refinement is sub-optimal. We also discuss the case of reduced regularity by standard geometric singularities to conjecture that, in this situation, non-uniformly refined meshes are not superior to quasi-uniform meshes for Crouzeix-Raviart boundary elements.