We measured resistivity and specific heat of high-quality CsFe$_2$As$_2$ single crystals, which were grown by using a self-flux method. The CsFe$_2$As$_2$ crystal shows sharp superconducting transition at 1.8 K with the transition width of 0.1 K. The sharp superconducting transition and pronounced jump in specific heat indicate high quality of the crystals. Analysis on the superconducting-state specific heat supports unconventional pairing symmetry in CsFe$_2$As$_2$.