A prototype detection unit of the KM3NeT deep-sea neutrino telescope has been installed at 3500m depth 80km offshore the Italian coast. KM3NeT in its final configuration will contain several hundreds of detection units. Each detection unit is a mechanical structure anchored to the sea floor, held vertical by a submerged buoy and supporting optical modules for the detection of Cherenkov light emitted by charged secondary particles emerging from neutrino interactions. This prototype string implements three optical modules with 31 photomultiplier tubes each. These optical modules were developed by the KM3NeT Collaboration to enhance the detection capability of neutrino interactions. The prototype detection unit was operated since its deployment in May 2014 until its decommissioning in July 2015. Reconstruction of the particle trajectories from the data requires a nanosecond accuracy in the time calibration. A procedure for relative time calibration of the photomultiplier tubes contained in each optical module is described. This procedure is based on the measured coincidences produced in the sea by the 40K background light and can easily be expanded to a detector with several thousands of optical modules. The time offsets between the different optical modules are obtained using LED nanobeacons mounted inside them. A set of data corresponding to 600 hours of livetime was analysed. The results show good agreement with Monte Carlo simulations of the expected optical background and the signal from atmospheric muons. An almost background-free sample of muons was selected by filtering the time correlated signals on all the three optical modules. The zenith angle of the selected muons was reconstructed with a precision of about 3{deg}.