The Phoenix Deep Survey is a multi-wavelength galaxy survey based on deep 1.4 GHz radio imaging (Hopkins et al., 2003). The primary goal of this survey is to investigate the properties of star formation in galaxies and to trace the evolution in those properties to a redshift z=1, covering a significant fraction of the age of the Universe. By compiling a sample of star-forming galaxies based on selection at radio wavelengths we eliminate possible biases due to dust obscuration, a significant issue when selecting objects at optical and ultraviolet wavelengths. In this paper, we present the catalogs and results of deep optical (UBVRI) and near-infrared (Ks) imaging of the deepest region of the existing decimetric radio imaging. The observations and data-processing are summarised and the construction of the optical source catalogs described, together with the details of the identification of candidate optical counterparts to the radio catalogs. Based on our UBVRIKs imaging, photometric redshift estimates for the optical counterparts to the radio detections are explored.