The magnetic states of the non-centrosymmetric, pressure induced superconductor CeCoGe3 have been studied with magnetic susceptibility, muon spin relaxation(muSR), single crystal neutron diffraction and inelastic neutron scattering (INS). CeCoGe3 exhibits three magnetic phase transitions at T_N1 = 21 K, T_N2 = 12 K and T_N3 = 8 K. The presence of long range magnetic order below T_N1 is revealed by the observation of oscillations of the asymmetry in the muSR spectra between 13 K and 20 K and a sharp increase in the muon depolarization rate. Single crystal neutron diffraction measurements reveal magnetic Bragg peaks consistent with propagation vectors of k = 2/3 between T_N1 and T_N2, k = 5/8between T_N2 and T_N3 and k = 1/2 below T_N3. An increase in intensity of the (1 1 0) reflection between T_N1 and T_N3 also indicates a ferromagnetic component in these phases. These measurements are consistent with an equal moment, two-up, two-down magnetic structure below T_N3, with a magnetic moment of 0.405(5) mu_B/Ce. Above T_N2, the results are consistent with an equal moment, two-up, one-down structure with a moment of 0.360(6) mu_B/Ce. INS studies reveal two crystal-field (CEF) excitations at 19 and 27 meV. From an analysis with a CEF model, the wave-functions of the J = 5/2 multiplet are evaluated along with a prediction for the magnitude and direction of the ground state magnetic moment. Our model correctly predicts that the moments order along the c axis but the observed magnetic moment of 0.405(5) mu_B is reduced compared to the predicted moment of 1.01 mu_B. This is ascribed to hybridization between the localized Ce^3+ f-electrons and the conduction band. This suggests that CeCoGe3 has a degree of hybridization between that of CeRhGe3 and the non-centrosymmetric superconductor CeRhSi3.