The low temperature lattice structure and magnetic properties of Co$_{2.75}$Fe$_{0.25}$O$_4$ ferrite have been investigated using experimental results from synchrotron x-ray diffraction (SXRD), dc magnetization, ac susceptibility, neutron diffraction and neutron depolarization techniques. The samples have been prepared by chemical co-precipitation of the Fe and Co nitrates solution in high alkaline medium and subsequent thermal annealing of the precipitates in the temperature range of 200- 900 $^circ$C. Rietveld refinement of the SXRD patterns at room temperature indicated two-phased cubic spinel structure for the samples annealed at temperatures 200-600 $^circ$C. The samples annealed at temperatures 700 $^circ$C and 900 $^circ$C (CF90) have been best fitted with single phased lattice structure. Refinement of the neutron diffraction patterns in the temperature range of 5-300 K confirmed antiferromagnetic (AFM) Co$_3$O$_4$ and ferrimagnetic (FIM) Co$_{2.75}$Fe$_{0.25}$O$_4$ phases for the sample annealed at 600 $^circ$C and single FIM phase of Co$_{2.75}$Fe$_{0.25}$O$_4$ for the CF90 sample. Magnetic measurements have shown a non-equilibrium magnetic structure, consisting of the high temperature FIM phase and low temperature AFM phase. The magnetic phases are sensitive to magnetic fields, where high temperature phase is suppressed at higher magnetic fields by enhancing the low temperature AFM phase, irrespective of annealing temperature of the samples.