In this work we experimentally demonstrate a photon-pair source with correlations in the frequency and polarization degrees of freedom. We base our source on the spontaneous four-wave mixing (SFWM) process in a photonic crystal fiber. We show theoretically that the two-photon state is the coherent superposition of up to six distinct SFWM processes, each corresponding to a distinct combination of polarizations for the four waves involved and giving rise to an energy-conserving pair of peaks. Our experimental measurements, both in terms of single and coincidence counts, confirm the presence of these pairs of peaks, while we also present related numerical simulations with excellent experiment-theory agreement. We explicitly show how the pump frequency and polarization may be used to effectively control the signal-idler photon-pair properties, defining which of the six processes can participate in the overall two-photon state and at which optical frequencies. We analyze the signal-idler correlations in frequency and polarization, and in terms of fiber characterization, we input the SFWM-peak experimental data into a genetic algorithm which successfully predicts the values of the parameters that characterize the fiber cross section, as well as predict the particular SFWM process associated with a given pair of peaks. We believe our work will help advance the exploitation of photon-pair correlations in the frequency and polarization degrees of freedom.