We report a comprehensive investigation of the structural, magnetic, transport and thermodynamic properties of a single crystal PrAlSi, in comparison to its nonmagnetic analogue LaAlSi. PrAlSi exhibits a ferromagnetic transition at $T_C$ = 17.8 K which, however, is followed by two weak phase transitions at lower temperatures. Based on the combined dc and ac magnetic susceptibility measurements, we propose the two reentrant magnetic phases below $T_C$ to be spin glasses or ferromagnetic cluster glasses. When the magnetic glassy states are suppressed by small field, several remarkable features appear. These include a linear, nonsaturating magnetoresistance as a function of field that is reminiscent of a topological or charge-compensated semimetal, and a large anomalous Hall conductivity amounting to $sim$2000 $Omega ^{-1}$cm$^{-1}$. Specific-heat measurements indicate a non-Kramers doublet ground state and a relatively low crystal electric field splitting of the Pr$^{3+}$ multiplets of less than 100 K. Shubnikov-de Hass oscillations are absent in LaAlSi, whereas they are clearly observed below about 25 K in PrAlSi, with an unusual temperature dependence of the dominating oscillation frequency $F$. It increases from $F$ = 18 T at 25 K to $F$ = 33 T at 2 K, hinting at an emerging Fermi pocket upon cooling into the ordered phase. These results suggest that PrAlSi is a new system where a small Fermi pocket of likely relativistic fermions is strongly coupled to magnetism. Whether hybridization between $f$ and conduction band is also involved remains an intriguing open problem.