Structural, transport and magnetic properties of single-crystalline samples of a praseodymium-based cubic compound PrMgNi4 were studied. The single-crystal X-ray structural analysis revealed that Mg atoms are substituted for the Pr atoms at the 4a site by 4.5%. The chi(T) data follow the Curie-Weiss law with an effective moment for the Pr3+ ion. The magnetic specific heat divided by temperature, Cm/T, shows a broad maximum at around 3 K, which is reproduced by a two-level model with a ground state doublet. On cooling below 1 K, Cm/T approaches a constant value, which behavior is reproduced by a random two-level model. The twofold degeneracy of the ground state is lifted by symmetry lowering due to the substituted Mg atoms for the Pr atoms or strong hybridizations between the 4f2 electron states and conduction bands, which hinders the long-range quadrupole order.