Neutron diffraction with static and pulsed magnetic fields is used to directly probe the magnetic structures in LiNiPO$_4$ up to 25T and 42T, respectively. By combining these results with magnetometry and electric polarization measurements under pulsed fields, the magnetic and magnetoelectric phases are investigated up to 56T applied along the easy $c$-axis. In addition to the already known transitions at lower fields, three new ones are reported at 37.6, 39.4 and 54T. Ordering vectors are identified with ${bf Q}_{mathrm{VI}}$ = (0, 1/3, 0) in the interval 37.6--39.4T and ${bf Q}_{mathrm{VII}}$ = (0, 0, 0) in the interval 39.4-54T. A quadratic magnetoelectric effect is discovered in the ${bf Q}_{mathrm{VII}}$ = (0, 0, 0) phase and the field-dependence of the induced electric polarization is described using a simple mean-field model. The observed magnetic structure and magnetoelectric tensor elements point to a change in the lattice symmetry in this phase. We speculate on the possible physical mechanism responsible for the magnetoelectric effect in LiNiPO4.