Two ordering states, antiferromagnetism and nematicity, have been observed in most iron-based superconductors (SCs). In contrast to those SCs, the newly discovered SC CaK(Fe$_{1-x}$Ni$_x$)$_4$As$_4$ exhibits an antiferromagnetic (AFM) state, called hedgehog spin-vortex crystal structure, without nematic order, providing the opportunity for the investigation into the relationship between spin fluctuations and SC without any effects of nematic fluctuations. Our $^{75}$As nuclear magnetic resonance studies on CaK(Fe$_{1-x}$Ni$_x$)$_4$As$_4$ (0$le xle$ 0.049) revealed that CaKFe$_4$As$_4$ is located close to a hidden hedgehog SVC AFM quantum-critical point (QCP). The magnetic QCP without nematicity in CaK(Fe$_{1-x}$Ni$_x$)$_4$As$_4$ highlights the close connection of spin fluctuations and superconductivity in iron-based SCs. The advantage of stoichiometric composition also makes CaKFe$_4$As$_4$ an ideal platform for further detailed investigation of the relationship between magnetic QCP and superconductivity in iron-based SCs without disorder effects.