Quantized bound states at a vortex core are discretized in YNi$_2$B$_2$C. By using scanning tunneling spectroscopy with an unprecedented 0.1 nm spatial resolution, we find and identify the localized spectral structure, where in addition to the first main peak with a positive low energy, a second subpeak coming from the fourfold symmetric gap structure is seen inside the energy gap. Those spectral features are understood by solving the Bogoliubov-de Gennes equation for a fully three-dimensional gap structure. A particle-hole asymmetric spectrum at the core site and quantum oscillation in the spectra are clearly observed.