The low-lying structure of $^{55}$Sc has been investigated using in-beam $gamma$-ray spectroscopy with the $^{9}$Be($^{56}$Ti,$^{55}$Sc+$gamma$)$X$ one-proton removal and $^{9}$Be($^{55}$Sc,$^{55}$Sc+$gamma$)$X$ inelastic-scattering reactions at the RIKEN Radioactive Isotope Beam Factory. Transitions with energies of 572(4), 695(5), 1539(10), 1730(20), 1854(27), 2091(19), 2452(26), and 3241(39) keV are reported, and a level scheme has been constructed using $gammagamma$ coincidence relationships and $gamma$-ray relative intensities. The results are compared to large-scale shell-model calculations in the $sd$-$pf$ model space, which account for positive-parity states from proton-hole cross-shell excitations, and to it ab initio shell-model calculations from the in-medium similarity renormalization group that includes three-nucleon forces explicitly. The results of proton-removal reaction theory with the eikonal model approach were adopted to aid identification of positive-parity states in the level scheme; experimental counterparts of theoretical $1/2^{+}_{1}$ and $3/2^{+}_{1}$ states are suggested from measured decay patterns. The energy of the first $3/2^{-}$ state, which is sensitive to the neutron shell gap at the Fermi surface, was determined. The result indicates a rapid weakening of the $N=34$ subshell closure in $pf$-shell nuclei at $Z>20$, even when only a single proton occupies the $pi f_{7/2}$ orbital.