The possibility that an unconventional depletion in the center of the charge density distribution of certain nuclei occurs due to a purely quantum mechanical effect has attracted theoretical and experimental attention in recent years. We report on ab initio self-consistent Greens function calculations of one of such candidates, $^{34}$Si, together with its Z+2 neighbour $^{36}$S. Binding energies, rms radii and density distributions of the two nuclei as well as low-lying spectroscopy of $^{35}$Si, $^{37}$S, $^{33}$Al and $^{35}$P are discussed. The interpretation of one-nucleon removal and addition spectra in terms of the evolution of the underlying shell structure is also provided. The study is repeated using several chiral effective field theory Hamiltonians as a way to test the robustness of the results with respect to input inter-nucleon interactions. The prediction regarding the (non-)existence of the bubble structure in $^{34}$Si varies significantly with the nuclear Hamiltonian used. However, demanding that the experimental charge density distribution and the root mean square radius of $^{36}$S are well reproduced, along with $^{34}$Si and $^{36}$S binding energies, only leaves the NNLO$_{text{sat}}$ Hamiltonian as a serious candidate to perform this prediction. In this context, a bubble structure, whose fingerprint should be visible in an electron scattering experiment of $^{34}$Si, is predicted. Furthermore, a clear correlation is established between the occurrence of the bubble structure and the weakening of the 1/2$^-$-3/2$^-$ splitting in the spectrum of $^{35}$Si as compared to $^{37}$S.