We propose a hybrid approach which employs the dynamical mean-field theory (DMFT) self-energy for the correlated, typically rather localized orbitals and a conventional density functional theory (DFT) exchange-correlation potential for the less correlated, less localized orbitals. We implement this self-energy (plus charge density) self-consistent DFT+DMFT scheme in a basis of maximally localized Wannier orbitals using Wien2K, wien2wannier, and the DMFT impurity solver w2dynamics. As a testbed material we apply the method to SrVO$_3$ and report a significant improvement as compared to previous $d$+$p$ calculations. In particular the position of the oxygen $p$ bands is reproduced correctly, which has been a persistent hassle with unwelcome consequences for the $d$-$p$ hybridization and correlation strength. Taking the (linearized) DMFT self-energy also in the Kohn-Sham equation renders the so-called double-counting problem obsolete.