The origin of spontaneous electronic nematic ordering provides important information for understanding iron-based superconductors. Here, we analyze a scenario where the $d_{xy}$ orbital strongly contributes to nematic ordering in FeSe. We show that the addition of $d_{xy}$ nematicity to a pure $d_{xz}/d_{yz}$ order provides a natural explanation for the unusual Fermi surface and correctly reproduces the strongly anisotropic momentum dependence of the superconducting gap. We predict a Lifshitz transition of an electron pocket mediated by temperature and sulphur doping, whose signatures we discuss by analysing available experimental data. We present the variation of momentum dependence of the superconducting gap upon suppression of nematicity. Our quantitatively accurate model yields the transition from tetragonal to nematic FeSe and the FeSe$_{1-x}$S$_{x}$ series, and puts strong constraints on possible nematic mechanisms.