In this paper we examine the electronic and geometrical structure of impurity-vacancy complexes in Si and Ge. Already Watkins suggested that in Si the pairing of Sn with the vacancy produces a complex with the Sn-atom at the bond center and the vacancy split into two half vacancies on the neighboring sites. Within the framework of density-functional theory we use two complementary ab initio methods, the pseudopotential plane wave (PPW) method and the all-electron Kohn-Korringa-Rostoker (KKR) method, to investigate the structure of vacancy complexes with 11 different sp-impurities. For the case of Sn in Si, we confirm the split configuration and obtain good agreement with EPR data of Watkins. In general we find that all impurities of the 5sp and 6sp series in Si and Ge prefer the split-vacancy configuration, with an energy gain of 0.5 to 1 eV compared to the substitutional complex. On the other hand, impurities of the 3sp and 4sp series form a (slightly distorted) substitutional complex. Al impurities show an exception from this rule, forming a split complex in Si and a strongly distorted substitutional complex in Ge. We find a strong correlation of these data with the size of the isolated impurities, being defined via the lattice relaxations of the nearest neighbors.