The specific globular cluster frequencies (S_N) for 24 dwarf elliptical (dE) galaxies in the Virgo and Fornax Clusters and the Leo Group imaged with the Hubble Space Telescope are presented. Combining all available data, we find that for nucleated dEs --- which are spatially distributed like giant ellipticals in galaxy clusters --- S_N(dE,N)=6.5 +- 1.2 and S_N increases with M_V, while for non-nucleated dEs --- which are distributed like late-type galaxies --- S_N(dE,noN)=3.1 +- 0.5 and there is little or no trend with M_V. The S_N values for dE galaxies are thus on average significantly higher than those for late-type galaxies, which have S_N < 1. This suggests that dE galaxies are more akin to giant Es than to late-type galaxies. If there are dormant or stripped irregulars hiding among the dE population, they are likely to be among the non-nucleated dEs. Furthermore, the similarities in the properties of the globular clusters and in the spatial distributions of dE,Ns and giant Es suggest that neither galaxy mass or galaxy metallicity is responsible for high values of S_N. Instead, most metal-poor GCs may have formed in dwarf-sized fragments that merged into larger galaxies.