The induced surface charges appear to diverge when dielectric particles form close contacts. Resolving this singularity numerically is prohibitively expensive because high spatial resolution is needed. We show that the strength of this singularity is logarithmic in both inter-particle separation and dielectric permittivity. A regularization scheme is proposed to isolate this singularity, and to calculate the exact cohesive energy for clusters of contacting dielectric particles. The results indicate that polarization energy stabilizes clusters of open configurations when permittivity is high, in agreement with the behavior of conducting particles, but stabilizes the compact configurations when permittivity is low.