We have investigated the microstructure, normal-state electrical connectivity, and critical current density of ex-situ MgB2 polycrystalline bulks prepared by systematically varying the sintering conditions under low pressure. Samples heated at a high temperature of ~900{deg}C for a long period showed an increased packing factor, a larger intergrain contact area, and a significantly enhanced electrical connectivity, all of which indicate solid-state self-sintering of MgB2. Sintered ex-situ MgB2 bulks from a laboratory-made ball-milled powder exhibited a greatly enhanced connectivity of 28%, which is the highest connectivity of pressureless ex-situ MgB2 bulks, wires, and tapes. Surprisingly, grain growth did not occur during long-duration (~100 h) sintering in the sintered ex-situ MgB2 bulks. This is in marked contrast to in-situ processed MgB2 samples for which significant grain growth occurred during heat treatment at ~900{deg}C, producing grains that are several tens of times larger than the initial boron grains. Consequently, the critical current density as a function of the external magnetic field at 20 K progressively improved with sintering due to the relatively small grain size and good intergrain connectivity. We thus conclude that solid-state self-sintering is an effective approach for producing strongly connected, dense ex-situ MgB2 polycrystals without grain growth.