In this article we extend the celebrated Berry-phase formulation of electric polarization in crystals to higher electric multipole moments. We determine the necessary conditions under which, and minimal models in which, the quadrupole and octupole moments are topologically quantized electromagnetic observables. Such systems exhibit gapped boundaries that are themselves lower-dimensional topological phases. Furthermore, they manifest topologically protected corner states carrying fractional charge, i.e., fractionalization at the boundary of the boundary. To characterize these new insulating phases of matter, we introduce a new paradigm whereby `nested Wilson loops give rise to a large number of new topological invariants that have been previously overlooked. We propose three realistic experimental implementations of this new topological behavior that can be immediately tested.