We propose a novel method to test the consistency of the multipole moments of compact binary systems with the predictions of General Relativity (GR). The multipole moments of a compact binary system, known in terms of symmetric and trace-free tensors, are used to calculate the gravitational waveforms from compact binaries within the post-Newtonian (PN) formalism. For nonspinning compact binaries, we derive the gravitational wave phasing formula, in the frequency domain, parametrizing each PN order term in terms of the multipole moments which contribute to that order. Using GW observations, this {it{parametrized multipolar phasing}} would allow us to derive the bounds on possible departures from the multipole structure of GR and hence constrain the parameter space of alternative theories of gravity. We compute the projected accuracies with which the second generation ground-based detectors, such as Advanced Laser Interferometer Gravitational-wave Observatory (LIGO), the third generation detectors such as Einstein Telescope and Cosmic Explorer, as well as space-based detector Laser Interferometer Space Antenna (LISA) will be able to measure these multipole parameters. We find that while Advanced LIGO can measure the first two or three multipole coefficients with good accuracy, Cosmic Explorer and Einstein Telescope may be able to measure the first four multipole coefficients which enter the phasing formula. Intermediate mass ratio inspirals, with mass ratio of several tens, in the frequency band of planned space-based LISA mission should be able to measure all the seven multipole coefficients which appear in the 3.5PN phasing formula. Our finding highlights the importance of this class of sources for probing the strong-field gravity regime. The proposed test will facilitate the first probe of the multipolar structure of Einsteins general relativity.