There is an approximately 9% discrepancy, corresponding to 2.4sigma, between two independent constraints on the expansion rate of the universe: one indirectly arising from the cosmic microwave background and baryon acoustic oscillations, and one more directly obtained from local measurements of the relation between redshifts and distances to sources. We argue that by taking into account the local gravitational potential at the position of the observer this tension - strengthened by the recent Planck results - is partially relieved and the concordance of the standard model of cosmology increased. We estimate that measurements of the local Hubble constant are subject to a cosmic variance of about 2.4% (limiting the local sample to redshifts z>0.010) or 1.3% (limiting it to z>0.023), a more significant correction than that taken into account already. Nonetheless, we show that one would need a very rare fluctuation to fully explain the offset in the Hubble rates. If this tension is further strengthened, a cosmology beyond the standard model may prove necessary.