We study the zero-temperature phase diagram of the spin-$frac{1}{2}$ Heisenberg model with breathing anisotropy (i.e., with different coupling strength on the upward and downward triangles) on the kagome lattice. Our study relies on large scale tensor network simulations based on infinite projected entangled-pair state and infinite projected entangled-simplex state methods adapted to the kagome lattice. Our energy analysis suggests that the U(1) algebraic quantum spin-liquid (QSL) ground-state of the isotropic Heisenberg model is stable up to very large breathing anisotropy until it breaks down to a critical lattice-nematic phase that breaks rotational symmetry in real space through a first-order quantum phase transition. Our results also provide further insight into the recent experiment on vanadium oxyfluoride compounds which has been shown to be relevant platforms for realizing QSL in the presence of breathing anisotropy.