We construct perturbatively controlled non-Fermi liquids in 3+1 spacetime dimensions, using mild power-law translation breaking interactions. Our mechanism balances the leading tree level effects from such gradients against quantum effects from the interaction between the Fermi surface and a critical boson. We exhibit this in a model where finite density fermions interact with a scalar field via a Yukawa coupling of the form $g(x)propto |x|^kappa$. The approximate non-Fermi liquid behavior arises in the limit of small $kappa$ and persists over an exponentially large window of scales, being cut off by the regime where the coupling becomes large, or by superconducting instabilities. The translation breaking coupling introduces anisotropic deformations of the Fermi surface depending on the direction of the gradient. An extension of this mechanism to 2+1 dimensions could provide a strongly translation-breaking, but weakly coupled non-fermi liquid, something we leave for further work.