The electronic properties of cuprate high temperature superconductors in their normal state are very two-dimensional: while transport in the ab plane is perfectly metallic, it is insulating along the c-axis, with ratios between the two exceeding 10^4. This anisotropy has been identified as one of the mysteries of the cuprates early on, and while widely different proposals exist for its microscopic origin, little is known empirically on the microscopic scale. Here, we elucidate the properties of the insulating layers with a newly developed scanning noise spectroscopy technique that can spatially map not only the current but also the current fluctuations in time. We discover atomic-scale noise centers that exhibit MHz current fluctuations 40 times the expectation from Poissonian noise, more than what has been observed in mesoscopic systems. Such behaviour can only happen in highly polarizable insulators and represents strong evidence for trapping of charge in the charge reservoir layers. Our measurements suggest a picture of metallic layers separated by polarizable insulators within a three-dimensional superconducting state.