We investigate a solid-state, reversible, alkali-ion battery (AIB) capable of regulating the density of alkali atoms in a vacuum system used for the production of laser-cooled atoms. The cold-atom sample can be used with in-vacuum chronoamperometry as a diagnostic for the voltage-controlled electrochemical reaction that sources or sinks alkali atoms into the vapor. In a combined reaction-diffusion-limited regime, we show that the number of laser-cooled atoms in a magneto-optical trap can be increased both by initially loading the AIB from the vapor for longer, and by using higher voltages across the AIB when atoms are subsequently sourced back into the vapor. The time constants associated with the change in atom number in response to a change in AIB voltage are in the range of 0.5 s - 40 s. The AIB alkali reservoir is demonstrated to survive oxidization during atmospheric exposure, simplifying reservoir loading prior to vacuum implementation as a replacement for traditional resistively-heated dispensers. The AIB capabilities may provide an improved atom number stability in next-generation atomic clocks and sensors, while also facilitating fast loading and increased interrogation times.