This paper presents a new method for the efficient numerical computation of Casimir interactions between objects of arbitrary geometries, composed of materials with arbitrary frequency-dependent electrical properties. Our method formulates the Casimir effect as an interaction between effective electric and magnetic current distributions on the surfaces of material bodies, and obtains Casimir energies, forces, and torques from the spectral properties of a matrix that quantifies the interactions of these surface currents. The method can be formulated and understood in two distinct ways: textbf{(1)} as a consequence of the familiar textit{stress-tensor} approach to Casimir physics, or, alternatively, textbf{(2)} as a particular case of the textit{path-integral} approach to Casimir physics, and we present both formulations in full detail. In addition to providing an algorithm for computing Casimir interactions in geometries that could not be efficiently handled by any other method, the framework proposed here thus achieves an explicit unification of two seemingly disparate approaches to computational Casimir physics.