Over the last decade, the evidence is mounting that several aspects of black hole accretion physics proceed in a mass-invariant way. One of the best examples of this scaling is the empirical Fundamental Plane of Black Hole Accretion relation linking mass, radio and X-ray luminosity over eight orders of magnitude in black hole mass. The currently favored theoretical interpretation of this relation is that the physics governing power output in weakly accreting black holes depends more on relative accretion rate than on mass. In order to test this theory, we explore whether a mass-invariant approach can simultaneously explain the broadband spectral energy distributions from two black holes at opposite ends of the mass scale but at similar Eddington accretion fractions. We find that the same model, with the same value of several fitted physical parameters expressed in mass-scaling units to enforce self-similarity, can provide a good description of two datasets from V404 Cyg and M81*, a stellar and supermassive black hole, respectively. Furthermore, only one of several potential emission scenarios for the X-ray band is successful, suggesting it is the dominant process driving the Fundamental Plane relation at this accretion rate. This approach thus holds promise for breaking current degeneracies in the interpretation of black hole high-energy spectra, and for constructing better prescriptions of black hole accretion for use in various local and cosmological feedback applications.