Brownian microparticles diffusing in optical potential energy landscapes constitute a generic testbed for nonequilibrium statistical thermodynamics and has been used to emulate a wide variety of physical systems, ranging from Josephson junctions to Stirling engines. Here we demonstrate that it is possible to scale down this approach to nanometric length-scales by constructing a tilted washboard potential for the rotation of plasmonic gold nanorods. The potential depth and tilt can be precisely adjusted by modulating the light polarization. This allows for a gradual transition from continuous rotation to discrete stochastic jumps, which are found to follow Kramers dynamics in excellent agreement with stochastic simulations. The results widen the possibilities for fundamental experiments in statistical physics and provide new insights in how to construct light-driven nanomachines and multifunctional sensing elements.