The main theme of this review is the many-body physics of vortices in quantum droplets of bosons or fermions, in the limit of small particle numbers. Systems of interest include cold atoms in traps as well as electrons confined in quantum dots. When set to rotate, these in principle very different quantum systems show remarkable analogies. The topics reviewed include the structure of the finite rotating many-body state, universality of vortex formation and localization of vortices in both bosonic and fermionic systems, and the emergence of particle-vortex composites in the quantum Hall regime. An overview of the computational many-body techniques sets focus on the configuration interaction and density-functional methods. Studies of quantum droplets with one or several particle components, where vortices as well as coreless vortices may occur, are reviewed, and theoretical as well as experimental challenges are discussed.