Prime numbers are the building blocks of our arithmetic, however, their distribution still poses fundamental questions. Bernhard Riemann showed that the distribution of primes could be given explicitly if one knew the distribution of the non-trivial zeros of the Riemann $zeta(s)$ function. According to the Hilbert-P{o}lya conjecture there exists a Hermitean operator of which the eigenvalues coincide with the real part of the non-trivial zeros of $zeta(s)$. This idea encourages physicists to examine the properties of such possible operators, and they have found interesting connections between the distribution of zeros and the distribution of energy eigenvalues of quantum systems. We apply the Mar{v{c}}henko approach to construct potentials with energy eigenvalues equal to the prime numbers and to the zeros of the $zeta(s)$ function. We demonstrate the multifractal nature of these potentials by measuring the R{e}nyi dimension of their graphs. Our results offer hope for further analytical progress.