Finite non-cyclic $p$-groups whose number of subgroups is minimal

Recent results of Qu and Tuarnauceanu describe explicitly the finite p-groups which are not elementary abelian and have the property that the number of their subgroups is maximal among p-groups of a given order. We complement these results from the bottom level up by determining completely the non-cyclic finite p-groups whose number of subgroups among p-groups of a given order is minimal.