A Kakeya set $S subset (mathbb{Z}/Nmathbb{Z})^n$ is a set containing a line in each direction. We show that, when $N$ is any square-free integer, the size of the smallest Kakeya set in $(mathbb{Z}/Nmathbb{Z})^n$ is at least $C_{n,epsilon} N^{n - epsilon}$ for any $epsilon$ -- resolving a special case of a conjecture of Hickman and Wright. Previously, such bounds were only known for the case of prime $N$. We also show that the case of general $N$ can be reduced to lower bounding the $mathbb{F}_p$ rank of the incidence matrix of points and hyperplanes over $(mathbb{Z}/p^kmathbb{Z})^n$.