Let $G$ be a higher rank semisimple linear algebraic group over a non-Archimedean local field. The simplicial complexes corresponding to any sequence of pairwise non-conjugate irreducible lattices in $G$ are Benjamini-Schramm convergent to the Bruhat-Tits building. Convergence of the relative Plancherel measures and normalized Betti numbers follows. This extends the work of Abert, Bergeron, Biringer, Gelander, Nokolov, Raimbault and Samet from real Lie groups to linear groups over arbitrary local fields. Along the way, various results concerning Invariant Random Subgroups and in particular a variant of the classical Borel density theorem are also extended.