We use photometric redshifts derived from new $u$-band through 4.5$mu$m Spitzer IRAC photometry in the 4.8,deg$^2$ of the XMM-LSS field to construct surface density maps in the redshift range 0.1-1.5. Our density maps show evidence for large-scale structure in the form of filaments spanning several tens of Mpc. Using these maps, we identify 339 overdensities that our simulated lightcone analysis suggests are likely associated with dark matter haloes with masses, $M_{rm halo}$, log($M_{rm halo}/M_{odot})>$13.7. From this list of overdensities we recover 43 of 70 known X-ray detected and spectroscopically confirmed clusters. The missing X-ray clusters are largely at lower redshifts and lower masses than our target log($M_{rm halo}/M_{odot})>$13.7. The bulk of the overdensities are compact, but a quarter show extended morphologies which include likely projection effects, clusters embedded in apparent filaments as well as at least one potential cluster merger (at $zsim1.28$). The strongest overdensity in our highest redshift slice (at $zsim1.5$) shows a compact red galaxy core potentially implying a massive evolved cluster.