We present a joint weak lensing and X-ray analysis of 4 deg$^2$ from the CFHTLS and XMM-LSS surveys. Our weak lensing analysis is the first analysis of a real survey using shapelets, a new generation weak lensing analysis method. We create projected
mass maps of the images, and extract 6 weak-lensing-detected clusters of galaxies. We show that their counts can be used to constrain the power spectrum normalisation $sigma_8 =0.92_{-0.30}^{+0.26}$ for $Omega_m=0.24$. We show that despite the large scatter generally observed in the M-T relation derived from lensing masses, tight constraints on both its slope and normalisation $M_*$ can be obtained with a moderate number of sources provided that the covered mass range is large enough. Adding clusters from Bardeau et al. (2007) to our sample, we measure $M_* = 2.71_{-0.61}^{+0.79} 10^{14} h^{-1} M_odot$. Although they are dominated by shot noise and sample variance, our measurements are consistent with currently favoured values, and set the stage for future surveys. We thus investigate the dependence of those estimates on survey size, depth, and integration time, for joint weak lensing and X-ray surveys. We show that deep surveys should be dedicated to the study of the physics of clusters and groups of galaxies. For a given exposure time, wide surveys provide a larger number of detected clusters and are therefore preferred for the measurement of cosmological parameters such as $sigma_8$ and $M_*$. We show that a wide survey of a few hundred square degrees is needed to improve upon current measurements of these parameters. More ambitious surveys covering 7000 deg$^2$ will provide the 1% accuracy in the estimation of the power spectrum and the M-T relation normalisations.
