Fossil groups (FGs) have been discovered twenty-five years ago, and are now defined as galaxy groups with an X-ray luminosity higher than $10^{42} h_{50}^{-2}$ erg s$^{-1}$ and a brightest group galaxy brighter than the other group members by at least 2 magnitudes. However, the scenario of their formation remains controversial. We propose here a probabilistic analysis of FGs, extracted from the large catalogue of candidate groups and clusters detected by Sarron et al. (2018) in the CFHTLS survey, based on photometric redshifts, to investigate their position in the cosmic web and probe their environment. Based on spectroscopic and photometric redshifts, we estimate the probability of galaxies to belong to a galaxy structure, and by imposing the condition that the brightest group galaxy is at least brighter than the others by 2 magnitudes, we compute the probability for a given galaxy structure to be a FG. We analyse the mass distribution of these candidate FGs, and estimate their distance to the filaments and nodes of the cosmic web in which they are embedded. We find that the structures with masses lower than $2.4times 10^{14}$ M$_odot$ have the highest probabilities of being fossil groups (PFG). Overall, structures with PFG$geq$50% are located close to the cosmic web filaments (87% are located at less than 1 Mpc from their nearest filament). They are preferentially four times more distant from their nearest node than from their nearest filament. We confirm that FGs have small masses and are rare. They seem to reside closeby cosmic filaments and do not survive in nodes. Being in a poor environment could therefore be the driver of FG formation, the number of nearby galaxies not being sufficient to compensate for the cannibalism of the central group galaxy.