Any population of asteroids, like asteroid families, will disperse in semi-major axis due to the Yarkovsky effect. The amount of drift is modulated by the asteroid spin state evolution which determines the balance between the diurnal and seasonal Yarkovsky force. The asteroids spin state is, in turn, controlled in part by the YORP effect. The otherwise smooth evolution of an asteroid can be abruptly altered by collisions, which can cause impulsive changes in the spin state and can move the asteroid onto a different YORP track. In addition, collisions may also alter the YORP parameters by changing the superficial features and overall shape of the asteroid. Thus, the coupling between YORP and Yarkovsky is also strongly affected by the impact history of each body. To investigate this coupling we developed a statistical code modeling the time evolution of semi--major axis under YORP-Yarkovsky coupling. It includes the contributions of NYORP (normal YORP), TYORP (tangential YORP) and collisions whose effects are deterministically calculated and not added in a statistical way. We find that both collisions and TYORP increase the dispersion of a family in semi-major axis by making the spin axis evolution less smooth and regular. We show that the evolution of a familys structure with time is complex and collisions randomize the YORP evolution. In our test families we do not observe the formation of a YORP-eye in the semi-major axis vs. diameter distribution, even after a long period of time. If present, the YORP-eye might be a relic of an initial ejection velocity pattern of the collisional fragments.