We study the properties of Lyman-alpha emitters (LAEs) and Lyman-break galaxies (LBGs) at z=3-6 using cosmological SPH simulations. We investigate two simple scenarios for explaining the observed Ly-a and rest-frame UV luminosity functions (LFs) of LAEs: (i) the escape fraction scenario, in which the effective escape fraction (including the IGM attenuation) of Ly-a photons is f_Lya ~0.1 (0.15) at z=3 (6), and (ii) the stochastic scenario, in which the fraction of LAEs that are turned on at z=3 (6) is Cstoc ~0.07 (0.2) after correcting for the IGM attenuation. Our comparisons with a number of different observations suggest that the stochastic scenario is preferred over the escape fraction scenario. We find that the mean values of stellar mass, metallicity and black hole mass hosted by LAEs are all smaller in the stochastic scenario than in the escape fraction scenario. In our simulations, the galaxy stellar mass function evolves rapidly, as expected in hierarchical structure formation. However, its evolution is largely compensated by a beginning decline in the specific star formation rate, resulting in little evolution of the rest-frame UV LF from z=6 to 3. The rest-frame UV LF of both LAEs and LBGs at z=3 & 6 can be described well by the stochastic scenario provided the extinction is moderate, E(B-V) ~0.15, for both populations, although our simulation might be overpredicting the number of bright LBGs at z=6. We also discuss the correlation function and bias of LAEs. The Ly-a LFs at z=6 in a field-of-view of 0.2 deg^2 show a significantly larger scatter owing to cosmic variance relative to that in a 1 deg^2 field, and the scatter seen in the current observational estimates of the Ly-a LF can be accounted for by cosmic variance.