Boulders on the surfaces of planets, satellites and small bodies, as well as their geological associations, provide important information about surface processes. We analyzed all available images of the surface of Mercury that have sufficient resolution and quality to detect boulders, and we mapped all the boulders observed. The lower size limit of detectable boulders was ~5 m. All boulders found on Mercury are associated with fresh impact craters hundreds of meters in diameter or larger. We compared boulder population on Mercury with population of boulders of the same size on the Moon, and found that boulders on Mercury are ~30 times less abundant than in the lunar highlands. This exact quantitative estimate is inherently inaccurate due to the limitation in the source data; however, the significant relative rarity of boulders on Mercury can be firmly and reliably established. We discuss possible causes of the observed difference. Higher thermal stresses and more rapid material fatigue due to diurnal temperature cycling on Mercury may cause rapid disintegration of the upper decimeters of the boulder surface and thus contribute to more rapid boulder obliteration; however, these factors alone cannot account for the observed difference. A proposed thicker regolith on Mercury is likely to significantly reduce boulder production rate. A higher micrometeoritic flux on Mercury is likely to result in micrometeoritic abrasion being a dominant contributor to boulder degradation; this high abrasion rate likely shortens the boulder lifetime. A combination of these factors appears to be able to account for the relative rarity of boulders on Mercury.