We employ electrostatic force microscopy to study the electrostatic environment of graphene sheets prepared with the micro-mechanical exfoliation technique. We detect the electric dipole of residues left from the adhesive tape during graphene preparation, as well as the dipole of water molecules adsorbed on top of graphene. Water molecules form a dipole layer that can generate an electric field as large as 10^9 V/m. We expect that water molecules can significantly modify the electrical properties of graphene devices.