We report a thermoelectric study of graphene in both zero and applied magnetic fields. As a direct consequence of the linear dispersion of massless particles, we find that the Seebeck coefficient Sxx diverges with 1 /, where n2D is the carrier density. We observe a very large Nernst signal Sxy (~ 50 uV/K at 8 T) at the Dirac point, and an oscillatory dependence of both Sxx and Sxy on n2D at low temperatures. Our results underscore the anomalous thermoelectric transport in graphene, which may be used as a highly sensitive probe for impurity bands near the Dirac point.