When two objects have gravitational interaction between them, they are no longer independent of each other. In fact, there exists gravitational correlation between these two objects. Inspired by E. Verlindes paper, we first calculate the entropy change of a system when gravity does positive work on this system. Based on the concept of gravitational correlation entropy, we prove that the entropy of a Schwarzschild black hole originates from the gravitational correlations between the interior matters of the black hole. By analyzing the gravitational correlation entropies in the process of Hawking radiation in a general context, we prove that the reduced entropy of a black hole is exactly carried away by the radiation and the gravitational correlations between these radiating particles, and the entropy or information is conserved at all times during Hawking radiation. Finally, we attempt to give a unified description of the non-extensive black-hole entropy and the extensive entropy of ordinary matter.