We investigated thermal conductivity of free-standing reduced graphene oxide films subjected to a high-temperature treatment of up to 1000 C. It was found that the high-temperature annealing dramatically increased the in-plane thermal conductivity, K, of the films from 3 W/mK to 61 W/mK at room temperature. The cross-plane thermal conductivity, Kc, revealed an interesting opposite trend of decreasing to a very small value of 0.09 W/mK in the reduced graphene oxide films annealed at 1000 C. The obtained films demonstrated an exceptionally strong anisotropy of the thermal conductivity, K/Kc ~ 675, which is substantially larger even than in the high-quality graphite. The electrical resistivity of the annealed films reduced to 1 - 19 Ohms/sq. The observed modifications of the in-plane and cross-plane thermal conductivity components resulting in an unusual K/Kc anisotropy were explained theoretically. The theoretical analysis suggests that K can reach as high as ~500 W/mK with the increase in the sp2 domain size and further reduction of the oxygen content. The strongly anisotropic heat conduction properties of these films can be useful for applications in thermal management.