As one of the most promising applications in future Internet of Things, Internet of Vehicles (IoV) has been acknowledged as a fundamental technology for developing the Intelligent Transportation Systems in smart cities. With the emergence of the sixth generation (6G) communications technologies, massive network infrastructures will be densely deployed and the number of network nodes will increase exponentially, leading to extremely high energy consumption. There has been an upsurge of interest to develop the green IoV towards sustainable vehicular communication and networking in the 6G era. In this paper, we present the main considerations for green IoV from five different scenarios, including the communication, computation, traffic, Electric Vehicles (EVs), and energy harvesting management. The literatures relevant to each of the scenarios are compared from the perspective of energy optimization (e.g., with respect to resource allocation, workload scheduling, routing design, traffic control, charging management, energy harvesting and sharing, etc.) and the related factors affecting energy efficiency (e.g., resource limitation, channel state, network topology, traffic condition, etc.). In addition, we introduce the potential challenges and the emerging technologies in 6G for developing green IoV systems. Finally, we discuss the research trends in designing energy-efficient IoV systems.