Rapid urbanization and climate change trends are intertwined with complex interactions of various social, economic, and political factors. The increased trends of disaster risks have recently caused numerous events, ranging from unprecedented category 5 hurricanes in the Atlantic Ocean to the COVID-19 pandemic. While regions around the world face urgent demands to prepare for, respond to, and to recover from such disasters, large-scale location data collected from mobile phone devices have opened up novel approaches to tackle these challenges. Mobile phone location data have enabled us to observe, estimate, and model human mobility dynamics at an unprecedented spatio-temporal granularity and scale. The COVID-19 pandemic has spurred the use of mobile phone location data for pandemic and disaster response. However, there is a lack of a comprehensive review that synthesizes the last decade of work leveraging mobile phone location data and case studies of natural hazards and epidemics. We address this gap by summarizing the existing work, and pointing promising areas and future challenges for using data to support disaster response and recovery.