The next generation of axion direct detection experiments may rule out or confirm axions as the dominant source of dark matter. We develop a general likelihood-based framework for studying the time-series data at such experiments, with a focus on the role of dark-matter astrophysics, to search for signatures of the QCD axion or axion like particles. We illustrate how in the event of a detection the likelihood framework may be used to extract measures of the local dark matter phase-space distribution, accounting for effects such as annual modulation and gravitational focusing, which is the perturbation to the dark matter phase-space distribution by the gravitational field of the Sun. Moreover, we show how potential dark matter substructure, such as cold dark matter streams or a thick dark disk, could impact the signal. For example, we find that when the bulk dark matter halo is detected at 5$sigma$ global significance, the unique time-dependent features imprinted by the dark matter component of the Sagittarius stream, even if only a few percent of the local dark matter density, may be detectable at $sim$2$sigma$ significance. A co-rotating dark disk, with lag speed $sim$50 km$/$s, that is $sim$20$%$ of the local DM density could dominate the signal, while colder but as-of-yet unknown substructure may be even more important. Our likelihood formalism, and the results derived with it, are generally applicable to any time-series based approach to axion direct detection.