Phonon modes at microwave frequencies can be cooled to their quantum ground state using conventional cryogenic refrigeration, providing a convenient way to study and manipulate quantum states at the single phonon level. Phonons are of particular interest because mechanical deformations can mediate interactions with a wide range of different quantum systems, including solid-state defects, superconducting qubits, as well as optical photons when using optomechanically-active constructs. Phonons thus hold promise for quantum-focused applications as diverse as sensing, information processing, and communication. Here, we describe a piezoelectric quantum bulk acoustic resonator (QBAR) with a 4.88 GHz resonant frequency that at cryogenic temperatures displays large electromechanical coupling strength combined with a high intrinsic mechanical quality factor $Q_i approx 4.3 times 10^4$. Using a recently-developed flip-chip technique, we couple this QBAR resonator to a superconducting qubit on a separate die and demonstrate quantum control of the mechanics in the coupled system. This approach promises a facile and flexible experimental approach to quantum acoustics and hybrid quantum systems.