We have performed high-resolution angle-resolved photoemission spectroscopy of ternary pnictide CaAuAs which is predicted to be a three-dimensional topological Dirac semimetal (TDS). By accurately determining the bulk-band structure, we have revealed the coexistence of three-dimensional and quasi-two-dimensional Fermi surfaces with dominant hole carriers. The band structure around the Brillouin-zone center is characterized by an energy overlap between hole and electron pockets, in excellent agreement with first-principles band-structure calculations. This indicates the occurrence of bulk-band inversion, supporting the TDS state in CaAuAs. Because of the high tunability in the chemical composition besides the TDS nature, CaAuAs provides a precious opportunity for investigating the quantum phase transition from TDS to other exotic topological phases.