In ultra- and deep-strong cavity quantum electrodynamics (QED) systems, many intriguing phenomena that do not conserve the excitation number are expected to occur. In this study, we theoretically analyze the optical response of an ultrastrong cavity-QED system in which an atom is coupled to the fundamental and third harmonic modes of a cavity, and report the possibility of deterministic three-photon down-conversion of itinerant photons upon reflection at the cavity. In the conventional parametric down-conversion, a strong input field is needed because of the smallness of the transition matrix elements of the higher order processes. However, if we use an atom-cavity system in an unprecedentedly strong-coupling region, even a weak field in the linear-response regime is sufficient to cause this rare event involving the fourth order transitions.