Cosmography is an ideal tool to investigate the cosmic expansion history of the Universe in a model-independent way. The equations of motion in modified theories of gravity are usually very complicated; cosmography may select practical models without imposing arbitrary choices a priori. We use the model-independent way to derive $f(z)$ and its derivatives up to fourth order in terms of measurable cosmographic parameters. We then fit those functions into the luminosity distance directly. We perform the MCMC analysis by considering three different sets of cosmographic functions. Using the largest supernovae Ia Pantheon sample, we derive the constraints on the Hubble constant $H_0$ and the cosmographic functions, and find that the former two terms in Taylor expansion of luminosity distance work dominantly in $f(Q)$ gravity.