General relativity can be formulated equivalently with a non-Riemannian geometry that associates with an affine connection of nonzero nonmetricity $Q$ but vanishing curvature $R$ and torsion $T$. Modification based on this description of gravity generates the $f(Q)$ gravity. In this work we explore the application of $f(Q)$ gravity to the spherically symmetric configurations. We discuss the gauge fixing and connections in this setting. We demonstrate the effects of $f(Q)$ by considering the external and internal solutions of compact stars. The external background solutions for any regular form of $f(Q)$ coincide with the corresponding solutions in general relativity, i.e., the Schwarzschild-de Sitter solution and the Reissner-Nordstrom-de Sitter solution with an electromagnetic field. For internal structure, with a simple model $f(Q)=Q+alpha Q^2$ and a polytropic equation of state, we find that a negative modification ($alpha<0$) provides support to more stellar masses while a positive one ($alpha>0$) reduces the amount of matter of the star.