Parameterized quantum circuits (PQCs), which are essential for variational quantum algorithms, have conventionally been optimized by parameterized rotational angles of single-qubit gates around predetermined set of axes. We propose a new method to optimize a PQC by continuous parameterization of both the angles and the axes of its single-qubit rotations. The method is based on the observation that when rotational angles are fixed, optimal axes of rotations can be computed by solving a system of linear equations whose coefficients can be determined from the PQC with small computational overhead. The method can be further simplified to select axes freely from continuous parameters with rotational angles fixed to $pi$. We show the simplified free-axis selection method has better expressibility against other structural optimization methods when measured with Kullback-Leibler (KL) divergence. We also demonstrate PQCs with free-axis selection are more effective to search the ground states of Heisenberg models and molecular Hamiltonians. Because free-axis selection allows designing PQCs without specifying their single-qubit rotational axes, it may significantly improve the handiness of PQCs.