We propose the definitions of many-body topological invariants to detect symmetry-protected topological phases protected by point group symmetry, using partial point group transformations on a given short-range entangled quantum ground state. Partial point group transformations $g_D$ are defined by point group transformations restricted to a spatial subregion $D$, which is closed under the point group transformations and sufficiently larger than the bulk correlation length $xi$. By analytical and numerical calculations,we find that the ground state expectation value of the partial point group transformations behaves generically as $langle GS | g_D | GS rangle sim exp Big[ i theta+ gamma - alpha frac{{rm Area}(partial D)}{xi^{d-1}} Big]$. Here, ${rm Area}(partial D)$ is the area of the boundary of the subregion $D$, and $alpha$ is a dimensionless constant. The complex phase of the expectation value $theta$ is quantized and serves as the topological invariant, and $gamma$ is a scale-independent topological contribution to the amplitude. The examples we consider include the $mathbb{Z}_8$ and $mathbb{Z}_{16}$ invariants of topological superconductors protected by inversion symmetry in $(1+1)$ and $(3+1)$ dimensions, respectively, and the lens space topological invariants in $(2+1)$-dimensional fermionic topological phases. Connections to topological quantum field theories and cobordism classification of symmetry-protected topological phases are discussed.