Recently, much research has been carried out on Hamiltonians that are not Hermitian but are symmetric under space-time reflection, that is, Hamiltonians that exhibit PT symmetry. Investigations of the Sturm-Liouville eigenvalue problem associated with such Hamiltonians have shown that in many cases the entire energy spectrum is real and positive and that the eigenfunctions form an orthogonal and complete basis. Furthermore, the quantum theories determined by such Hamiltonians have been shown to be consistent in the sense that the probabilities are positive and the dynamical trajectories are unitary. However, the geometrical structures that underlie quantum theories formulated in terms of such Hamiltonians have hitherto not been fully understood. This paper studies in detail the geometric properties of a Hilbert space endowed with a parity structure and analyses the characteristics of a PT-symmetric Hamiltonian and its eigenstates. A canonical relationship between a PT-symmetric operator and a Hermitian operator is established. It is shown that the quadratic form corresponding to the parity operator, in particular, gives rise to a natural partition of the Hilbert space into two halves corresponding to states having positive and negative PT norm. The indefiniteness of the norm can be circumvented by introducing a symmetry operator C that defines a positive definite inner product by means of a CPT conjugation operation.