Time-dependent $mathcal{PT}$-symmetric quantum mechanics is featured by a varying inner-product metric and has stimulated a number of interesting studies beyond conventional quantum mechanics. In this paper, we explore geometric aspects of time-dependent $mathcal{PT}$-symmetric quantum mechanics. We not only find a geometric phase factor emerging naturally from cyclic evolutions of $mathcal{PT}$-symmetric systems, but also formulate a series of differential geometry concepts, including connection, curvature, parallel transport, metric tensor, and quantum geometric tensor. Our findings constitute a useful, perhaps indispensible, tool to tackle physical problems involving $mathcal{PT}$-symmetric systems with time-varying systems parameters. To exemplify the application of our findings, we show that the unconventional geometrical phase [Phys. Rev. Lett. 91, 187902 (2003)], consisting of a geometric phase and a dynamical phase proportional to the geometric phase, can be expressed as a single geometric phase identified in this work.