The lifetime of quasars is fundamental for understanding the growth of supermassive black holes, and is an important ingredient in models of the reionization of the intergalactic medium. However, despite various attempts to determine quasar lifetimes, current estimates from a variety of methods are uncertain by orders of magnitude. This work combines cosmological hydrodynamical simulations and 1D radiative transfer to investigate the structure and evolution of the He II Ly$alpha$ proximity zones around quasars at $z simeq 3-4$. We show that the time evolution in the proximity zone can be described by a simple analytical model for the approach of the He II fraction $x_{rm HeII}left( t right)$ to ionization equilibrium, and use this picture to illustrate how the transmission profile depends on the quasar lifetime, quasar UV luminosity, and the ionization state of helium in the ambient IGM (i.e. the average He II fraction, or equivalently the metagalactic He II ionizing background). A significant degeneracy exists between the lifetime and the average He II fraction, however the latter can be determined from measurements of the He II Ly$alpha$ optical depth far from quasars, allowing the lifetime to be measured. We advocate stacking existing He II quasar spectra at $zsim 3$, and show that the shape of this average proximity zone profile is sensitive to lifetimes as long as $sim 30$ Myr. At higher redshift $zsim 4$ where the He II fraction is poorly constrained, degeneracies will make it challenging to determine these parameters independently. Our analytical model for He II proximity zones should also provide a useful description of the properties of H I proximity zones around quasars at $z simeq 6-7$.