We reveal a divergent issue associated with the mean-field theory for Bose gases in optical lattices constructed by the widely used straightforward mean-field decoupling of the hopping term, where the corresponding mean-field Hamiltonian generally assumes no lower energy bound once the spatial dependence of the mean-field superfluid order parameter is taken into account. Via a systematic functional integral approach, we solve this issue by establishing a general finite temperature mean-field theory that can treat any possible spatial dependence of the order parameter without causing the divergent issue. Interestingly, we find the theory generally assumes an intrinsic non-hermitian structure that originates from the indefiniteness of the hopping matrix of the system. Within this theory, we develop an efficient approach for investigating the physics of the system at finite temperature, where properties of the system can be calculated via straightforward investigation on the saddle points of an effective potential function for the order parameter. We illustrate our approach by investigating the finite temperature superfluid transition of Bose gases in optical lattices. Since the underlying finite temperature mean-field theory is quite general, this approach can be straightforwardly applied to investigate the finite temperature properties of related systems with phases possessing complex spatial structures.