This paper considers mean field games in a multi-agent Markov decision process (MDP) framework. Each player has a continuum state and binary action, and benefits from the improvement of the condition of the overall population. Based on an infinite horizon discounted individual cost, we show existence of a stationary equilibrium, and prove its uniqueness under a positive externality condition. We further analyze comparative statics of the stationary equilibrium by quantitatively determining the impact of the effort cost.