We studied structural, electronic and magnetic properties of a cubic perovskite BaFeO$_{3-delta}$ ($0 le delta le 0.5$) within the density functional theory using a generalized gradient approximation and a GGA+U method. According to our calculations, BaFeO$_3$ in its stoichiometric cubic structure should be half-metallic and strongly ferromagnetic, with extremely high Curie temperature ($T_C$) of 700 - 900 K. However, a such estimate of $T_C$ disagrees with all available experiments, which report that $T_C$ of the BaFeO$_3$ and undoped BaFeO$_{3-delta}$ films varies between 111 K and 235 K or, alternatively, that no ferromagnetic order was detected there. Fitting the calculated x-ray magnetic circular dichroism spectra to the experimental features seen for BaFeO$_3$, we concluded that the presence of oxygen vacancies in our model enables a good agreement. Thus, the relatively low $T_C$ measured in BaFeO$_3$ can be explained by oxygen vacancies intrinsically presented in the material. Since iron species near the O vacancy change their oxidation state from $4+$ to $3+$, the interaction between Fe$^{4+}$ and Fe$^{3+}$, which is antiferromagnetic, weakens the effective magnetic interaction in the system, which is predominantly ferromagnetic. With increasing $delta$ in BaFeO$_{3-delta}$, its $T_C$ decreases down to the critical value when the magnetic order becomes antiferromagnetic. Our calculations of the electronic structure of BaFeO$_{3-delta}$ illustrate how the ferromagnetism originates and also how one can keep this cubic perovskite robustly ferromagnetic far above the room temperature.