The main aim of this article is to study the relation between $F$-injective singularity and the Frobenius closure of parameter ideals in Noetherian rings of positive characteristic. The paper consists of the following themes, including many other topics. We prove that if every parameter ideal of a Noetherian local ring of prime characteristic $p>0$ is Frobenius closed, then it is $F$-injective. We prove a necessary and sufficient condition for the injectivity of the Frobenius action on $H^i_{fm}(R)$ for all $i le f_{fm}(R)$, where $f_{fm}(R)$ is the finiteness dimension of $R$. As applications, we prove the following results. (a) If the ring is $F$-injective, then every ideal generated by a filter regular sequence, whose length is equal to the finiteness dimension of the ring, is Frobenius closed. It is a generalization of a recent result of Ma and which is stated for generalized Cohen-Macaulay local rings. (b) Let $(R,fm,k)$ be a generalized Cohen-Macaulay ring of characteristic $p>0$. If the Frobenius action is injective on the local cohomology $H_{fm}^i(R)$ for all $i < dim R$, then $R$ is Buchsbaum. This gives an answer to a question of Takagi. We consider the problem when the union of two $F$-injective closed subschemes of a Noetherian $mathbb{F}_p$-scheme is $F$-injective. Using this idea, we construct an $F$-injective local ring $R$ such that $R$ has a parameter ideal that is not Frobenius closed. This result adds a new member to the family of $F$-singularities. We give the first ideal-theoretic characterization of $F$-injectivity in terms the Frobenius closure and the limit closure. We also give an answer to the question about when the Frobenius action on the top local cohomology is injective.