We introduce a subalgebra $overline F$ of the Clifford vertex superalgebra ($bc$ system) which is completely reducible as a $L^{Vir} (-2,0)$-module, $C_2$-cofinite, but it is not conformal and it is not isomorphic to the symplectic fermion algebra $mathcal{SF}(1)$. We show that $mathcal{SF}(1)$ and $overline{F}$ are in an interesting duality, since $overline{F}$ can be equipped with the structure of a $mathcal{SF}(1)$-module and vice versa. Using the decomposition of $overline F$ and a free-field realization from arXiv:1711.11342, we decompose $L_k(mathfrak{osp}(1vert 2))$ at the critical level $k=-3/2$ as a module for $L_k(mathfrak{sl}(2))$. The decomposition of $L_k(mathfrak{osp}(1vert 2))$ is exactly the same as of the $N=4$ superconformal vertex algebra with central charge $c=-9$, denoted by $mathcal V^{(2)}$. Using the duality between $overline{F}$ and $mathcal{SF}(1)$, we prove that $L_k(mathfrak{osp}(1vert 2))$ and $mathcal V^{(2)}$ are in the duality of the same type. As an application, we construct and classify all irreducible $L_k(mathfrak{osp}(1vert 2))$-modules in the category $mathcal O$ and the category $mathcal R$ which includes relaxed highest weight modules. We also describe the structure of the parafermion algebra $N_{-3/2}(mathfrak{osp}(1vert 2))$ as a $N_{-3/2}(mathfrak{sl}(2))$-module. We extend this example, and for each $p ge 2$, we introduce a non-conformal vertex algebra $mathcal A^{(p)}_{new}$ and show that $mathcal A^{(p)}_{new} $ is isomorphic to the doublet vertex algebra as a module for the Virasoro algebra. We also construct the vertex algebra $ mathcal V^{(p)} _{new}$ which is isomorphic to the logarithmic vertex algebra $mathcal V^{(p)}$ as a module for $widehat{mathfrak{sl}}(2)$.