We introduce a new family of Poisson vertex algebras $mathcal{W}(mathfrak{a})$ analogous to the classical $mathcal{W}$-algebras. The algebra $mathcal{W}(mathfrak{a})$ is associated with the centralizer $mathfrak{a}$ of an arbitrary nilpotent element in $mathfrak{gl}_N$. We show that $mathcal{W}(mathfrak{a})$ is an algebra of polynomials in infinitely many variables and produce its free generators in an explicit form. This implies that $mathcal{W}(mathfrak{a})$ is isomorphic to the center at the critical level of the affine vertex algebra associated with $mathfrak{a}$.