We introduce a new family of affine $W$-algebras associated with the centralizers of arbitrary nilpotent elements in $mathfrak{gl}_N$. We define them by using a version of the BRST complex of the quantum Drinfeld--Sokolov reduction. A family of free generators of the new algebras is produced in an explicit form. We also give an analogue of the Fateev--Lukyanov realization for these algebras by applying a Miura-type map.