Guided by the recent discovery of SU($2$)$_1$ and SU($3$)$_1$ chiral spin liquids on the square lattice, we propose a family of generic time-reversal symmetry breaking SU($N$)-symmetric models, of arbitrary $Nge 2$, in the fundamental representation, with short-range interactions extending at most to triangular units. The evidence for Abelian chiral spin liquid (CSL) phases in such models is obtained via a combination of complementary numerical methods such as exact diagonalizations (ED), infinite density matrix renormalization group (iDMRG) and infinite Projected Entangled Pair State (iPEPS). Extensive ED on small clusters are carried out up to $N=10$, revealing (in some range of the Hamiltonian parameters) a bulk gap and ground-state degeneracy on periodic clusters as well as linear dispersing chiral modes on the edge of open systems, whose level counting is in full agreement with SU($N$)$_1$ Wess-Zumino-Witten conformal field theory predictions. Using an SU($N$)-symmetric version of iDMRG for $N=2,3$ and $4$ to compute entanglement spectra on (infinitely-long) cylinders in all topological sectors, we provide additional unambiguous signatures of the SU($N$)$_1$ character of the chiral liquids. An SU($4$)-symmetric chiral PEPS is shown to provide a good variational ansatz of the $N=4$ ground state, constructed in a manner similar to its $N=2$ and $N=3$ analogs. The entanglement spectra in all topological sectors of an infinitely long cylinder reveal specific features of the chiral edge modes originating from the PEPS holographic bulk-edge correspondence. Results for the correlation lengths suggest some form of long-range correlations in SU($N$) chiral PEPS, which nevertheless do not preclude an accurate representation of the gapped SU($N$) CSL phases. Finally, we discuss the possible observation of such Abelian CSL in ultracold atom setups.