This work is concerned with model reduction of stochastic differential equations and builds on the idea of replacing drift and noise coefficients of preselected relevant, e.g. slow variables by their conditional expectations. We extend recent results by Legoll & Leli`evre [Nonlinearity 23, 2131, 2010] and Duong et al. [Nonlinearity 31, 4517, 2018] on effective reversible dynamics by conditional expectations to the setting of general non-reversible processes with non-constant diffusion coefficient. We prove relative entropy and Wasserstein error estimates for the difference between the time marginals of the effective and original dynamics as well as an entropy error bound for the corresponding path space measures. A comparison with the averaging principle for systems with time-scale separation reveals that, unlike in the reversible setting, the effective dynamics for a non-reversible system need not agree with the averaged equations. We present a thorough comparison for the Ornstein-Uhlenbeck process and make a conjecture about necessary and sufficient conditions for when averaged and effective dynamics agree for nonlinear non-reversible processes. The theoretical results are illustrated with suitable numerical examples.