We develop a numerical code to calculate the neutrino transfer with multi-energy and multi-angle in three dimensions (3D) for the study of core-collapse supernovae. The numerical code solves the Boltzmann equations for neutrino distributions by the discrete-ordinate (S_n) method with a fully implicit differencing for time advance. The Boltzmann equations are formulated in the inertial frame with collision terms being evaluated to the zeroth order of v/c. A basic set of neutrino reactions for three neutrino species is implemented together with a realistic equation of state of dense matter. The pair process is included approximately in order to keep the system linear. We present numerical results for a set of test problems to demonstrate the ability of the code. The numerical treatments of advection and collision terms are validated first in the diffusion and free streaming limits. Then we compute steady neutrino distributions for a background extracted from a spherically symmetric, general relativistic simulation of 15Msun star and compare them with the results in the latter computation. We also demonstrate multi-D capabilities of the 3D code solving neutrino transfers for artificially deformed supernova cores in 2D and 3D. Formal solutions along neutrino paths are utilized as exact solutions. We plan to apply this code to the 3D neutrino-radiation hydrodynamics simulations of supernovae. This is the first article in a series of reports on the development.