The effect of orbital degrees of freedom on the exchange interactions in the spin-1 quasi-one-dimensional antiferromagnet CaV2O4 is systematically studied. For this purpose a realistic low-energy model with the parameters derived from the first-principles calculations is constructed. The exchange interactions are calculated using both the theory of infinitesimal spin rotations near the mean-field ground state and the superexchange model, which provide a consistent description. The obtained behaviour of exchange interactions substantially differs from the previously proposed phenomenological picture based on the magnetic measurements and structural considerations, namely: (i) Despite quasi-one-dimensional character of the crystal structure, consisting of the zigzag chains of edge-sharing VO6 octahedra, the electronic structure is essentially three-dimensional, that leads to finite interactions between the chains; (ii) The exchange interactions along the legs of the chains appear to dominate; and (iii) There is a substantial difference of exchange interactions in two crystallographically inequivalent chains. The combination of these three factors successfully reproduces the behaviour of experimental magnetic susceptibility.