We present results on the mass of the nucleon and the Delta using two dynamical degenerate twisted mass quarks and the tree-level Symanzik improved gauge action. The evaluation is performed at four quark masses corresponding to a pion mass in the range of about 300-600 MeV on lattices of 2.1-2.7 fm. We check for cut-off effects by evaluating these baryon masses on lattices of spatial size 2.1 fm at beta=3.9 and beta=4.05 and on a lattice of 2.4 fm at beta=3.8. The values we find are compatible within our statistical errors. Lattice results are extrapolated to the physical limit using continuum chiral perturbation theory. Performing a combined fit to our lattice data at beta=3.9 and beta=4.05 we find a nucleon mass of 964pm 28 (stat.) pm 8 (syst.) MeV. The nucleon mass at the physical point provides an independent determination of the lattice spacing. Using heavy baryon chiral perturbation theory at O(p^3) we find a_{beta=3.9}=0.0890pm 0.0039(stat.) pm 0.0014(syst.) fm, and a_{beta=4.05}= 0.0691pm 0.0034(stat.) pm 0.0010(syst.) fm, in good agreement with the values determined from the pion decay constant. Isospin violating lattice artifacts in the Delta-system are found to be compatible with zero for the values of the lattice spacings used in this work. Performing a combined fit to our lattice data at beta=3.9 and beta=4.05 we find for the masses of the Delta^{++,-} and Delta^{+,0} 1316 pm 60 (stat.) MeV and 1330 pm 74 (stat.) MeV respectively. We confirm that in the continuum limit they are also degenerate.