We present a formulation of Regge Calculus where arbitrary coordinates are associated to each vertex of a simplicial complex and the degrees of freedom are given by the metric on each simplex. The lengths of the edges are thus determined and are left invariant under arbitrary transformations of the discrete set of coordinates, provided the metric transforms accordingly. Invariance under coordinate transformations entails tensor calculus and our formulation follows closely the usual formalism of the continuum theory. The definitions of parallel transport, Christoffel symbol, covariant derivatives and Riemann curvature tensor follow in a rather natural way. In this correspondence Einstein action becomes Regge action with the deficit angle $theta$ replaced by $sin theta$. The correspondence with the continuum theory can be extended to actions with higher powers of the curvature tensor, to the vielbein formalism and to the coupling of gravity with matter fields (scalars, fermionic fields including spin $3/2$ fields and gauge fields) which are then determined unambiguously and discussed in the paper. An action on the simplicial lattice for $N=1$ supergravity in $4$ dimensions is derived in this context. Another relavant result is that Yang-mills actions on a simplicial lattice consist, even in absence of gravity, of two plaquettes terms, unlike the one plaquette Wilson action on the hypercubic lattice. An attempt is also made to formulate a discrete differential calculus to include differential forms of higher order and the gauging of free differential algebras in this scheme. However this leads to form products that do not satisfy associativity and distributive law with respect to the $d$ operator. A proper formulation of theories that contain higher order differential forms in the context of Regge Calculus is then still lacking.