As a consequence of motions driven by external forces, self-fields (which are different from the static case) originate within an electron bunch. In the case of magnetic external forces acting on an ultrarelativistic beam, the longitudinal self-interactions are responsible for CSR (Coherent Synchrotron Radiation)-related phenomena, which have been studied extensively. On the other hand, transverse self-interactions are present too. At the time being, existing theoretical analysis of transverse self-forces deal with the case of a bunch moving along a circular orbit only, without considering the situation of a bending magnet with a finite length. In this paper we propose an electrodynamical analysis of transverse self-fields which originate, at the position of a test particle, from an ultrarelativistic electron bunch moving in an arc of a circle. The problem will be first addressed within a two-particle system. We then extend our consideration to a line bunch with a stepped density distribution, a situation which can be easily generalized to the case of an arbitrary density distribution. Our approach turns out to be also useful in order to get a better insight in the physics involved in the case of simple circular motion and in order to address the well known issue of the partial compensation of transverse self-force.