An extended volume of fluid method is developed for two-phase direct numerical simulations of systems with one viscoelastic and one Newtonian phase. A complete set of governing equations is derived by conditional volume-averaging of the local instantaneous bulk equations and interface jump conditions. The homogeneous mixture model is applied for the closure of the volume-averaged equations. An additional interfacial stress term arises in this volume-averaged formulation which requires special treatment in the finite-volume discretization on a general unstructured mesh. A novel numerical scheme is proposed for the second-order accurate finite-volume discretization of the interface stress term. We demonstrate that this scheme allows for a consistent treatment of the interface stress and the surface tension force in the pressure equation of the segregated solution approach. Because of the high Weissenberg number problem, an appropriate stabilization approach is applied to the constitutive equation of the viscoelastic phase to increase the robustness of the method at higher fluid elasticity. Direct numerical simulations of the transient motion of a bubble rising in a quiescent viscoelastic fluid are performed for the purpose of experimental code validation. The well-known jump discontinuity in the terminal bubble rise velocity when the bubble volume exceeds a critical value is captured by the method. The formulation of the interfacial stress together with the novel scheme for its discretization is found crucial for the quantitatively correct prediction of the jump discontinuity in the terminal bubble rise velocity.