In the collapsar scenario, gamma ray bursts are caused by relativistic jets expelled along the rotation axis of a collapsing stellar core. We discuss how the structure and time-dependence of such jets depends on the stellar envelope and central engine properties, assuming a steady jet injection. It takes a few seconds for the jet to bore its way through the stellar core; most of the energy output during that period goes into a cocoon of relativistic plasma surrounding the jet. This material subsequently forms a bubble of magnetized plasma that takes several hours to expand, subrelativistically, through the envelope of a high-mass supergiant. Jet break-through and a conventional burst would be expected not only in He stars but possibly also in blue supergiants. Shock waves and magnetic dissipation in the escaping bubble can contribute a non thermal UV/X-ray afterglow, and also excite Fe line emission from thermal gas, in addition to the standard jet deceleration power-law afterglow.