We study the quantum phase transition of the Dicke model in the classical oscillator limit, where it occurs already for finite spin length. In contrast to the classical spin limit, for which spin-oscillator entanglement diverges at the transition, entanglement in the classical oscillator limit remains small. We derive the quantum phase transition with identical critical behavior in the two classical limits and explain the differences with respect to quantum fluctuations around the mean-field ground state through an effective model for the oscillator degrees of freedom. With numerical data for the full quantum model we study convergence to the classical limits. We contrast the classical oscillator limit with the dual limit of a high frequency oscillator, where the spin degrees of freedom are described by the Lipkin-Meshkov-Glick model. An alternative limit can be defined for the Rabi case of spin length one-half, in which spin frequency renormalization replaces the quantum phase transition.