The speed of quantum gates and measurements is a decisive factor for the overall fidelity of quantum protocols when performed on physical qubits with finite coherence time. Reducing the time required to distinguish qubit states with high fidelity is therefore a critical goal in quantum information science. The state-of-the-art readout of superconducting qubits is based on the dispersive interaction with a readout resonator. Here, we bring this technique to its current limit and demonstrate how the careful design of system parameters leads to fast and high-fidelity measurements without affecting qubit coherence. We achieve this result by increasing the dispersive interaction strength, by choosing an optimal linewidth of the readout resonator, by employing a Purcell filter, and by utilizing phase-sensitive parametric amplification. In our experiment, we measure 98.25% readout fidelity in only 48 ns, when minimizing read-out time, and 99.2% in 88 ns, when maximizing the fidelity, limited predominantly by the qubit lifetime of 7.6 us. The presented scheme is also expected to be suitable for integration into a multiplexed readout architecture.