Classical machine learning has succeeded in the prediction of both classical and quantum phases of matter. Notably, kernel methods stand out for their ability to provide interpretable results, relating the learning process with the physical order parameter explicitly. Here, we exploit quantum kernels instead. They are naturally related to the fidelity and thus it is possible to interpret the learning process with the help of quantum information tools. In particular, we use a support vector machine (with a quantum kernel) to predict and characterize quantum phase transitions. The general theory is tested in the Ising chain in transverse field. We show that for small-sized systems, the algorithm gives accurate results, even when trained away from criticality. Besides, for larger sizes we confirm the success of the technique by extracting the correct critical exponent $ u$. The characterization is completed by computing the kernel alignment between the quantum and ideal kernels. Finally, we argue that our algorithm can be implemented on a circuit based on a varational quantum eigensolver.