We investigate kinetic entropy-based measures of the non-Maxwellianity of distribution functions in plasmas, i.e., entropy-based measures of the departure of a local distribution function from an associated Maxwellian distribution function with the same density, bulk flow, and temperature as the local distribution. First, we consider a form previously employed by Kaufmann and Paterson [{it J.~Geophys.~Res.,} {bf 114}, A00D04 (2009)], assessing its properties and deriving equivalent forms. To provide a quantitative understanding of it, we derive analytical expressions for three common non-Maxwellian plasma distribution functions. We show that there are undesirable features of this non-Maxwellianity measure including that it can diverge in various physical limits and elucidate the reason for the divergence. We then introduce a new kinetic entropy-based non-Maxwellianity measure based on the velocity-space kinetic entropy density, which has a meaningful physical interpretation and does not diverge. We use collisionless particle-in-cell simulations of two-dimensional anti-parallel magnetic reconnection to assess the kinetic entropy-based non-Maxwellianity measures. We show that regions of non-zero non-Maxwellianity are linked to kinetic processes occurring during magnetic reconnection. We also show the simulated non-Maxwellianity agrees reasonably well with predictions for distributions resembling those calculated analytically. These results can be important for applications, as non-Maxwellianity can be used to identify regions of kinetic-scale physics or increased dissipation in plasmas.