Using the solution phase space method, we investigate the thermodynamics of black holes in Einstein-aether-Maxwell theory, for which the traditional Wald method (covariant phase space method) fails. We show the first laws of thermodynamics and definitive entropy expressions at both Killing and universal horizons for some examples of exact black hole solutions, including 3-dimensional static charged quasi-BTZ black hole, two 4-dimensional static charged black holes and 3-dimensional rotating solution. At Killing horizons the entropies are exactly one quarter of the horizon area, but at universal horizons of 3-dimensional black holes, the entropies have a corrected term in addition to the one proportional to the horizon area.