We investigate the magnetic characteristics of a persistent coronal hole (CH) extracted from EUV imagery using HMI filtergrams over the timerange February 2012-October 2012. The magnetic field, its distribution as well as the magnetic fine structure in form of flux tubes (FT) are analyzed in different evolutionary states of the CH. We find a strong linear correlation between the magnetic properties (e.g. signed/unsigned magnetic field strength) and area of the CH. As such, the evolutionary pattern in the magnetic field clearly follows the three-phase evolution (growing, maximum and decaying phase) as found from EUV data (Part I). This evolutionary process is most likely driven by strong FTs with a mean magnetic field strength exceeding 50 G. During the maximum phase they entail up to 72% of the total signed magnetic flux of the CH, but only cover up to 3.9% of the total CH area, whereas during the growing and decaying phase, strong FTs entail 54-60% of the signed magnetic flux and cover around 1-2% of the CHs total area. We conclude that small scale-structures of strong unipolar magnetic field are the fundamental building blocks of a CH and govern its evolution.