The nearby low-luminosity active galactic nucleus (LLAGN) NGC 4258 has a weak radio continuum component at the galactic center. We investigate its radio spectral properties on the basis of our new observations using the Nobeyama Millimeter Array at 100 GHz and archival data from the Very Large Array (VLA) at 1.7-43 GHz and the James Clerk Maxwell telescope at 347 GHz. The NGC 4258 nuclear component exhibits (1) an intra-month variable and complicated spectral feature at 5-22 GHz and (2) a slightly inverted spectrum at 5-100 GHz (a spectral index of ~0.3) in time-averaged flux densities, which are also apparent in the closest LLAGN M81. These similarities between NGC 4258 and M81 in radio spectral natures in addition to previously known core shift in their AU-scale jet structures produce evidence that the same mechanism drives their nuclei. We interpret the observed spectral property as the superposition of emission spectra originating at different locations with frequency-dependent opacity along the nuclear jet. Quantitative differences between NGC 4258 and M81 in terms of jet/counter jet ratio, radio loudness, and degree of core shift can be consistently understood by fairly relativistic speeds (bulk Lorentz factors of >~ 3) of jet and their quite different inclinations. The picture established from the two closest LLAGNs is useful for understanding the physical origin of unresolved and flat/inverted spectrum radio cores that are prevalently found in LLAGNs, including Sgr A*, with starved supermassive black holes in the present-day universe.