Jets and outflows are thought to play important roles in regulating star formation and disk evolution. HD 163296 is a well-studied Herbig Ae star that hosts proto-planet candidates, a protoplanetary disk, a protostellar jet, and a molecular outflow, which makes it an excellent laboratory for studying jets. We aim to characterize the jet at the inner regions and check if there are large differences with the features at large separations. A secondary objective is to demonstrate the performance of Multi Unit Spectroscopic Explorer (MUSE) in high-contrast imaging of extended line emission. MUSE in the narrow field mode (NFM) can provide observations at optical wavelengths with high spatial ($sim$75 mas) and medium spectral ($Rsim$2500) resolution. With the high-resolution spectral differential imaging (HRSDI) technique, we can characterize the kinematic structures and physical conditions of jets down to 100 mas. We detect multiple atomic lines in two new knots, B3 and A4, at distances of <4 from the host star with MUSE. The derived $dot{M}_{rm jet} / dot{M}_{rm acc}$ is about 0.08 and 0.06 for knots B3 and A4, respectively. The observed [Ca II]/[S II] ratios indicate that there is no sign of dust grains at distances of <4. Assuming the knot A4 traces the streamline, we set an upper limit of 2.2 au on the size of the launching region. Although MUSE has the ability to detect the velocity shifts caused by high- and low-velocity components, we found no significant evidence of velocity decrease transverse to the jet direction. Our work demonstrates the capability of using MUSE NFM observations for the detailed study of stellar jets in the optical down to 100~mas. The derived $dot{M}_{rm jet} / dot{M}_{rm acc}$, no dust grain, and jet radius at the star support the magneto-centrifugal models as a launching mechanism for the jet.