For a tuple $A=(A_1, A_2, ..., A_n)$ of elements in a unital Banach algebra ${mathcal B}$, its {em projective joint spectrum} $P(A)$ is the collection of $zin {mathbb C}^n$ such that the multiparameter pencil $A(z)=z_1A_1+z_2A_2+cdots +z_nA_n$ is not invertible. If ${mathcal B}$ is the group $C^*$-algebra for a discrete group $G$ generated by $A_1, A_2, ..., A_n$ with respect to a representation $rho$, then $P(A)$ is an invariant of (weak) equivalence for $rho$. This paper computes the joint spectrum of $(1, a, t)$ for the infinite dihedral group $D_{infty}=<a, t | a^2=t^2=1>$ with respect to the left regular representation $lambda_D$, and gives an in-depth analysis on its properties. A formula for the Fuglede-Kadison determinant of the pencil $R(z)=1+z_1a+z_2t$ is obtained, and it is used to compute the first singular homology group of the joint resolvent set $P^c(R)$. The joint spectrum gives new insight into some earlier studies on groups of intermediate growth, through which the corresponding joint spectrum of $(1, a, t)$ with respect to the Koopman representation $rho$ (constructed through a self-similar action of $D_{infty}$ on a binary tree) can be computed. It turns out that the joint spectra with respect to the two representations coincide. Interestingly, this fact leads to a self-similar realization of the group $C^*$-algebra $C^*(D_{infty})$. This self-similarity of $C^*(D_{infty})$ is manifested by some dynamical properties of the joint spectrum.