In the paper Einstein metrics on compact simple Lie groups attached to standard triples, the authors introduced the definition of standard triples and proved that every compact simple Lie group $G$ attached to a standard triple $(G,K,H)$ admits a left-invariant Einstein metric which is not naturally reductive except the standard triple $(Sp(4),2Sp(2),4Sp(1))$. For the triple $(Sp(4),2Sp(2),4Sp(1))$, we find there exists an involution pair of $sp(4)$ such that $4sp(1)$ is the fixed point of the pair, and then give the decomposition of $sp(4)$ as a direct sum of irreducible $ad(4sp(1))$-modules. But $Sp(4)/4Sp(1)$ is not a generalized Wallach space. Furthermore we give left-invariant Einstein metrics on $Sp(4)$ which are non-naturally reductive and $Ad(4Sp(1))$-invariant. For the general case $(Sp(2n_1n_2),2Sp(n_1n_2),2n_2Sp(n_1))$, there exist $2n_2-1$ involutions of $sp(2n_1n_2)$ such that $2n_2sp(n_1))$ is the fixed point of these $2n_2-1$ involutions, and it follows the decomposition of $sp(2n_1n_2)$ as a direct sum of irreducible $ad(2n_2sp(n_1))$-modules. In order to give new non-naturally reductive and $Ad(2n_2Sp(n_1)))$-invariant Einstein metrics on $Sp(2n_1n_2)$, we prove a general result, i.e. $Sp(2k+l)$ admits at least two non-naturally reductive Einstein metrics which are $Ad(Sp(k)timesSp(k)timesSp(l))$-invariant if $k<l$. It implies that every compact simple Lie group $Sp(n)$ for $ngeq 4$ admits at least $2[frac{n-1}{3}]$ non-naturally reductive left-invariant Einstein metrics.