Let $mathbb{C}^{ntimes n}$ be the set of all $n times n$ complex matrices. For any Hermitian positive semi-definite matrices $A$ and $B$ in $mathbb{C}^{ntimes n}$, their new common upper bound less than $A+B-A:B$ is constructed, where $(A+B)^dag$ denotes the Moore-Penrose inverse of $A+B$, and $A:B=A(A+B)^dag B$ is the parallel sum of $A$ and $B$. A factorization formula for $(A+X):(B+Y)-A:B-X:Y$ is derived, where $X,Yinmathbb{C}^{ntimes n}$ are any Hermitian positive semi-definite perturbations of $A$ and $B$, respectively. Based on the derived factorization formula and the constructed common upper bound of $X$ and $Y$, some new and sharp norm upper bounds of $(A+X):(B+Y)-A:B$ are provided. Numerical examples are also provided to illustrate the sharpness of the obtained norm upper bounds.