We study the Japan and U.S. patent records of several decades to demonstrate the effect of collaboration on innovation. We find that statistically inventor teams slightly outperform solo inventors while company teams perform equally well as solo companies. By tracking the performance record of individual teams we find that inventor teams performance generally degrades with more repeat collaborations. Though company teams performance displays strongly bursty behavior, long-term collaboration does not significantly help innovation at all. To systematically study the effect of repeat collaboration, we define the repeat collaboration number of a team as the average number of collaborations over all the teammate pairs. We find that mild repeat collaboration improves the performance of Japanese inventor teams and U.S. company teams. Yet, excessive repeat collaboration does not significantly help innovation at both the inventor and company levels in both countries. To control for unobserved heterogeneity, we perform a detailed regression analysis and the results are consistent with our simple observations. The presented results reveal the intricate effect of collaboration on innovation, which may also be observed in other creative projects.