Reference-frame-independent measurement-device-independent quantum key distribution (RFI-MDI-QKD) is a novel protocol which eliminates all possible attacks on detector side and necessity of reference-frame alignment in source sides. However, its performance may degrade notably due to statistical fluctuations, since more parameters, e.g. yields and error rates for mismatched-basis events, must be accumulated to monitor the security. In this work, we find that the original decoy-states method estimates these yields over pessimistically since it ignores the potential relations between different bases. Through processing parameters of different bases jointly, the performance of RFI-MDI-QKD is greatly improved in terms of secret key rate and achievable distance when statistical fluctuations are considered. Our results pave an avenue towards practical RFI-MDI-QKD.