Improving the modelling of redshift-space distortions- II. A pairwise velocity model covering large and small scales