like Black color is a carbon base color and the particle size of carbon is large than the other colors that's why its rubbing properties are poor. Similarly red and blue are in the same case. So to improve the color fastness we add more binder to improve the fastness properties of these colors. It doesn't mean that we can not achieve the best results with these colors. The required results can achieve but production cost will be increase. On the other hand the construction of the fabric also effects the fastness properties.
Read more: http://textilelearner.blogspot.com/2011/08/color-fastness-to-rubbing-rubbing_1201.html#ixzz4KzypYWpO