Jakob Nielsen has some great advice on how many users to test with. He advises that you will expose the majority of your issues from testing with five users at a time but critically you should then be running multiple tests. So if you have a budget for 10 users, test with 5 and fix the issues, then test with the remaining 5. This is much more beneficial than testing with 10 first off because it gives you an extra iteration of improvements to the design.