Just to substantiate some of this with HCI (Human-Computer Interaction) research: 5-20 is considered the "ideal" for exactly the reasons mentioned (2 users will probably overlap with about 50%+ of the bugs they find, and so on). This number was arrived at by Jakob Nielsen and substantiated by a few other HCI researches (see Article). The idea is that you need 5 users from each 'group'. Your users need to be comparable - so you'd need 5 no0bies and 5 experienced players, because the bugs and issues they'd find would be quite different. I absolutely recommend letting people test your game who are unfamiliar with conventional games. Some things that an experienced player will take for granted could be absolutely nightmarish to a new user. Account for that! Another excellent suggestion mentioned was to give users tasks to complete - this hones their attention, gives them clear and measurable goals, and makes them feel like they accomplish things (sound familiar, anyone? User testing works just like most games! ) If anyone's interested in how to create those tasks and look at playtesting over the whole course of development (not just beta), here's a presentation on different methods and what you can get out of playtesting at various stages of development.
