Depends on your gun(s). If you don't own any K-98 Mausers, for example, the bit for Mauser action screws is likely to go unused for a very long time.
Yes, you'll definitely put a lot of wear on a few bits, and almost none on some others. For me, the smaller bits get used a lot, the big ones less often, and the "middle" sized ones least. You'll figure out quickly what your high-use bits are. Buy extras of those: when you lose or break that bit, it's nice to have the spare already on hand.
The idea of buying only what you'll use the most may seem efficient, but it's false economy. When you do need that one oddball bit, you won't have it, and you'll use the next closest bit, or <gasp!> a taper-ground screwdriver. Either way, you have a better than 50/50 chance of buggering up the slot, which is what you're trying to avoid in the first place.

The Brownells or Chapman basic sets are in the $30-50 range, and are well worth it.
While you're at it, get a set of punches, both flat-faced and roll-pin. Craftsman punches are nice: if you use them a lot, eventually you'll bend or break one, and Sears still replaces them free.