Depends on the trip. Family camping, you can never go too big in my opinion. Especially with kids. The more room for everyone to stretch out and not be on top of each other, the better.
I've managed in solo tents in the past, even without a vestibule, by either cramming my gear in by my legs or strapping my pack to a tree and covering it with a plastic bag. It depends on what kind of trip you're planning, how big a pack you'll be carrying, if you're hiking or just driving into a site, etc. Obviously for hiking trips you'd be packing as light as possible so you might be able to squeeze gear into a tent rated for the exact number of people you have.