DH and I average about $4,000/yr., but that reflects some specific circumstances:
1. We travel an average of 3 wks./yr. to visit family and friends in CA and FL. Those trips include a mix of VRBO rentals, Best Westerns, and staying with the folks we are visiting. We eat out once/day at most, and buy groceries to cook a lot.
2. DH can't travel outside of US for medical reasons, and finds travel here difficult; so our budget reflects those constraints. We would travel more if we could.
3. When I travel outside US with girlfriends, the budget goes up another $3-4000.
Also, a note to Hiredgun-- It seems you have a wonderful opportunity to make priceless family memories. While DH and I did travel some with our son when he was growing up, it was usually to visit family and friends, only occasionally to see the world (ie., U.S., though he's probably been to Disneyworld about 5 times). We couldn't ER until ages 59 and 62, after he was grown and gone.
If we'd had the cash, freedom, time (and the health of that earlier season in life), I would have taken those five years, spent the $20,000/yr., and take the family to see the world. How many families get that chance?
When those five years are over, and your son grown, you can adjust your plans and budget. But you can never get those five years back, or his adolescence, or his willingness and availability to travel with you. What gifts!!!!!!! (IMHO)
FWIW....