Andy, I've run tracert from my Oahu IP address to other ER & financial discussion boards. (A poster researching early retirement would be quite likely to run across these same boards.) Although I've been on some of these boards for years with a succession of computers, this very rudimentary study of course would need many more data runs under many different times & load conditions to have any statistical significance. I ran these tracert commands on a Saturday afternoon, sequentially, one run each. I have no idea where these servers are located, how they're configured, or how they run. If it'll help your analysis then I can e-mail the DOS screen dumps, but I understand if this isn't much use to you or your server hosts.
Here are the numbers:
E-R.org: 1261, 1349 (both servers)
M*: 800+ before timeouts
Bogleheads: 1412
FIRECalc: 1217
REHP: 1429
FundAlarm: 1303
ER status board: 877
RADDR's board: 893
I ran tracert for a local board (whose servers seem to be in Los Angeles) that's also running vBulletin. (I don't view any other boards using vBulletin.) Hawaii Threads: 458.
You would think that numbers varying from 893-1429 msec would not be very noticeable-- a difference of barely more than half a second. When I go to most of those boards, I can't really tell whether there's a lag. You would also expect Hawaii Threads to be pretty snappy compared to the rest, yet it's almost as slow as E-R.org.
However there's very little lag on FIRECalc, REHP, FundAlarm, the ER status board, and RADDR's board. Even though the routes are roughly the same delay as E-R.org, all of those other boards are faster and some are much faster. The browsing routine is click, new screen, click, new screen, with no lag. The screen draws almost as fast as I click. FundAlarm is particularly snappy because Roy Weitz has been using the same barebones software since 1996. I've noticed that the PHPBB boards are pretty fast, too, particularly RADDR's & the ER status boards.
Coincidentally the slowest boards (almost as slow as E-R.org) seem to be M* and Hawaii Threads. One is clogged with animated ads and the other is the only other vBulletin board I browse.
Here's another interesting behavior on my IE6.0 browser. When I click on an E-R.org link, the status bar fills up its progress box with the little green blocks. Sometimes it's slow, sometimes they zip right into the box. No matter how fast the box fills up, once it's full the browser sits there for at least a second, then it clears the screen. Then it fills in the background color and draws a graphic or two. Then, like a C+ program on a 1980s Silicon Graphics Unix box, it finally fills in the rest. Instead of click/new screen it's click, wait on boxes, draw, draw, fill, click, wait, draw, draw, fill. I have the same behavior from M* and Hawaii Threads. Again I don't know if it's the adserver or the vBulletin bloat. It's not my computer or my graphics card because I don't have this issue with any other discussion boards or websites. I do not believe that this delay is related to my ISP's packet routing, either, because I'm seeing about the same tracert numbers everywhere.
Here's what I think, and again this is just my opinion. First, I think that E-R.org is getting slower as we get more users. I know more people have signed up and I remember that at the end of 2006 we were starting to set records for numbers of people logged in at once. It's a little difficult for me to draw conclusions from the logs you've provided but I suspect the trend is continuing. It may noticeably slow down a server when a couple dozen people are simultaneously trying to view the same thread.
Second, I think E-R.org is crippled by [-]slow, bloated[/-] feature-rich vBulletin software and an adserver. These two changes have dramatically slowed things down from the Dory days.
I think that the only way to speed up the current configuration is to host servers that are so blindingly fast (or have so much capacity) that they can even run vBulletin and handle advertising. Another option would be to scrap vBulletin in favor of something like PHPBB. A third option would be to configure the servers so that the people who see ads are on a different server than everyone else-- the people who shouldn't see ads (like me) would be on a separate server that wouldn't have to coordinate with an adserver. But these changes would only be appreciated by the users, and they might not be feasible from a sysadmin perspective.
While it's easy for me to armchair quarterback, I also appreciate that you have no incentive to change. The only people who notice that the board is slow are those of us who were around when the board was fast. The new posters may notice that things are slow from time to time, but there are many complicating factors that would cause anyone less than a sysadmin to accept the status quo. It's only us veteran posters who find things unacceptably slow all the time. The incentive problem is that us veterans would rarely, if ever, click on an ad while the new users are much more likely to click on ads. There's no incentive to make the veterans happy because they don't generate any revenue.
I like vBulletin's "Ignore Poster" feature but I'm ready to give it up for more responsive software. I've always felt that the vBulletin change was for SocialKnowledge's benefits of standardization & board administration and not for the users. The users have had to deal with a lot of learning and conversion problems and frankly from a user's perspective the conversion hasn't been worth the results. If this was Microsoft Vista then I'd be going back to XP, asking for a refund, and looking into Linux.
I've never had a real job, but if I was maximizing a discussion board's revenue model then I'd want many new users clicking on ad links and I wouldn't care about the turnover. I'd actually prefer that the veterans move along-- they wouldn't generate revenue and their attempts to use the board would just slow down the servers that should be supporting the new ad-clicking users. Of course the board would eventually become filled with new users who have no sense of what's been previously discussed, who don't bother to read the archives, and who keep repeating the same topics ad nauseum. That might also encourage the veterans to move on in search of fresh content (problem solved!). The veterans would feel that their nice quiet bar has expanded into a noisy, chaotic nightclub... but you & I have already had this discussion.
Again it's just my opinion, and among the hordes of new users my opinion is definitely in the minority. While your routing work may help some users (particularly AOL), I think you're just tweaking the margins without solving the root issues.
But I'd be interested to hear what the board's other veteran posters think.