I just paid $3.939/gal for 87 octane yesterday, in Millersville, MD. While that's bad, I can remember in 1979, Mom complaining when gas went up to around $1.10/gal. At the time, she was driving a 1975 LeMans with a 350-2bbl, and even in the best of conditions, it would get maybe 15 mpg.
Adjusting for inflation, that's like $4.26/gal in today's dollars. But worse, in that timeframe we had the oil embargo, and it was common for stations to run out of gas. There were also long lines at the pump. I can't remember if they actually rationed gas or did "odd/even" days like they did during the first oil embargo, but it was still pretty bad.
Bad enough, that in February of 1980, Mom traded that LeMans for a new Malibu coupe with a 229 V6 that could easily get 20 mpg. That doesn't sound so hot by today's standards, but it was a pretty big deal at the time. For us, at least.
For comparison, the 2003 Regal I filled up yesterday gets about 19-20 mpg at the worst, and can break 30 on the highway. So in this case at least, I'd say I'm in a lot better shape than my Mom was, back in 1979! Or '80, even, as the Regal still does better than that Malibu did.
As for people giving up their big cars again? What big cars? America gave up the last vestiges of those in the 1990's, with the exception of Ford's Panthers...but even those were increasingly fleet sales in later years (police, taxi, rental agencies, livery/limo services etc). We've moved on from cars to trucks, SUVs, and crossovers. And for the most part, the heavy sellers in those departments already get fairly good fuel economy. And people tend to be less reluctant to part with their full-sized trucks, than they were with cars.