Two words: elasticity and emotions. Nowadays we really have little extra capacity in the oil and gas system. We use such huge volumes everyday that the draw down rate really doesn't give us a comfortable amount of time or ability to adapt. Related to this is that during hurricanes, for example, a fair amount of domestic infrastructure can be taken out, oil platforms, refineries, nat gas pipes, all further compounding problems.
The fragile nature of this system or the perception (or misperception?) of this fragility can easily lead to an emotional reaction of panic and fear. This certainly can happen among end users such as ourselves but, more interesting to me, is that energy traders and producers can also have a similar reaction and then act on it. Sadly, they can also create the appearance of a panic without there actually being one. This is when prices often go up for no good reason.
Remember the many natural gas traders that were scamming the system during early deregulation of the California utility market, right before Enron went down? I suspect traders and energy companies are still getting panicked and also working in some "extra" manipulation at times. Some low lives probably think this is in their best self interest--and part of grand and good system.
A sick sort of rationalization process among a few manipulators. They skew themselves.
It's hard to fight purposeful manipulation of perceptions or spin for one's own advantage, especially when done in carefully planned and long term way.
Not that I'm paranoid.
edit: There is the real problem and the appearance of the problem. In today's complex and large world it is very, very--very--hard for any of us to get at the real problems. We mostly depend on others for our information, such as TV news or papers or, um, gov't. We end up not only having to look for and finding the real original problem but also wading thru enormous quantities of additional human biases that skew our lenses in the process.