Something that is driving up prices in California is the cost to start construction. Construction permit for El Dorado County where I live is around $100,000 to $125,000. This does not include a water meter, which runs around $30,000 for a 3/4" inch line. Water costs about $100 a month depending on landscaping and # of occupants.
In Yavapai County, AZ (Prescott area) the same construction permits are around $12,000 to $15,000.
The price of raw land can also be pretty expensive in California. Where I live it's 5 acre parcels and sell for around $250,000 but may include a water meter.
So, for new construction single family custom home, not a subdivision, a owner/builder has to plunk down $350,000 to $375,000 before pounding even one nail into a board.
Also, property taxes may drive where people are moving and why. California property tax rate is 1% of the home's value and capped to 2% annual increase by law. There are other fees added to the tax bill as well as the basic property taxes; emergency ambulance fees, school bond fees, etc will add up. This is not the place for a first time home buyer. Many first time home buyers are leaving California due to these costs and I'd bet a good amount are heading to Florida, Arizona, Nevada, etc.