Basically the argument goes that because the market is not able to balance what people actually want, housing markets such as San Francisco, New York and many city centers to cost much more comparatively to places in the periphery. In addition, home owners don't want to see change. They like things the way they are and become an entrenched entity against any densification seeking to put all new growth somewhere else.
I agree with all of this but also would like to note that markets for density are highly dependent on agglomeration.
If land prices are rising, as they are empirically, firms economize on land. This behavior increases density and contributes to growth.But what causes land prices to rise, or at least be high enough to support economization and higher densities? I would say that there needs to be a key catalyst, perhaps a major employer moving into an area or a major landowner or government entity focusing energies into a single place. These infrastructure investments increase land value and in turn make new dense developments possible. The demand for this type of living is real, but the ability to supply it can be harder and more locationally dependent than general sprawl.
It's also based on access. Just because someone runs a light rail line to a destination doesn't mean that a market for density is going to magically appear. If we think about where suburban centers pop up, it generally has to do with the transportation network and infrastructure that was set up to support it.
Ultimately the densest places are those that grew up close to where the major employment centers are located or proximate enough to the other largest employment center in the region with access enough to feed on it. Tyson's for example feeds off of the DC metro area and is suffocating. In order to get denser, the infamous edge city has to upgrade its circulation system and throughput. The Silver Line starts to do this and plans for a better grid and streetcar system are in the works.
But sometimes landowners believe their land is worth more than it actually is which stifles density plans as well. For example, in Houston in Midtown along the Main Street Corridor, there are some land owners just holding out for super high density projects that the market can't bear quite yet.
The typical price per square foot for land in the Midtown area grew from $4 per square foot in the early 1990s to more than $50 per square foot in 2006. This is in part due to land speculation fueled by the new light-rail line, with some buyers purchasing land in anticipation of higher land values in the future.Or burdensome regulations such as parking requirements take the possibility of building higher density out of the mix. Once you get over a certain height, steel instead of wood must be used for construction and costs increase again. But all of this isn't possible if the land values are low or if demand isn't there. Demand typically increases when existing densities exist. But for many cities or station areas, this can be tricky. We can say that there is a demand for denser living, but we also need to know where the market exists to expand the agglomerations that exist, because unlike sprawl, we can't just build into nowhere land.