So we would hope they learned from that mistake when they were planning BART and actually decide to connect places, but give people a faster option, but they decided to double down with aweful all in the same of saving money. Sure they are saving money using existing ROW for BART, but they are also skipping destinations they need to connect to make it successful.
Light Rail is Dark Purple, Caltrain is Red, Plannded BART is Steel Blue, Green are areas of high employment density.
You can see that the planned BART line skips all of the North Valley tech employment and instead makes people depend on a slow light rail system to connect. Even when BART is complete to Berryesa, it won't be as effective as it would have been going under or through this employment cluster into downtown. Yes it would have cost more but the investment would have been there for hundreds of years.
Additionally, as I've mentioned in previous posts (1, 2), when silicon valley does get dense, it's in horrible suburban layouts. You can see below along the San Jose LRT line how buildings suck ridership right out of the system with parking and bad design.
The last image above below shows how many buildings you could fit in this space if they had better non auto oriented design. And I guarantee this would drive ridership along the line.
Now there have also been discussions of how Silicon Valley needs to become Manhattan in order to keep talent that wants to live in urban places instead of valley sprawl. An article in the Awl made this claim but in reality, Silicon Valley doesn't need a hefty core of ultra tall buildings, it just needs to use the space it has better and become the DC or Paris of the Western United States. There's so much opportunity, yet it is completely wasted.
So in my eyes the transit is part of the problem for not making the connections that increase property values to do this type of infill, but its also the fault of developers who don't understand that a classic way of building for pedestrians is needed to attract pedestrians and quality of life that people are moving to San Francisco to attain. Sure some people don't want that, but we have more than enough supply of single family homes if there's more of a choice.