Housing is far from pure capitalism in most places (especially California). So many rules and regulations and hoops you have to jump through to get approval to build etc. that that process alone makes building "affordable" housing almost impossible due to the costs dealing with the aforementioned issues. And then you throw in gov't controls such as rent control which only cause prices to rise and you have the housing (and affordability) crisis we have today.
We need more capitalism in housing allowing supply to keep up with demand, allowing for ADU's and the conversion of garages etc. There are things we can do but NIMBYism is a powerful powerful force (and it's also bi-partisan).