I would guess the NIMBYs "somewhat support" affordable housing (i.e. "somewhere else, just not here"), and that they're mixed in with further home owners who are afraid of lower prices affecting their "investment." I imagine it's mostly real estate speculators that make up the bulk of the ones against affordable housing.
Actually, the government directing the CHMC to pivot from "direct public housing investment and urban planning" to "mortgage supports and bribing developers" is about the same time our housing market started to get out of control.