The Western Cape government is to use part of the contested Tafelberg site in Sea Point for affordable housing.