Verse 30 sets up a contrast between those who have much and those who have little. One of the undercurrents of the ideal Nephi community is egalitarianism, and Jacob is decrying this inequality. He will say more in the book of Jacob. At this point, Jacob is careful not to condemn the things of the world. The condemnation is for those who allow the things of this world to alter the way they behave toward others.
The next two verses are a doubled pair. The mention of the deaf in verse 31 is paired with the blind in verse 32. In both cases, the problem is not deafness or blindness, but a willingness to turn a deaf ear and blind eye to others. It is, in different words, a continuation of the condemnation of riches.
Those who are rich as to the things of the world may have acquired those things by ignoring their figurative brothers and sisters. They are deaf and blind to the needs of others, not because of a physical impairment, but a spiritual one.
Just as Jacob noted that those who have the law and do not follow it will perish, those who do not truly see or hear what Jehovah wishes them to do will likewise perish.