The idea of the great City of Jerusalem being destroyed was an extraordinary thought. The estimated population for all of Jerusalem in Lehi’s day was maybe 10,000 people. Most people did not live in the city but were living on farms in their local villages. Jerusalem was "great"—not so much in the sense that there were lots of people residing there. It was "great" because it was holy and the temple was there. It was the Holy Place mentioned several times in Deuteronomy 12. It was the home of God. For many reasons, the idea that God’s holy place could be destroyed seemed sacrilegious, ridiculous, even impossible.
Seely, David Rolph, and Fred E. Woods. "How Could Jerusalem, ‘That Great City,’ Be Destroyed?" In Glimpses of Lehi’s Jerusalem, edited by John W. Welch, David Rolph Seely and Jo Ann H. Seely, 595–610. Provo, UT: Foundation for Ancient Research and Mormon Studies, 2004.