The two key words to consider in this question are Nephites and destroyed.
It is well to remember that the word Nephite apparently had a different meaning near the close of the Book of Mormon than it did before A.D. 231. (See 4 Nephi 20, 36-39.) From approximately 544 B.C. to A.D. 34 (Jacob 1:13-14) the term Nephite referred either to the descendants of Nephi and those who went with him into the wilderness (2 Nephi 5:5-8) or to those who joined with this group. From about A.D. 34 to nearly A.D. 194, there were not “any manner of -ites” among the descendants of Lehi. (4 Nephi 17.) By A.D. 194, however, a group of people revolted from the church and took upon themselves the name of Lamanites (4 Nephi 20), and by A.D. 231 those who had remained true to the church became known as Nephites (4 Nephi 36). For the remainder of the Book of Mormon, the term Nephite apparently refers to the descendants of those who had been faithful in A.D. 231.
Concerning whether or not all of the Nephites were destroyed, Hugh Nibley has written the following:
Are there not many Latter-day Saints who will insist that every American of pre-Columbian descent must be a Lamanite because, forsooth, there were once Nephites and Lamanites, and the Nephites were destroyed? Yet the Book of Mormon itself makes such an interpretation impossible. The Nephites were destroyed, we are told, but … what does the Book of Mormon mean by “destroyed”? The word is to be taken, as are so many other key words in the book, in its primary and original sense: “to unbuild; to separate violently into its constituent parts; to break up the structure.” To destroy is to wreck the structure, not to annihilate the parts. Thus in 1 Nephi 17:31 we read of Israel in Moses’ day that, “According to his word he did destroy them; and according to his word he did lead them …” bringing them together after they had been “destroyed,” i.e., scattered, and needed a leader. “As one generation hath been destroyed among the Jews,” according to II Nephi 25:9, “… even so they have been destroyed from generation to generation according to their iniquities.” A complete slaughter of any one generation would of course be the end of their history altogether, but that is not what “destroyed” means. Of the Jews at Jerusalem Nephi says (1 Nephi 17:43), “I know that the day must shortly come that they must be destroyed, save only a few… .” Later, “after the Messiah hath arisen from the dead … behold Jerusalem shall be destroyed again …” (II Nephi 25:15). In these two cases what actually happened was that the Jews were all scattered “save a few only” that remained in the land. The Israelites upon entering the Promised Land, we are told, drove out “the children of the land, yea, unto the scattering them to destruction” (1 Nephi 17:32). Here it is plainly stated that the destruction of the Canaanites was their scattering—as is known to have been the case. Likewise of the Nephites: “… and after thy seed shall be destroyed, and dwindle in unbelief, and also the seed of thy brethren, behold these things shall be hid up” (1 Nephi 13:35), where both Nephites and Lamanites dwindle in unbelief afterthey have been destroyed.Only once in the Book of Mormon do we read of a case of annihilation, when we are specifically told that “every living soul of the Ammonihahites was destroyed” (Alma 16:9), where not only the social structure but each individual is undone. In other instances the Lord promises that he will not utterly destroy the descendants of Lehi’s youngest son, Joseph (II Nephi 3:3), or of Lemuel (id., 4:9), and even Nephi is told that God “will not suffer that the Gentiles will utterly destroy the mixture of thy seed which are among thy brethren” (1 Nephi 13:30), even though the promises and fulfillment were that the Nephites should be “destroyed” (Ether 8:21), and even though Moroni can say: “there is none, save it be Lamanites” (Ether 4:3). (Lehi in the Desert and the World of the Jaredites, pp. 240-42.)
Also, it is of interest to note that the Lord has referred to Nephites in this dispensation. (See D&C 3:16-19.)