Brant Gardner notes that while the reader must wait until the book of Jacob for the extended allegory of the olive tree, Lehi and Nephi were obviously familiar with it, as Nephi made reference to the allegory in connection with the unfolding of Lehi's vision:
Yea, even my father spake much concerning the Gentiles, and also concerning the house of Israel, that they should be compared like unto an olive-tree, whose branches should be broken off and should be scattered upon all the face of the earth. (1 Nephi 10:12)
[Brant Gardner, "Brant Gardner's Page, Book of Mormon Commentary, [http://www.highfiber.com/~nahualli/LDStopics/1Nephi/1Nephi10.htm], p. 4]
“Whose Branches Should Be Broken Off and Should Be Scattered Upon All the Face of the Earth”
In 1 Nephi 10:12 (and 1 Nephi 15:12), Nephi brings up the idea that in the teachings of Lehi, he compared the house of Israel to an Olive tree "whose branches should be broken off and should be scattered upon all the face of the earth." This is not only subtle testimony to the truth of Isaiah's prophecies, but to the powerful allegory of Zenos recorded in Jacob 5.
McConkie and Millet state that the Lord chose an olive tree to dramatize the destiny of his chosen people. An olive tree almost never dies. It may be pruned and worked with over numerous generations before the fruit is such as to satisfy the owner of the vineyard; this is often after many and varied cuttings and trimmings and replantings. So it is with the house of Israel. [Joseph Fielding McConkie and Robert L. Millet, Doctrinal Commentary on the Book of Mormon, Vol. 1, p. 69]