The withdrawal of the Lord’s spirit is from the people as a whole, not from each individual person. Even though the spirit of the Lord has withdrawn, it is yet available to righteous individuals. Mormon was one of those, and he tells us that while the spirit may not have been present for the benefit of the people of Nephi as a whole, it was nevertheless important in his life.
Mormon tells us that he was “visited of the Lord.” This might be phrase that actually refers to the spirit, but there is no reason to be so limited in our reading. Mormon deserves that we take him at his word, and understand that he actually was visited “of the Lord.” Mormon declares himself a special witness of the Savior, so this is the most likely reading of the passage. (It is the reading understood by Ludlow. Daniel H. Ludlow, A Companion to Your Study of the Book of Mormon [Salt Lake City: Deseret Book Co., 1976], 299.) In this connection we find yet another parallel to Joseph Smith, who also had a visitation from the Savior by the time he was fifteen.