It is abundantly clear that even after eight years Laman and Lemuel cannot become reconciled to their departure from the land of their inheritance. With no understanding of the greater purposes of the Lord, with no ability to fall back on their faith in the overarching purposes of the Almighty, Laman and Lemuel are left with a secular view of the world in which it is painfully clear that living in relative ease in Jerusalem was better than eight years of difficult travel in the wilderness.
Though the specific speaker is not mentioned, the sentiments clearly belong to both Laman and Lemuel. The telling phrase is "we might have been happy." It is certain that they were not happy, and their state is an important contrast to Nephi, who, while undergoing the same hardships, would nevertheless be likely to proclaim that he was happy. Because Laman and Lemuel's happiness was based on physical comforts, and not the comfort of the Spirit, they were unable to experience the joys of communion that Nephi had, experiences which certainly brought him more true joy than he had ever previously known.