The disappearance of their daughters obviously affected the Lamanites. They would have understood that they were taken because the alternative would have been some attack by wild beasts. That, however, would have left traces and probably not resulted in all of the daughters vanishing. The clear conclusion was that they were taken, and the Lamanites blamed the obvious enemy, the people of Limhi.
Whatever the conditions of the peace that had been established, this was certainly a breach of that peace, and therefore the Lamanites came with armies. Certainly they intended to recover their daughters, but their anger also dictated that they enact revenge, hence the desire to destroy the people of Limhi.