Notice how quickly Benjamin sets up teachers to keep the people in remembrance of their covenant. As part of the covenant, the people had promised to be obedient for the rest of their lives (Mosiah 5:5). Benjamin didn't want to give them any time to forget what they had promised to remember. This demonstrates Benjamin's understanding of human nature—that we are quick to promise the Lord big things, but that we are quicker to forget what we promised.
The use of the word, "oath" is interesting because it demonstrates that Benjamin's people had entered into an oath and a covenant. An oath is half of a covenant. It represents the promise of one party instead of the contract between two parties. In this case, Benjamin's people promise of to be obedient to the Lord for the rest of their lives. This is the oath that they are to remember.