The four brothers approach the city that they understand to be the city with the people who left Zarahemla. How they know that is not mentioned. The four probably come as a less threatening number than the full sixteen.
They happen to arrive when the king is outside his city. Even though there were only four, they were still surrounded and imprisoned. After two days they came before the king. In Mesoamerican art there are many representations of captives brought before the king, so this aspect of the story rings true. Similarly, the representations show the prisoners bound. Some are also unbound. Once in the presence of the king, the prisoners had no weapons and were certainly surrounded by the king’s guards, so it would have been safe to loosen their bands.