In the previous chapter, Isaiah had noted that all had gone astray and pointed specifically at the house of Jacob. At the end of the chapter, Isaiah chided them for following after the trappings of men. That falling away from Jehovah’s law has consequences. In the previous verses, the Lord told Israel that they would want to hide in caves.
In this chapter he continues to describe fallen Israel, symbolized by fallen Jerusalem. These four verses begin to describe how their humiliation will be complete. When we remember that Isaiah is the prophet during the Assyrian invasion, it is easy to see these consequences as fulfilled prophecy. The right of rule was taken from Jerusalem and many of the best of the country of Judah were taken away, similar to the whole of the northern kingdom.
The ”babes shall rule over them” phrase is not meant to be literal, but rather a description of their fallen state when foreigners, whom they did not respect, would have authority over them.