The branches from the wild olive-tree are brought in to replace the dying top branches. This again is referring to the time when the kingdom of Israel is being destroyed and other nations are brought in to replace them:
’And the king of Assyria brought men from Babylon, and from Cuthah, and from Ava, and from Hamath, and from Sepharvaim, and placed them in the cities of Samaria instead of the children of Israel: and they possessed Samaria, and dwelt in the cities thereof…
Then one of the priests whom they had carried away from Samaria came and dwelt in Beth-el, and taught them how they should fear the Lord. (2 Kings 17:24,28)
These wild branches did not produce fruit right away. For centuries they believed in the Lord and yet continued to worship their own pagan gods. Because of this they were looked down on by their Jewish neighbors to the south. They eventually became “the Samaritans” and practiced a form of Judaism which was mixed with their pagan traditions. It was not until after the ministry of the Savior that they began to blossom, and the Gentiles were grafted in in greater numbers.