The reason Jacob did not simply ignore Sherem’s slander is that it was Jacob’s obligation to answer. Under this legal system, once a formal accusation had been raised, silence or failure to respond was a confession of guilt. Whereas we have the right to remain silent under our law today, they did not. So Jacob spoke up boldly, "having the spirit of the Lord insomuch that Jacob did confound him in all his words." Such a protestation of innocence could be transformed into a legal accusation against the accuser, raising a counterclaim of some kind, and indeed, that was the effect of what Jacob said in his reply.
Jacob, responding to Sherem’s sentiments ("How can you say these things? Your ideas are confused"), rebuffed Sherem with scriptures regarding the coming of the Messiah and withstood him with contrary testimony. Jacob may have even responded to Sherem with an oath. If a party to a lawsuit swore an oath by God, that was the most powerful piece of evidence that a person could bring before a court. In Neo-Babylonian texts, after the Jews were taken captive to Babylon, we notice that the Babylonian legal system shifted from what we might call religious trials to more evidentiary-based trials. People no longer used ordeals, consulted the gods, or required people to take oaths before the rising of the sun, and so on, as they had in previous centuries. What they now asked for was documents, witnesses, and evidence. But that legal development had not yet occurred in the world that Lehi and Nephi and Jacob would have known. Their trials would likely have still involved putting more weight on oaths and ordeals to show the will of God than some kind of logical argumentation.
In the ancient world these oaths would invoke curses upon the oathmaker, such as, "If I am not telling the truth, then may I die of this awful disease, or may all my crops fail, or may all my animals die, etc." Sometimes they would even take a little animal, cut the animal’s throat, and say, "If I’m not telling the truth, then may I die like this animal has died, may the gods make this happen to me." In Alma 46:21, when Captain Moroni gathered the troops, they all ripped their coats, and started stomping on the coats as a symbol of what would happen to them if they did not fight. They were making their covenant of allegiance and loyalty to Moroni with that same kind of symbolic action. They took these oaths very, very seriously.
But do we have any evidence that Jacob swore an oath? It may be found in the word truly. "They [the scriptures] truly testify of Christ." The word truly is like the word verily, which is the word amen. When you used that word, you were often doing so in some kind of oath-swearing context. Maybe Jacob went that far. He was saying something like, "How can I counter your accusations except by my taking an oath." That would then shift the burden of proof back to the other person, because the one who has taken the oath was presumed to be telling the truth. It was a very drastic level of testimony, because if they were wrong, they were not only causing themselves harm but they were offending God himself.