There are a number of alternative ways to express desire-clauses in the Book of Mormon. First, the following clause can be either a that-clause or an infinitive clause (which begins with to). Second, the verb desire can be directly followed by a noun phrase (usually a pronoun), by a prepositional phrase beginning with of, or by no complement at all. This gives six possibilities, with varying frequency in the text:
(1) to desire of X that X do something (5 examples):
(2) to desire of X to do something (1 example):
(3) to desire X that X do something (11 examples, including 2 Nephi 6:4):
(4) to desire X to do something (4 examples):
(5) to desire that X do something (41 examples, excluding 2 Nephi 6:4):
(6) desire to do something (35 examples):
Interestingly, Joseph Smith changed the structure in only one of the examples listed under (3). In other words, he did not delete the noun phrase complement in ten parallel examples, all equally non-English in style (at least from the point of view of speakers of modern English).
The only difference between these ten examples and the one in 2 Nephi 6:4 is that the pronoun in 2 Nephi 6:4 is the first person me rather than the third person him or them.
Summary: Restore in 2 Nephi 6:4 the original reading (“my brother hath desired me that I should speak unto you”); Joseph Smith removed the pronominal complement in only this example, leaving ten others unchanged.