Because they never "knew" the Lord in the sense of accepting him as their redeemer through baptism, the Lord never "knew" them because they were not his, they were not called by his name. The fate of those who do not accept the Lord's redemption is that they are not redeemed.