What Ben said is what has worked for me. Slow it down as slow as you have to in order to sing and play at the same time, and then just practice that.
In my mind, adding vocals is like adding a third hand to the banjo. Your two hands have to be trained to work with each other to produce specific notes, and that takes time. Now you’re adding another note to the mix with your voice, and the timing of that has to be in sync with your hands, which is why I think of it as a third hand.