I wouldn’t give one up unless you actually dislike it. There’s not much possibility that splitting your time between two is going to hurt your overall musicianship; if anything, it will most likely help it. You might want to read through these threads, too:
In my experience, playing better banjo helps me play better guitar, because I gain more understanding of the banjo’s role and how to complement that. Playing better guitar helps me play better banjo, because my ear is more familiar with how guitar rhythm lines up with banjo.
If you find that you take to learning one more easily than the other, you may enjoy putting more time into it than the other, which is completely natural. But unless you end up uninterested in one, you probably will not be benefited from stopping.
Great question, I tried to learn the guitar and the mandolin whist learning to play banjo. I loved the banjo much more than I did the guitar and mandolin and decided to give them up in favour of the banjo. Years down the road I truly regret my decision.