Hello Super 55,
That stretch is difficult and I have a hard time too, but it can be done with time and a lot of effort on practicing
stretching. I imagine Ben does have longer fingers, I think he once said he's around 6'2" or 6'4", can't remember for sure. That really isn't the issue though. Being able to stretch your fingers is what matters and that takes time, patience and hours of practice. Steve Kaufman has short stubby fingers and when someone complains that they can't reach certain frets, he holds up his hand and asks you to compare your hand to his. I know this because it happened to me. My fingers were significantly longer than his and I'm just average.
If you watch the A part video closely at 7:14 where he goes through it slow, at the exact spot you're speaking of (7:20), it looks like he keeps the third fret fretted while going to the seventh fret with his pinky, but then lifts before going to the fifth fret, but he holds it down long enough to get the sustain needed to make it sound right. I wouldn't worry too much about lifting a little bit, but do keep it fretted long enough to get the right sustain.
Hope this helps and Happy Thanksgiving,