Hello. I'm a guy, so I'm not sure my opinion counts. Not trying to be sexist, but really, gals have a different set of problems than most men. Not saying we don't have our share, either.
That aside. I'd like for you to do an experiment. Break it off with him. Don't tell him. Don't fret. Just stop. It may just be that he likes you and wants to continue with you for old times' sake. It may be that you're the only girl, but that's not likely. No offense, ladies, or gentlemen who disagree, but most men don't always stick to one girl.
That's not to say he's cheating on you, but, really, there's a lot of women out there, and most guys don't think too well with certain body parts. I think we may all agree on that.
Another thing. You seem to like him a lot. If not, you wouldn't be an emotional train wreck. Also, you're reaching out for advice from a large community of people you do not know. Take whatever we say with a grain of salt, sugar, spice, or whatever you like to take things with. You are you, and you are the one reaping the benefits. So, if you feel you'd get more sugar from plowing the same field, or from trying the one across the way. Go for it. Just consider that the field you've been plowing might yield better results for the time spent caring for it.
A third(?) option. Talk to him about the other women. Not in a complaining tone, or vindictive, or victim-voice. Ask him how he thinks about these women. Really. Get his opinion as a friend, not as an emotional train wreck. If you think he's not telling the truth, don't be quick to point blame, or, in the other case, break down in tears. Let him know it's important you know what's going on. Let him know your thoughts. And don't let him sweet-talk you into thinking it's ok. Remember one thing; the more frosting put on the cake, the less delicious the substance of the cake will be. Trust me. I know. I've been that guy before.
It could be that he's feeling your feelings, and he's wary of what's going on. Culturally, he may not even realize how you feel. I know it's a long shot, but it very well could be.
So. Take this advice with a grain. And have my blessings on whatever path you choose.