I am so sorry to hear of your struggle! It must feel very lonely without your children in this situation.
I have 2 kids, 4 and 9, and my situation is somewhat similar, except my husband wants ***, but when he wants it, it's like midnight, and I am NOT up for it. He's too busy for me or the kids. He expects me to take care of the kids all the time, and when I need to leave them at home with him, it's called "babysitting."
I don't really have advice for you. I'm in a struggle of my own (see my topic -- Soulmate Confusion). I just wanted you to know that you have my sympathy. Best of luck to you!