I am so sorry for the situation that you are living with in this matter. I am an advocate in a verbal sense for the children more that anything else in any family setting.
With that said, I personally believe that there are 3 times in a marriage when divorce is the right thing to do. Abuse (of any type), Addiction (not just drugs and alcohol), and Adultery. In your case you have the later for sure.
Please don't buy into his story of the so called date rape drug prospect. He still allowed himself to be in a position that kept him in close proximity to another female, and in that respect he was looking for something.
As far as cohabitating with your husband, for the sake of the children having thier father in the same home, it is possible. You need to set ground rules with your husband to ensure that the children are not completely aware of why Mommy and Daddy are not sleeping in the same room any longer. You also need to make sure that neither you nor your husband are ever talking to a perspective "date" or companion in front of each other, and more importantly, never in front of the children.
As far as forgiving him for cheating, you really shouldn't. If he does it once, there is a high probability that he will do it again. Just a fact in life.
You already mention that you are fearful about bringing up the affair "when you argue", and that statement assures that you will hold onto that hurt for a long time. That is your right in life, but if you reflect that in front of the children, they will be hurt. Protect the children above all else! Your love life as well as that of your husbands will need to take deeper back seats in life to show some type of "normalcy". It will not be an easy balancing act for you or for your husband.
You wont hurt the kids more with a "co-existance" situation as long as you and he can and do remain totally civil and respectful to each other as well as the sanctity of the family image that you are trying to show to the children. You will possibly be hurt more, but that will depend on the ability of you and he to act like adults in life.
I wish you the very best outcome in this.