Best thing to do hear is Clearly And Calmly Explain to him How you Feel And Why You Feel the Way that you do.
Hun this is precisley Why i recommend to people asking about this swinging or threesome and things like that to Stay Away from it . In a marraige or not.
But especially in a Marriage. Because This is Precisley What it does,. It Destroys Marriages.
How do you know what he will do until you talk to him.
Sit him Down Stay Calm And Relaxed and Talk to him Openly and Honestly about how you feel and why you feel the way that you do.
No I don't think you will need to get a divorce. But You will Need to Do some Marriage Counselling and Just Work It out together
I know your gonna hear this from a lot of people including me but this is a door neither of you should have opened to start with
Now is the time To Repair the Damage This has Done . Not Create More
You guys can get through this if you really want to it's gonna take time and a lot of hard work i'm not going to lie to you but it can be done
Good Luck Girl Hope this helps