I know that 5'10", 185 pounds is technically overweight, but I'll bet you actually don't look that pudgy. (Speaking as someone who is six inches shorter and, cough-cough, has a few pounds on you. And whose husband hasn't had, um, problems with the fat issue in the bedroom.)
I don't think the not having sex is about the weight per se. He might prefer you a little thinner, but what he said strikes me as the kind of thing someone might toss out in the heat of anger.
Sex can be kind of a thermometer for the health of a marriage. There are lots of reasons couples quit having sex - physical problems, control issues or stress in the marriage, getting bored. You may be able to figure out some of what's going on in your particular situation if you think back to what's happened in the last few years. For example, did the sex sort of taper off in quality and quantity? Are the two of you having money or kid issues? What kinds of things are hot buttons for the two of you other than sex?
A frank, non-accusatory discussion with your husband about your marriage might be in order. A good place to start might be seeing a therapist. Often men are leery of seeing therapists because they don't want to be ganged up on by their wife and the therapist, or they don't want to be dunned with complaints. They also tend to not enjoy discussions about "the relationship". But see if you can get him to go - explain to him that you really just want the marriage to work, and this isn't so much about who's right or wrong.
If he won't, try to go yourself. The therapist may have some ideas for things you can do or try yourself, or some idea about what's really going on. And yes, the therapist can also help you work on feeling good about yourself again, which is incredibly important - not only do you deserve that, but if you don't feel good about yourself, it's going to be hard to do anything positive about your marriage.
Good luck!