Is it wrong to not be with someone because they don't have the same sexual preferences as you?

I'm thinking of dating this guy again who I casually dated for a little over a month about 2 years ago. I really liked him but at the time he wasn't ready for commitment. We have never had sex before and the topic came up. He told me he's not into kinky things like handcuffs, whips, and blindfolds but I am. I know sex isn't everything but it is a part of a relationship. As the saying goes, try the cow before you buy it. Should I just walk the other way?