The main issue for me is that many Rothbardians (at least, 95% of the self-identified "Rothbardians" on Facebook) see the State as the only source of oppression that people face, and so all their arguments will typically reject the notion that any group of people could have some sort of common struggle in favor of "Well, everyone is being oppressed by the State." I think this is a largely myopic worldview. My problem isn't so much with Rothbardianism itself as with what people do with it. If Rothbardianism is supposed to be a set of neutral legal ethics, then why do so many people use it to make normative statements about what other people "should" feel oppressed by?
For example, the argument "Let's not recognize gay marriage, let's get the government out of gay marriage" simply doesn't have much to offer people, simply because people already enjoying the benefits of state-recognized heterosexual marriage aren't exactly clamoring to give up those benefits. Women who aren't happy with their treatment in society aren't going to feel as though the state is the only thing making their lives harder. If it is "leftist" of me to feel as though some rightist libertarians are either unwilling or unable to come to terms with the fact that women do in fact feel this way, then so be it.
I just think they are going to have a hard time growing their numbers if they continue to gloss over certain "undesirable" aspects of society by treating them as arising purely from "voluntary" relationships, or denying that a conflict exists at all.