Speaking for the party, I'll say LP originally brought together a coalition of minarchists, anarchists, constitutionalists, and miscellaneous individualists. The consistency of the non-aggression principle that unifies them has never been in question, just the consistency (or agreement over) its application. Some anarchist Libertarians believe there is no just entity called the state at all, thus there are no real borders, a view that leads to the open borders view of immigration.
Some libertarians think there is no difference between social tolerance (in the movement's support of personal liberty, regardless of controversy) and endorsing social liberalism outright (approving of the behavior, and passing laws to force everybody else to). Some think of the ideology as being domestic only, thus (wrongly) do not tie the non-aggression principle to foreign policy. And so on.
I think some are holding the LP to a higher standard than they do for GOP persons who much more wildly vary in their embrace of 'conservative' policy. The fact is, we were open-minded enough about Barr to once nominate him for President (out of respect for his time spent in our party, and for once being a senior elected official), but he has revealed himself over time to not be a consistent Libertarian. The Kochs at one point appeared to want to drive the party as its controlled vehicle, but they were purged from influence. A stray figure here and there may appear in the party to try to hijack it away from its core agenda, but the agenda wins as a rule.