If a property owner doesn't have such an unconditional right, then he is not really a property owner. The authority to include or exclude is fundamental to property rights, just as the liberty of (dis)association is fundamental to secession rights (in fact, those are really just manifestations of the same principle).
The problem of "imprisonment" is a possibility, of course - but I suspect it's likely to be a transient edge case that market incentives will find a way to resolve in one way or another (such as by your suggestion of paid escorts or the like). And when there are degenerate cases where that doesn't happen for whatever reason (such as the absence of any adequately functional market system), then (as always) either submission or forceful violence is the ultimate resort. In situations in which some agents (such as Israel, for example) are hell-bound and determined beyond all reason or incentives to the contrary to imprison (exterminate ?) others, then from where else shall the authority to enforce "travel with conditions" come?