Under the extremes of libertarianism the logic for why slavery would not happen isn't that "it wouldn't be allowed", remember, they view a government system as bad, there's not strictly a government to enforce a lack of slavery.
The extreme libertarian position is that the market will self regulate moral bads, so slavery would only be disallowed inasmuch as it was uneconomical to forcefully enslave people. This, under their reasoning, might be true because you're under contract with a security company who keeps you from getting enslaved, among other services, and will actively go to corporate war to protect the sanctity of their contracts for fear of losing business in the future.
This is obviously a fantasy.
Libertarians generally have no qualms with slavery, not in a strict sense. Some libertarians certainly dislike it, but don't have a strict philosophical backing for why it wouldn't be allowed under true zero government systems.