The Trouble with Mill, On Liberty
I hold this truth to be self-evident: that all men have rights to liberty. These are basic rights, not in need of proof or any justification other than the fact that they are obviously true and axiomatic. There being nothing that would count as evidence that a system of morals could be acceptable in which men did not have rights to liberty, we could not coherently interpret any such system as a system of morals. I think this verificationist argument shows that the proposition that all men have rights to liberty is a logical truth.
Keep in mind that these basic rights are defeasible; basicity does not entail absolute rules. I hesitate to call the rights to liberty "inalienable" for that reason. However, they could be interpreted as inalienable in the sense that even while one may forfeit one's right to liberty, it still remains the case that were the circumstances causing that forfeiture removed one would have that right to liberty.
In any event, Mill's book suffers for committing itself to rule-based moral theory and assuming that the right to liberty needs to be delimited by a rule and proven. It needs to be neither. This is the big problem with the book.
Of course, Mill will think the right to liberty needs proof if he thinks he needs to reduce it to a rule and prove that rule. The extent of the right to liberty is indefinite and not to be delimited by any rule. The variety of moral claims - duties and rights - is not subsumable under a system of rules. There is no finite decision procedure of moral deliberation that has any plausibility.
That Mill is a rule-based theorist commits him to progressivism. Any rule will be progressive, as it will be markedly different from ordinarily-held moral values. For those values are not subsumable under any system of rules. Thus does Mill become incompetent with respect to the need to preserve tradition. He praises not it but progress away from it. He realizes he should say more than this, and he says, "Nobody denies that people should be so taught and trained in youth as to know and benefit by the ascertained results of human experience." But he isn't able to develop this point. (Hayek was able to explain it. Although a utilitarian, he wasn't ensconced in rule-based moral theory as Mill was.)
The result is that Mill's liberty teeters unstably, with its progressivism and dubious basis in utilitarianism. The book has many an eloquent passage defending liberty of speech, thought, and way of life on the basis that these promote utility. Of course this is a reasonable argument to make. But that it is the foundation of the right to liberty is a mistake. We have that right even without those benefits.