Many like us ['social libertarians'] feel the poor deserve a right to have a chance to do the same things that those better off do.
For a brief time in my life, I believed as you do. That time coincided with my having been plunged precipitously into unexpected poverty. Didn't take long, however, to recover my original political principles, if not my former middle-class existence.
I remain a member of "the poor" and am now Coordinator of a national movement in Canada of low-income people who work to provide support and self-help to one another. We do not see ourselves as "the poor", "the needy", "the less fortunate", "the vulnerable", ad nauseam. Rather, we see ourselves as no less than individuals and human beings with something to contribute. We still have, e.g., our skills, intelligence, fortitude that we've always had.
Now when it comes to rights and whether we have a right to claim a basic income, minimum wage, health insurance, ..., we recognize that claiming what ethicists and philosophers call "positive rights" via government intervention means forcing others to support us and, ultimately, infringing upon their "negative rights" (those rights which we all have, without reference to manmade law or edict).
Just as we resent being referred to as the needy, the poor, etc., we reject the notion that we have any right to require people to help us. We are, in other words, as emphatically independent as anyone else, be they rich or poor.
Of course, I can speak only on behalf of the low-income people who are connected through my organization. But there are far more of us who think this way than may be supposed. It can be difficult, however, to combat the victimology and dependence that the nanny state encourages - which makes our ability to organize and connect up all the more important.