Yesterday I put forth the argument that ethical libertarianism requires moving us towards a level playing field. I used the example of school funding, but it can (and should) be extended to things like:
- Universal Healthcare - because health conditions directly impact economic and academic success
- Eliminating inheritances - This should be obvious; any supposed libertarian who is in favor of inheritances is a hypocrite
- Establishing universal after-school programs - hopefully for equally obvious reasons
It also means that we have to address the historical and social reasons that the playing field is so uneven now - even if we are not personally responsible for it. Haiti is a perfect example. Regardless of the corruption of past Haitian governments, the financial obligations the United States (via the IMF, etc.) placed on that country are currently crippling it.
Likewise, we can look at our own history of slavery and racism; the results of that long, long mistake are still visible in the current economic landscape. The school example I gave yesterday is actually part of that equation. During the period of white flight (which isn't over, I should add), property values plummeted as those with money left. 
This wasn't - and isn't - due to anything inherent to people of color moving into an area. It's because of the racism of residents who view (note the present tense) the neighborhood as being less valuable because people of color moved in. As property values dropped, more "undesirable" people could afford the houses and moved in - and the property values dropped more. This altered the funding of the local schools - thus hampering the children of those who lived there. People who had the money to move away (again, at the beginning of this cycle that meant affluent white people) continued to do so, and suddenly you have a very unequal distribution of wealth, and a very uneven playing field.
So again, to ethically and morally hold to the tenets of libertarianism, we must
- acknowledge the historical processes that caused this inequality, even if we're not personally responsible
- actively work to reduce those inequalities so that we can get to a level playing field
 I am again simplifying for the sake of clarity.
 Because it's not just people of color. One can make the argument that (in my region of the country at least) that many white poor folks are from ethnic groups that were marginalized during the waves of immigration at the beginning of the 20th century. They can be ostracized equally effectively.