We don't get rid of civil governments because they are righteous and necessary institutions for civil liberties, as ordained by God, the Giver of our rights. Governments are not evil in and of themselves; it's the people who are involved in those positions of government which hold the helm that makes it a good government or a bad government.
Once again, those blessings are only blessings when they are used for the glory of God and the good of other men. The misuse of those blessings is a curse, and it is a curse which is taken advantage of by people who have corrupt hearts and sinister minds, both towards God and their neighbor.
What does anarchy bring? It ensures that "might makes right," morality and law are subjected to relative standards by the whims and wits of each man (be it for fair or for foul), and no arena for appeals and universal justice, to name a few. No, we don't need to get rid of the State, but we do need to limit the State's role in human affairs. Anarchy is not the solution; it's just tyranny on the other side of totalitarianism.