You are right, but not dealing with the current system of tax and inheritance laws. States have existing income and inheritance tax laws that favor unions between heterosexual partners, those same unions should be extended to any consenting adults. In this case, state law does have the need to recognize such unions for tax and inheritance benefits to be applied.
In an ideal situation with the absence of any such benefits, you would be correct. However, even under that scenario states would still be involved with enforcing such contacts through courts unless you lived in a stateless society with competing contract arbitrators.