There is a law in many states about no campaign materials within x number of feet from the ballot boxes. If you wear a shirt, you will be violating that law most likely, assuming you have one in your state.
Besides, I wouldn't wear it while voting anyway - it gives them the ability to know who you voted for and toss out your ballot!