Are you saying that it is not true that some sperm cells carry only a Y chromosome and other sperm cells carry only an X chromosome, such that at conception, a zygote is typically either XX or XY?
Deliberately obtuse $#@!heads. At conception you are XX or XY. No matter the development process and when the Y gene is activated.
The XX-is-female / XY-is-male paradigm works in something over 98.5% of the population. But nobody really knows how large that remaining percentage is. Genetically, you need the
SRY Gene region in order to develop male physiology. Generally the SRY gene region is found on the Y-Chromosome, but it sometimes (rarely) occurs on the X chromosome. Individuals with XX chromosome containing SRY Genes may develop male physiology, though their sperm count is seldom high enough to procreate. There is also the possibility that the SRY gene may be missing from the Y chromosome, so you could get an XY individual that won't develop male physiology and will instead develop as the default sex physiology (which as idiom pointed out, is female). While these individuals have a uterus and ovaries, the ovaries are seldom capable of generating viable eggs. They can sometimes carry a child to birth by the use of donated eggs and fertility procedures ... ergo, XY individuals giving birth.
There are other disorders of sex development as well (
read here). And it's not all just chromosomes and genes. If the right hormones are not present in sufficient quantities at the appropriate times (as idiom said, sex differentiation occurs at about six weeks) then sexual physiology will not develop in accordance with the XX-is-female/XY-is-male paradigm. The incidence is really low (1 in 80,000) but it's non-zero.
If you know of individuals who are having fertility problems, there's a chance they fall into the realm of people with disorders of sex development.
So if you have an XX individual with a penis and testicles, which bathroom do you want that individual to use (granted, there's probably fewer than 4,000 of them in the US)?
===
I'm happy with the way it's been done since before I was born. The OB/GYN looks at the kid coming out of the chute and says "Male physiology means male child" or "Female physiology means female child" - the old sex assigned at birth paradigm.