The exception verifies the rule. With humans, biologically, there are two healthy sex phenotypes. Everything else is a) extremely rare and b) the result of something going wrong.
Also, the fact that the sex/gender split in meaning was invented in the english language and many languages only have one word for it really confuses things. My other language, Swedish, only has "kön" for humans, which translates as sex. The Swedish words for sex are used both in instances where male/female would be used, and in the instances where man/woman would be used.