Until recently, scientists believed that a major difference between men’s and women’s breasts is that men’s breasts do not have the capacity to develop lobules and milk ducts. These are the structures that make it possible for women to nurse.
However, male-to-female transexuals who are taking a certain combination of drugs have been able to produce copious amounts of breast milk and have successfully breast feed babies. So while drugs alone can’t change a woman’s clitoris into a penis, the right combination of progestin, estrogen and domperidone can change a man’s breasts into fully functional female breasts.
Also, women who have a condition called AIS (where they have male pattern XY chromosomes instead of XX chromosomes) have a good chance of being able to nurse a baby if they take domperidone. So the potential to develop fully functional lobes and ducts seems to be there regardless of a person’s sex.