My Brother and sister-in-law had a child who died soon after birth. He had Fraser Syndrome, which is extremely rare and the genes usually occur in Romany Gypsys or when 2 people who are related and have the same gene defect have a child together. As my brother and sister-in-law were not related, this seems to support the story about my maternal grandmother's affair.
There are other complications; I have found a record on Family Search.org that indicates my grandmother married a man in 1929, about two weeks before my mother was born and that she married again in 1931 - to the man who would father my mother's siblings. I cannot find any other record of the original marriage.
Part of the story we heard about my grandmother was that she was jilted at the altar. Was that in 1929 and the marriage didn't actually go ahead, or is it that they married and he left shortly after?
In looking into Gypsy/Romany culture I have been struck by the continuing discrimination against them, even today. There's a wonderful video on Vimeo about 3 Gypsys; one who works as a political leader in Germany trying to change the discrimination against Gypsys by the state (yes, it continues even after everything that happened in the Second World War), one who works as an unpaid teacher in Romania, and a traditional storyteller in Hungary. There is no documentary, just film following these individuals lives and providing insight into their culture, traditions and their daily realities.