"Huh, that actually makes a lot of sense. I think I've heard that it's also common for siblings who were separated at birth to be attracted to each other after reuniting, so that would go to support that theory."
'Genetic sexual attraction' is the term for that. Siblings and other close relatives like parent-child.
The evolutionary theory is that it's good to mate with people similar to you for compatibility, but not too similar for recessive mutation reasons. So we like people similar to us, blocking out the people we grow up with since those are probably our relatives. Whether the similarity/familiarity are measured by appearance or smell, I dunno.
Of course, immune system diversity could favor getting some exotic genes.