It's always interesting the ways the gender stereotypes never quite carry across cultures properly.
The association between sweets and femininity certainly exists in the west, but in Japan, my understanding is that masculine people are expected to actively dislike sweets. In the west, a hypermasculine person being fond of sweets would be seen as quirky, maybe eccentric, but not contradictory. Of course the west has it's own arbitrary gender norms so glass houses and all that