To me gender in a game is a non-issue. A player can create and mold their character in whatever image they want within the bounds of the campaign we're playing. So if a man wanted to play a woman, or a woman a man because that was the vision they had for their character, I don't think I would have a problem with it. Unfortunately, this has never come up for me so I'll just have to wander into hypothetical land for a minute here.
If a player desired to play the opposite gender in a game I was running, I think I would immediately ask them, why? If they answered that it was part of the way they saw the character, or they think it would be interesting or it just feels right, then I don't think I would care. If they looked at me all creepy and said, she/he is hot, and then snickered while pressing their hands together, I might refuse... and probably not invite them back to play again.
That's really all I can say. Does gender swapping matter in your game? If you have some experience with this, let me know. I am really interested in hearing about it from both the players and gamemasters perspective. Please comment away.