Please specify which society. No two worlds are the same.
Is it still a social taboo for men to wear panties in our society?
Why worry about it. Unless you drop your trousers, no one will know anyway. And if you do drop the trousers, they should just be grateful you aren't going commando.
Yes.