When it comes to Huqooq ul Ibad, such as Zakat and dealings with other people in an Islamic society, it becomes society's matter.
There is this third category.. when in an Islamic society, someone starts corrupting the Islamic teachings, then it becomes even more condemn-able.. such as if someone says that polygamy is Haram until polyandry is allowed.. or something of this sort such as men and women come out in open in indecent dresses.. then it comes under corruption of Islamic society, which is to be stopped forcefully if required.. Why? because when a man or a woman wears something that he or she thinks is fine, but a general population is offended, then it must be stopped forcefully..
Imagine if a man roams on a street in his underwear and he thinks that he is providing enough covering, and my children play in that area, or I see a man staring at women in my neighborhood indecently, I will immediately go to the police to register a complain against him.
In an Islamic society, dress code is not what majority has decided for itself, but it is what Allah and His Prophet (PBUH) has ordered us to wear...