But if one US pastor has his way, that could all be about to change – for the men at least.
In the eyes of Pastor Ed Young, Christians should be setting a good example to the world not only in the way they live but also in the way they dress.
He's launched a new blog, Pastor Fashion, dishing out tips to men on everything from different tie knots to picking out the right shirt.
"Pastors aren't typically known for their fashion," he writes.
"But why not?! Why can't the men and women of God set the standard for the rest of the world in fashion as well as faith?
"We're not trying to be like the world … we just want to be fashionable while we try to change it!"
Some of the posts offer helpful fashion advice with a dose of humour.
In one post, he warns pastors about buying suits that are too big.
"If you're too comfortable in a suit, it probably doesn't fit. Suits are not sweat suits."
He adds: "T-shirts should be white or grey and not overly long. You are not a Hillsong Worship Leader."
Videos uploaded to the blog show Pastor Young talk men through shirt collars, skinny jeans and getting their turn-ups just right.
To see more advice from the Dallas-based pastor, go to www.pastorfashion.com