I'm an atheist, so follow no religion, but for me, religion is (usually) a beautiful thing. It enriches peoples lives, gives them a structure for their lives, teaches them morals to live by and so on. I'd never advocate eradicating religion from the world because for a lot of people it's a positive influence throughout their life.
However, I do think religion should only be dealt with on an individual scale, rather than having it implemented onto a large section of the public, such as having a state/national religion. When religion infringes on peoples rights, that's when I personally believe it becomes a damaging factor. For example the woman
who was denied an abortion in Ireland because abortion is still illegal in Ireland due to it being a Catholic country. Or the continued debate about gay marriage and whether it should be allowed in case it offends. I think that is when religion causes issues. When people read the Bible (or other Holy Books) literally, and attempt to live in accordance to every single law, then it could be problematic and damage the religion.
I think it was Kant who said that without religion, the moral order of society would collapse. I disagree. I think in a modern society, people can distinguish between right and wrong, moral and immoral without having a religious knowledge of such concepts. For example: in a society where homosexuality is more open and discussed, I think it's wrong for religion to still be an influencing factor on how they are treated by law and in society. Many of the ideas found in books like the Bible or the Qur'an need to be understood in the context of the time in which they were written, before being implemented onto the rest of the public. Like Dru says, when zealots attempt to scare people with their faith, that's when religion becomes a problem. Having faith isn't a problem, but ascribing to a religion which may have such extreme views can be a problem.
But that's just my jumbled opinion.