This is like an expletive. The above two lines are the venting of his anger. He’d rather be like a pagan, for instance believing in ancient Greek gods celebrating nature, than part of a world that is destroying nature’s beauty and calling itself Christian. He is not saying he doesn’t believe in God. Instead he expresses his anger at the world to God and possibly even at God.