The Most Reverend Justin Welby was speaking at the official opening of the Evangelical Alliance's new headquarters in London's King's Cross.
Progress on racial unity and other issues, he told the audience, had to "begin with repentance" and the recognition that we are all sinners.
"We need to start by acknowledging that we have been - and continue to be - mired in racism in one way or another," he said.
"However, we are a long way from where we need to be. We at the Church of England are quite open about that… We come to a God who isn't remotely interested in our ethnic background.
"The Church spent hundreds and hundreds of years persecuting Jewish people when the founder of our faith was Jewish.
"But God is someone who is utterly indifferent about what ethnic background someone is from. In fact, quite the contrary, He gave His son to break down those barriers."
He said there was a tendency to divide into 'us and them' when people together, but that these differences disappeared in relationship with God.
"That's why my highest priority in my own life and ministry is a renewal of prayer and communities of prayer because it's when we come close to God that we are convicted of these things and we know that we need to repent," he said.
The Archbishop's comments coincided with the 50th anniversary of Martin Luther King's "I have a dream" speech in which the civil rights campaigner called for an end to racial segregation and discrimination.
Archbishop Welby also suggested the Church needed to improve its public image and become better known for what it stands for rather than against.
"We are not against things. We are in favour of things. So the key phrase we remember [from King's speech] is not a condemnation of the racism or the terrible Jim Crow laws that existed at that time but what we remember is the positive: I have a dream."
He continued: "And that is the most extraordinary work of the spirit. Alliances must be seen as for. One of the things that I think is most noticeable where we make a bad impression in society at the moment is when we're seen as against things.
"We come across too easily as negative. We don't intend to, but we do. What is so impressive about what is going on here [at the Evangelical Alliance] is that this is for what the Church is for: It's for foster children, it's for adoption, it's for bringing the Church together as one people. We are an alliance that has to be for."
The official opening of the EA headquarters was also joined by the Reverend Yemi Adedeji, director of the alliance's One People Commission.
"Martin Luther King exactly 50 years ago stood there and said he had a dream; a dream where one day brothers and sisters from everywhere – black and white – will come together and they will eat on the same table and they will share fellowship together and they will be one in Christ," said Rev Adedeji.
"Today, as we open this building the One People Commission wants to share these words with you; that we have a dream.
"And the dream for us as evangelicals in the UK is that in years to come for us there will not be one majority Church but there will be one Church that has all ethnicities together in Christ doing what we have been called to do. Our dream is that we will be Christ-centred and have a Christ majority. That is our passion."