Shadow justice secretary Robert Jenrick has defended remarks he made in March about the Handsworth area of Birmingham, in which he called it “one of the worst-integrated places” he had ever been to.

In a recording reportedly made during a dinner and published by the Guardian, Jenrick said he had not seen “another white face” in the hour-and-a-half he spent in Handsworth filming a video about litter.

Jenrick said on Tuesday he had no regrets about the language he used and would not “shy away” from issues of integration.

West Midlands mayor Richard Parker said he believed Jenrick’s comments were racist, while Labour Party chair Anna Turley said they reduced people “to the colour of their skin”.

Handsworth’s Independent MP Ayoub Khan said the remarks were “not only wildly false but also incredibly irresponsible”.

Conservative leader Kemi Badenoch said she did not know the context of the recording but that Jenrick may have been “making an observation” about his visit to the area.

“I heard that one of the MPs of that area was accusing him of racism. I completely disagree with that, I want to make that very clear,” she told BBC Breakfast.

“I wasn’t there so I can’t say how many faces he saw, but the point is that there are many people in our country who are not integrating,” she said.

The authenticity of the recording at the Aldridge-Brownhills Conservative Association is not disputed by Jenrick’s team.

In the recording, he goes on to say: “That’s not the kind of country I want to live in. I want to live in a country where people are properly integrated.

“It’s not about the colour of your skin, or your faith, of course it isn’t. But I want people to be living alongside each other, not parallel lives.”

Asked on BBC Radio 5 Live on Tuesday whether he regretted the comments made in the recording, Jenrick said: “No, not at all, and I won’t shy away from these issues.”

He said he brought up skin colour “because it’s incredibly important we have a fully integrated society”.

“It’s a very dangerous place if we have a country where people are living in ghettoised communities, where people are not living together side-by-side in harmonious communities.

“We’ve seen the damage that can do in our society,” he said.

“We’ve had major failures of integration in this country for my whole lifetime.

“We’ve got to fix it, and that’s the comment I was making in Birmingham the other day.”

Labour’s West Midlands mayor Parker said he was “appalled” and “disgusted” by Jenrick’s comments made in the recording.

Asked if he believed the remarks were racist, he told BBC Radio WM: “I do. Because he’s set out intentionally to draw on a particular issue – people’s colour – to identify the point he wanted to make.”

He added: “It shows a lack of respect and understanding for those communities.

“And I doubt whether or not if he went to a largely white community anywhere in the West Midlands he’d be making a comment similar to what he made about Handsworth.”

Labour’s Turley said: “This weekend Kemi Badenoch said she stood against a politics that ‘reduces people to categories and then pits them against each other’.

“Robert Jenrick in his leaked comments reduces people to the colour of their skin and judges his own level of comfort by whether there are other white faces around.

“People of colour should not have to justify their Englishness, or their Britishness, or their presence in this country, to Robert Jenrick or anyone else.”

Asked if the number of white people seen in an area is the right measure for integration on BBC Radio 4’s Today programme, Badenoch said: “The right measure for integration is that people don’t care what people look like.”

She added: “I think we should look at these things in the spirit of what was intended, which I believe knowing Rob and hearing him speak, is that he wants, as I do, a country that is well-integrated”.

In a letter to Jenrick, the Bishop of Birmingham, Dr Michael Volland rejected the comments as “entirely wrong” with the potential to “stir up division” and “feed into a harmful narrative that provides fuel for a fire of toxic nationalism”.

Handsworth MP Ayoub Khan told the Guardian that Jenrick had “misrepresented a storied and diverse community, awkwardly distorting the product of an all-out bin strike to fit his culture-warrior narrative filled with far-right cliches”.

Former Conservative mayor of the West Midlands Andy Street told BBC Newsnight Jenrick was “wrong”, saying the area was “a very integrated place”.

Street also rejected Jenrick’s recorded comment that Handsworth was “the closest I’ve come to a slum in this country”.

Street noted the “incredible hope, optimism and people taking part in education which is based around British values and thinking how they can make a contribution to the future of their region, their city and their area”.

On Tuesday morning, the BBC spoke to a number of Handsworth residents who expressed different perspectives about Jenrick’s comments.

Rani Rawji, who works for a business improvement district in the area, said it had Christians, Muslims, Sikhs and Hindus working, living and shopping happily together.

She said: “Everybody’s happy. We don’t seem to have an issue.”

However, Mariaj Khan agreed with the description of the area as a slum and said he was not offended by the MP’s comments because he felt they were true.

Mr Khan said he only saw Asian and black people, adding: “I never saw a white face around here.”

He said that there were white people in the area but they tended not to be British.

The reaction to Jenrick’s comments comes on the day he gave a speech to the Conservative Party conference, in which he hit out at “activist” judges.

The MP claimed judges with links to pro-migrant charities had undermined public trust in the courts.

Additional reporting by Tanya Gupta

Share this content