Justice Secretary David Lammy has ordered an independent probe into how a failed asylum seeker who sexually assaulted a 14-year-old girl was mistakenly freed from jail.
Ethiopian national Hadush Kebatu was arrested on Sunday following a two-day manhunt after he was wrongly released from HMP Chelmsford instead of being sent to an immigration detention centre.
Mr Lammy told MPs that Kebatu would be deported back to Ethiopia imminently. But he said: "This has not changed the fact that Mr Kebatu's victims are rightly outraged about what has happened, and I am livid on their behalf and on behalf of the public.
"This was a mistake which should not have happened. The victims expect better. The public expects better, and this Government expects better from a critical public service which plays a vital role in our first duty to keep the British people safe and from harm."
Kebatu was released due to "human error", Mr Lammy said. But he warned there must be accountability for the blunder.
He told MPs: "I've been clear from the outset that a mistake of this nature is unacceptable. We must get to the bottom of what happened and take immediate action to try and prevent similar releases in error to protect the public from harm."
The Justice Secretary said he had tasked Dame Lynn Owens, a former deputy commissioner of the Metropolitan Police, with investigating how he was mistakenly released. She is expected to speak to Kebatu's victims to understand the impact of his release on them.
Beefed up checks would imposed on all prisoner releases after an urgent review of processes by the boss of His Majesty's Prison and Probation Service (HMPPS).
There will now be more senior oversight on ensuring checks are correctly applied, including a clear checklist for governors to determine that every step has been followed the evening before any release takes place.
Any foreign nationals being deported will now only be released with a duty governor present - and removals from HMP Chelmsford have been suspended this week.
Mistaken releases have been rising year on year since 2021 - going from nine per month on average in 2023, to 17 per month in the period spanning January to June 2024, the Justice Secretary told MPs.
In fiery scenes, Mr Lammy clashed with his Tory opponent Robert Jenrick, as he sought to heap blame on the Conservatives for leaving behind a crumbling justice system.
He said prisons have been left reeling by swingeing cuts, with funding slashed by 24% between 2010 and 2015. Staffing has been cut by 30%, and more than half of frontline prison officers now have less than five years' experience, he said.
Mr Lammy said: "It's little wonder when the system has been brought to its knees that errors like this happen. We must also be honest that the previous government's approach to this crisis, piecemeal, complex emergency releases in the hope that the system wouldn't collapse has added to a level of complexity and pressure that makes errors more likely."
But Mr Jenrick, the Shadow Justice Secretary, branded the scenes from the weekend were an "absolute farce", adding: "There's no point coming to the House professing to be living at the consequences of your own policies."
A furious Mr Lammy said he had a "brass neck" for failing to take the blame for the mess the Tories made of the justice system.
It comes amid warnings about the impact of a visa crackdown on prisons that are struggling to recruit staff.
Paul Crossey, governor of HMP Feltham, told The Mirror: "One of the things that I'm very concerned about is the recent changes for prison officers about visas. I have a high proportion of foreign national officers who work here.
"They're hardworking public servants. They're very worried because they come over with a skilled worker visa and the changes made in July means they won't be able to stay. I've had to unfortunately terminate people that haven't got visas and no longer have a right to work and 30 days later they end up being deported."
He added: "I've got to the point now at Feltham where historically we've found it really difficult to retain staff, I've got to the point now where the staffing position is looking quite good... the visa situation could really derail this."
Justice Minister Jake Richards said his department is in talks with the Home Office on the issue.
A Government spokesperson said: “We are clear that net migration must come down. Under current arrangements, prison officers already working in the UK on a Skilled Worker visa will be able to remain in post and apply to extend their visa when it’s due.”
“We are exploring options to support prison staffing, including efforts to recruit officers already settled in the UK.”